Lots of technical advice at
http://www.adager.com/TechnicalPapers.html
-----Original Message-----
From: HP-3000 Systems Discussion [mailto:[log in to unmask]] On Behalf
Of Denys Beauchemin
Sent: 24 September 2004 16:41
To: [log in to unmask]
Subject: Re: [HP3000-L] IMAGE Guru needed
That's a good thought Walter. The one that was coming to my mind when I
read the original message was that the master dataset in question was using
a non-hashing key. This would be an explanation as to why the performance
dropped so suddenly. The sordid :-) chain is also very applicable, but
seems to fall down after Friedrich posted that he had fixed the problem by
virtue of expanding a master dataset.
Since that fixed the performance problem, I would suggest that sorted chains
are not an issue. However, non-hashing keys are definitely something I
would look at especially since the drop in performance was reported as being
quite abrupt. If it was just the matter of a master set filling up, the
performance would decrease, but at a gradual, yet ever-accelerating rate. I
do not think it would be abrupt.
That is why I suggested Friedrich post the specifications of the master
dataset in question. I wanted to see what type of key is in use.
Denys
-----Original Message-----
From: HP-3000 Systems Discussion [mailto:[log in to unmask]] On Behalf
Of Walter Murray
Sent: Thursday, September 23, 2004 11:48 PM
To: [log in to unmask]
Subject: Re: [HP3000-L] IMAGE Guru needed
"Friedrich Harasleben" <[log in to unmask]> wrote:
> A batch job which used to run for about 30-45 minutes (stores data in
an
> image detail with about 3.200.000 entries) runs now 4 hours and longer
> to handle the same amount of data.
Others have suggested problems with synonyms in master data sets, but
Friedrich says he is adding entries to a large DETAIL data set. In that
case, if I experienced a sudden dramatic increase in run time, the first
thing I would look for would be very long sorted chains, and a recent change
in the item being used as the sort item.
When DBPUT inserts a new entry in a sorted chain, it does a backward search
beginning at the last entry. That works fine, as long as most new entries
have to be inserted near the end of the chain. If a change in the values of
the sort items causes new entries to have be inserted near the beginning of
the chain, then every DBPUT is going to cause almost the entire chain to be
read.
For a five-year-old example, suppose your average chain length is 50,000
entries, sorted on date, and you are still using 6-digit dates, format
YYMMDD. All goes well, with most new entries being inserted at or near the
end of a chain, up until 991231. As soon as 000101 rolls around,
performance suffers big-time, with each DBPUT having to do a backward
chained read through about 50,000 entries.
For a more realistic example, imagine long chains with Purchase Order Number
as the sort item. All goes well as long as newer P.O. numbers are higher
than old ones. Then you order new pre-printed purchase orders, and the P.O.
numbers are rolled back to 000001.
Long sorted chains are to be discouraged, of course, but they could explain
this kind of sudden change in throughput. Just a thought.
Walter
* To join/leave the list, search archives, change list settings, *
* etc., please visit http://raven.utc.edu/archives/hp3000-l.html *
* To join/leave the list, search archives, change list settings, *
* etc., please visit http://raven.utc.edu/archives/hp3000-l.html *
______________________________________________________________________
This email has been scanned by the MessageLabs Email Security System.
For more information please visit http://www.messagelabs.com/email
______________________________________________________________________
The contents of this email are confidential to the intended recipient
and may not be disclosed. Although it is believed that this email and
any attachments are virus free, it is the responsibility of the recipient to confirm this.
Smith & Williamson Corporate Finance Limited - A member of M&A
International Inc. http://www.mergers.net Registered in England No.
4533970. Authorised and regulated by the Financial Services Authority
Smith & Williamson Investment Management Limited, Registered No. 976145. Authorised and regulated by the Financial Services Authority.
Smith & Williamson Pension Consultancy Limited - Independent
Intermediary. Registered No. 3133226. Authorised and regulated by the
Financial Services Authority.
Smith & Williamson Fund Administration Limited, Registered No. 1934644. Authorised and regulated by the Financial Services Authority.
Smith & Williamson Limited - A member of Nexia International.
Registered in England No. 4534022. Regulated by the Institute of
Chartered Accountants in England & Wales for a range of investment
business activities.
Registered Office: No 1 Riding House Street, London W1A 3AS
Telephone: 020 7637 5377 http://www.smith.williamson.co.uk
Nexia Audit Limited - A member of Nexia International. Registered in
England No. 4469576. Registered to carry on audit work and regulated by the Institute of Chartered Accountants in England & Wales for a range of investment business activities.
Registered Office: No 1 Riding House Street, London W1A 3AS
Telephone: 020 7637 5377 http://www.nexiaaudit.co.uk
NCL Investments Limited, Registered No. 1913794.
Member of the London Stock Exchange authorised and regulated by the Financial Services Authority.
Registered Office: Bartlett House, 9-12 Basinghall Street, London EC2V 5NS
Telephone: 020 7600 2801
______________________________________________________________________
This email has been scanned by the MessageLabs Email Security System.
For more information please visit http://www.messagelabs.com/email
______________________________________________________________________
* To join/leave the list, search archives, change list settings, *
* etc., please visit http://raven.utc.edu/archives/hp3000-l.html *
|
