Lots of technical advice at http://www.adager.com/TechnicalPapers.html -----Original Message----- From: HP-3000 Systems Discussion [mailto:[log in to unmask]] On Behalf Of Denys Beauchemin Sent: 24 September 2004 16:41 To: [log in to unmask] Subject: Re: [HP3000-L] IMAGE Guru needed That's a good thought Walter. The one that was coming to my mind when I read the original message was that the master dataset in question was using a non-hashing key. This would be an explanation as to why the performance dropped so suddenly. The sordid :-) chain is also very applicable, but seems to fall down after Friedrich posted that he had fixed the problem by virtue of expanding a master dataset. Since that fixed the performance problem, I would suggest that sorted chains are not an issue. However, non-hashing keys are definitely something I would look at especially since the drop in performance was reported as being quite abrupt. If it was just the matter of a master set filling up, the performance would decrease, but at a gradual, yet ever-accelerating rate. I do not think it would be abrupt. That is why I suggested Friedrich post the specifications of the master dataset in question. I wanted to see what type of key is in use. Denys -----Original Message----- From: HP-3000 Systems Discussion [mailto:[log in to unmask]] On Behalf Of Walter Murray Sent: Thursday, September 23, 2004 11:48 PM To: [log in to unmask] Subject: Re: [HP3000-L] IMAGE Guru needed "Friedrich Harasleben" <[log in to unmask]> wrote: > A batch job which used to run for about 30-45 minutes (stores data in an > image detail with about 3.200.000 entries) runs now 4 hours and longer > to handle the same amount of data. Others have suggested problems with synonyms in master data sets, but Friedrich says he is adding entries to a large DETAIL data set. In that case, if I experienced a sudden dramatic increase in run time, the first thing I would look for would be very long sorted chains, and a recent change in the item being used as the sort item. When DBPUT inserts a new entry in a sorted chain, it does a backward search beginning at the last entry. That works fine, as long as most new entries have to be inserted near the end of the chain. If a change in the values of the sort items causes new entries to have be inserted near the beginning of the chain, then every DBPUT is going to cause almost the entire chain to be read. For a five-year-old example, suppose your average chain length is 50,000 entries, sorted on date, and you are still using 6-digit dates, format YYMMDD. All goes well, with most new entries being inserted at or near the end of a chain, up until 991231. As soon as 000101 rolls around, performance suffers big-time, with each DBPUT having to do a backward chained read through about 50,000 entries. For a more realistic example, imagine long chains with Purchase Order Number as the sort item. All goes well as long as newer P.O. numbers are higher than old ones. Then you order new pre-printed purchase orders, and the P.O. numbers are rolled back to 000001. Long sorted chains are to be discouraged, of course, but they could explain this kind of sudden change in throughput. Just a thought. Walter * To join/leave the list, search archives, change list settings, * * etc., please visit http://raven.utc.edu/archives/hp3000-l.html * * To join/leave the list, search archives, change list settings, * * etc., please visit http://raven.utc.edu/archives/hp3000-l.html * ______________________________________________________________________ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email ______________________________________________________________________ The contents of this email are confidential to the intended recipient and may not be disclosed. Although it is believed that this email and any attachments are virus free, it is the responsibility of the recipient to confirm this. Smith & Williamson Corporate Finance Limited - A member of M&A International Inc. http://www.mergers.net Registered in England No. 4533970. Authorised and regulated by the Financial Services Authority Smith & Williamson Investment Management Limited, Registered No. 976145. Authorised and regulated by the Financial Services Authority. Smith & Williamson Pension Consultancy Limited - Independent Intermediary. Registered No. 3133226. Authorised and regulated by the Financial Services Authority. Smith & Williamson Fund Administration Limited, Registered No. 1934644. Authorised and regulated by the Financial Services Authority. Smith & Williamson Limited - A member of Nexia International. Registered in England No. 4534022. Regulated by the Institute of Chartered Accountants in England & Wales for a range of investment business activities. Registered Office: No 1 Riding House Street, London W1A 3AS Telephone: 020 7637 5377 http://www.smith.williamson.co.uk Nexia Audit Limited - A member of Nexia International. Registered in England No. 4469576. Registered to carry on audit work and regulated by the Institute of Chartered Accountants in England & Wales for a range of investment business activities. Registered Office: No 1 Riding House Street, London W1A 3AS Telephone: 020 7637 5377 http://www.nexiaaudit.co.uk NCL Investments Limited, Registered No. 1913794. Member of the London Stock Exchange authorised and regulated by the Financial Services Authority. Registered Office: Bartlett House, 9-12 Basinghall Street, London EC2V 5NS Telephone: 020 7600 2801 ______________________________________________________________________ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email ______________________________________________________________________ * To join/leave the list, search archives, change list settings, * * etc., please visit http://raven.utc.edu/archives/hp3000-l.html *