Wirt,

I am not that disagreeable!  But, I am late at getting back to this
thread as you!

My plan is to have our users have access to the data in an easier to
use format than SuprTool.  They will never get COBOL.  I would like to
have windows look and feel apps, therefore not direct HP3000 tools
such as QUIZ.

So, I have created a data warehouse. This allows people to get to the
information in ways they can understand.  Such as Paradox, Excel and
for AppDev FoxPro.

The issue of freshness is really a matter of taste.  I take a weekly
download of data.  That is about 500MB of our data.  This is done
pretty automatically via program control.  A suprtool on the HP3000
box and a FoxPro import table application.  The fact that is a week
old on Friday is not as much of a cost to have reports tie out during
the week.  On a real time system, these numbers would constantly be
changing.

The problem is with the huge files that we can't afford to get full
refreshes every week.  This is why I have brought compression to this
list.  Right know it is an ad hoc where our vendor downloads the file
to their PC's and ZIP the file then upload the file via reflections
to allow us to download it via FTP.  I would like to get a job to
perform the operation to download it our FTP server.

The other reason to download the information to our servers is just
that.  THey are our servers.

Thank you all for your help, I really do like the HP3000!  We may
move to an ODBC interface that may be easier in many respects, but I
have to get my job done as well!

Peter
On  8 Jan 98 at 16:52, WirtAtmar wrote:

> From:          WirtAtmar <[log in to unmask]>
> Date:          Thu, 8 Jan 1998 16:52:34 EST
> To:            [log in to unmask], [log in to unmask]
> Subject:       Re: pkzip compatible software
> Organization:  AOL (http://www.aol.com)

> Peter,
>
> Let me first apologize for being so long getting back to you. As to
> your various comments, let me substantially disagree, ideally
> without becoming disagreeable:
>
> > We have to get the files downloaded.  We have not gotten the support we
> >  need from our vendor.  We have contracted out the use of HP3000, so we
> >  have no control over it.
>
> A lack of support from a vendor should never be tolerated.
> Presumably you're paying him money. If things should come to the
> worst, that money should be withheld until a minimum level of
> acceptable service is established.
>
>
> >  The tables we are talking about have 1.5 million records.
> >
> >  Therefore, we need to keep our business going and we needed these reports.
> >  It is not my place to determine why we have such an odd arrangement, but
> >  with weekly files we are able to use SQL in FoxPro.  FoxPro is the fastest
> >  Database for PC's.  So, in terms of speed I don't think the HP3000 with
> >  300 users and only 14 jobs running at once can compete with the PC.
>
> Actually, I think that you might be surprised. An HP3000 with a good
> report writer can not only ask significantly more complex queries
> than you can in FoxPro, it can also optimize its queries in such a
> manner that the HP3000 can "blow the doors off" a PC, if the
> processor speeds are anywhere equivalent, even with 300 users on the
> machine.
>
> The question of efficiency and speed isn't dependent on the number
> of records in a dataset (1.5 million records is only moderately
> sized nowadays), it's a matter of how quickly you can isolate the
> records of interest and how many times you have to touch them. A
> good report writer will isolate and keep in memory the appropriate
> records so as to absolutely maximize speed.
>
> It's important to understand that by the mere process of using
> Suprtool to find and gather up the records to be downloaded, you've
> already done 90-98% of the effort necessary to create the reports
> you need on the HP3000.
>
> There is the belief that if you download information to a PC that
> you're off- loading work from the HP3000. Generally, not only is
> such a belief not true -- it's most normally backwards. You're
> spending a great deal more HP3000 CPU time creating and downloading
> such long lists than you would have if the reports had been executed
> locally -- not to mention the extraordinary amount of additional
> work you're going to have to do to get all of this information down
> to a PC and into a PC-based database. "Printing" the records to a
> disc file is no different to the host than printing records to a
> piece of paper.
>
> You're downloading all 1.5 million records on the *contingency* that
> all of these records will be needed for the reports you want to
> prepare. That may be true sometimes, but most often it's not. Any
> one report will probably only require a selected subset of those
> records. A good report writer, with a well- indexed database, will
> allow you to create reports with extraordinary efficiencies and ease
> on the HP3000 by simply touching only the necessary records.
>
> Under any circumstance, large massive reports shouldn't be run in
> competition with users. In reality, there really are very few
> "ad-hoc" reports in most businesses. Most normally, the great bulk
> of reports that are executed within an organization are the same
> report being re-run every week with merely different parameter
> settings. Such reports should be put into batch scheduled operation,
> executed completely automatically in the middle of the night, and
> printed to either disc files or printers spread throughout your
> campus -- so that the reports can be in the hands of the people who
> make decisions with absolutely the least possible amount of fuss or
> bother.
>
> You can do all of this on an HP3000 with surprising ease.
>
>
>
> >  Also, the Suprtool and COBOL I have seen does not allow for easy queries
> >  like SQL/FoxPro does.  So, not only is faster it is easier to get the
> >  information we need.
>
> Neither Suprtool nor COBOL describe themselves as report writers.
>
>
> >  Except for those two reasons: Speed and ease of use ther is no reason not
> >  to put them on the HP3000.  Let me say, if you have not figured it out, I
> >  am PC programmer.  I have done the Mini/MainFrame and think it is good for
> >  24x7 operations.  Otherwise bring it down to a PC and let people use tools
> >  that will be easy to use.
>
> I really do believe your statements about being a "PC programmer"
> and "letting people use tools that will be easy to use" are the
> reasons why most people want to download their information into a
> PC: it's something that they're already familiar with and they
> naturally want to continue in that direction. However, don't let
> your past experiences blind you to far better ways of solving the
> same problems. Clearly, you currently have a process that you
> describe as tedious. I'll stand by my first bit of advice: asking
> about compression algorithms is almost certainly solving the wrong
> problem in the long run.
>
> When you download to a PC, you lose these several attributes: (i)
> reliability, (ii) ease, (iii) efficiency, (iv) speed, and (v) query
> flexibility. But most importantly, you lose (vi) reconcilibility
> (everyone has a different snapshot of the data) and you lose (vii)
> easy automatability.
>
> In the 1940's, it was common to see old black & white pictures of
> rows and rows of accountants sitting at their wooden desks, all
> wearing green eye- shades, and all pulling levers on their
> hand-cranked calculating machines, furiously adding up long lists of
> numbers, trying to determine if the company is making or losing
> money.
>
> At the end of the 20th Century, due to current tendencies to
> download information, we've progressed to the point such that the
> picture has evolved to be now one of rows and rows of accountants
> sitting in their cubicles, all wearing green eye-shades, punching
> buttons on their PCs, furiously adding up long lists of numbers,
> trying to determine if the company is making or losing money.
>
> It's hard to call this progress. The current choice has, for some
> time now, boiled down to: either continue to buy more PCs and more
> cubicles and download longer and longer streams of data -- or
> process the data on the HP3000, automatically, with an exceptional
> degree of reliability and flexibility, with no human intervention --
> and free the people in the cubicles do far more useful work.
>
> Wirt Atmar
>
>
Peter E. Hecht
[log in to unmask]
http://www.execpc.com/~phecht