Someone wrote:
> When you discover a user that is hogging the system just issue the altproc
> commnand against their session and drop them to the "E' queue. That way
> their "quick" report won't be so quick!
>
> Someone else wrote:
>
> > A programmer running
> > a "quick" report or QUERY FIND ALL, REPORT with SORT will not only
> > knock back the batch jobs, but also impact all the legitimate on-line
> > processes. Once you have found the culprit an education session,
> > followed by a course in manners accompanied by a big stick should
> > resolve your problems without resorting to changing the queues.
> > Sometimes that does not work. We had one programmer who never used
> > batch and selfishly ignored the impact on other users. The solution was
> > simple. Everyday we used GLANCE to knock his session back to the DQ and
> > left him there. This went on for a year and he never found out and we
> > never had a problem with him hogging the system.
Based on these comments, I don't believe the nature of the HP3000's scheduler
is being fully understood.
The two central goals of a scheduler are (i) fairness and (ii) a desire to
never let the CPU set idle, to drive CPU utilization to 100%, if at all
possible.
These are, however, exactly the same goals that drive the design of an
industrial factory or a bank or an airlines, and the mathematics that is used
to describe CPU schedulers comes directly from the operations research people
in industrial engineering. There is no significant difference between the
various scenarios other than the speed at which their respective processes
are dispatched and executed.
If you reread my posting (it's not as silly as it may sound and it doesn't
hurt to think of these things in slow motion), and then reread Jeff Kell's,
and then read these two web pages:
http://www.sct.edu/cs/classes/cs503/ops3.htm
http://moonbase.wwc.edu/~aabyan/352/Scheduling.html
you're going to have a very good idea of how processes are queued, scheduled,
and executed on the HP3000.
A couple of notes that may make reading the web pages easier: a
first-come/first-served (FCFS) queue mentioned in these pages is the same
thing as the "linear queue" on the HP3000, a non-preemptive queue that is
reserved primarily for system processes. A "round-robin" queue is almost the
same thing as the "circular queues" of the HP3000's user processes, with the
significant exception that a process that stays within a circular queue on an
HP3000 "decays" in priority every time it makes a trip through the queue.
Although "flat" round-robin queues are often described as the oldest,
simplest and fairest of the queuing structures, the addition of priority
decay makes the HP3000's queuing structure even more fair yet again -- and
more efficient.
It is easy to demonstrate (an academic way of saying, "easy to prove") that a
"shortest job first" (SJF) scheduler is always the most efficient. The
problem is, in real life, you have no way of knowing a priori which of the
several processes that are currently in a queue is the shortest and which is
the longest. The decay algorithm solves this problem for you, adaptively. If
a process repeats over and over again in the C queue, its priority drops with
each pass through the queue, ultimately to the bottom limit of the C queue.
While new processes that have just entered the queue are normally put at the
end of the queue (and I presume they are done that way on the HP3000), their
position doesn't really matter. At the end of each timeslice, the scheduler
reassesses its process list. The circular queues on the HP3000 are not
"flat." Newly arriving processes' priorities are set so high that the
dispatcher will always cause them to execute first, maintaining the illusion
to the casual user that he or she has the machine all to himself.
Someone running a massive report in the C queue will have no more effect on
the terminal users than would someone running the same report in the D queue.
Indeed, if you look, the HP-fresh-from-the-factory settings have the limit of
the C queue set just slightly higher than the D queue.
> ------QUANTUM-------
> QUEUE BASE LIMIT MIN MAX ACTUAL BOOST TIMESLICE
> ----- ---- ----- --- --- ------ ----- ---------
> CQ 152 200 1 2000 8 DECAY 200
> DQ 202 238 2000 2000 2000 DECAY 200
> EQ 240 253 2000 2000 2000 DECAY 200
Because the factory settings don't have the queues overlapping, a massive C
user will effectively shut down the D queue jobs if the heavy C user can get
his data into main memory and make his process CPU-bound. Nevertheless, he
won't have any more impact on the light C queue users than if he were in D
queue.
If you keep your scheduler tuned the way that HP has it set by default, doing
report development in the C queue is all right, so long as there aren't too
many people doing it. But you do want to get your IMPORTANT production
reports put into batch as soon as possible and run in the D queue -- and get
your background, non-time-critical, even more massive reports put into the
E-queue.
Doing this creates an extremely simple arrangement, where you help the HP3000
by putting the shortest jobs first by putting them into the higher queues,
and thus help keep overall system processing optimal.
If you did want to adjust your HP3000's priority structure to allow all
queues to move a bit, and to prevent any one user from shutting down all
lower queues, a reasonable restructuring would be to move the bottom limit of
the C queue somewhere into the middle of the D's priority numbers and leave
the CQ algorithm on "decay." The D queue numbers (base and limit) would be
left where they are, but the algorithm would be changed to "oscillate." The
same would be done for the E queue. Its algorithm should also be set to
"oscillate" and its upper base raised to perhaps the same as the base of the
D queue, but its bottom limit left where HP set it.
In this manner, the C queue "hog" will automatically begin to start sharing
his time with the D and E queue jobs. In fact, depending on where you set the
bottom of the C queue (at 50% up from the bottom of the DQ, 80% up, 90% up,
etc.), the C queue heavy user could actually get less time than that of the D
queue users. This is the part that requires tuning and experimentation.
How well this works for you depends heavily on what kind of processes you're
putting into the various queues. I generally recommend to our business users,
who quite often have no DP staffs on-hand, that they don't fiddle with the
queuing priorities that HP has set; rather, the far better thing to do is to
be religious about the proper assignment of processes to their appropriate
queues -- and think of a bank with one teller in it.
Wirt Atmar
|
