Hi,
Thanks to everyone who responded to the proposal. All your inputs are
highly appreciated.
I will be replying to all the mails separately, but I would like to make
some general observations here.
One of the underlying assumptions behind this proposal was that
users/system administrators would not want to change all (or most) of
the job scripts on their system to take advantage of the new
feature. We wanted to provide a way of controlling the jobs on the
system in a better way than is possible today, without having to
change all the job scripts and UDC's. We need feedback from the
customers whether this assumption is correct in the first place.
But with this assumption, the option of specifying the Queue name in
either :JOB or :STREAM command was ruled out, since that involves
changing the job files. Hence, we needed somehow to map each job to a
queue, and user.acct seemed the best way to do this.
Now comes the issue of how to make this mapping known to the system. We
considered 3 options:
1. The file method described in the proposal
2. Through :NEWQUEUE, :ALTQUEUE commands
3. Through :NEWUSER, :ALTUSER commands
(3) was ruled out because we thought system administrators may not want
to do this for thousands of existing users. Again, this may be a valid
assumption, or it may be not. We need feedback on that.
Now, having said that, let me go ahead with specific points.
Scott Gavin writes:
> > To be able to selectively allow and disallow users from submitting jobs so
> > as to be able to provide a better throughput for crucial jobs, it is
> > necessary to have some mechanism of distinguishing between jobs of various
> > users. To achieve this, we need to have multiple job accepting queues with
> > the following properties.
>
> Historically, the problem has been that you end up with a job queue containing
> something like:
>
> 1 - Long applicaiton job
> 2 - Short application job
> 3 - Short application job
> 4 - COBOL compile
> 5 - Long application job
> 6 - COBOL compile
> 7 - COBOL compile
> 8 - COBOL compile
> 9 - Short application job
>
[snip..Some very valid points deleted to save bandwidth]
> Give us the ability to say:
>
> :STREAM filename;QUEUE=shortap
>
> and to put:
>
> !JOB APAGEC,MGR.DEVACCT;QUEUE=COMPILE
I understand what you are saying here Gavin, but again, this approach
needs the job scripts to be changed. If everyone (or atleast most of
the customers) are ready to do that, we can certainly consider that.
Apart from having to change the existing jobs, there is another
drawback to this approach of explicitly specifying the queue name in
the :JOB or :STREAM command. That is the following:
Suppose a system has 3 queues, 1 for Long jobs, 1 for short jobs and 1
default queue. Now, all the jobs have been hardcoded with the queuename
that they should go into. But suppose, the system administrator feels
that 3 queues are not enough and they need more granularity in terms
of job sizes and decides to create 2 more queues. Then all the jobs
which could be scattered all over the system need to be changed to
take advantage of these 2 new queues. Whereas in the other approach
which decides queue membership based on user.acct, system
administrator will have a centralized control over which user belongs
to which queue. Ofcourse, as you rightly pointed out, each user might
have jobs of varying lengths and you need to be able to stream jobs of
different lengths into different queues. But, if we rule out explicit
specification of queue in the initiation command (since too many jobs
may need to be changed), the next best approach available is to map
user.acct to queues, in my opinion.
> > --------------------
> > 1. Queue name - A name to identify the queue. The naming convention will
> > be same as existing rules for MPE file names. The Queue name can be
> > given by the system administrator when creating the Queue and will be
> > used to refer to the queue in all other commands.
>
> I assume you mean 8 character, uppercase, alpha+alphanumeric. I think
> 'rules for MPE file names' is a little too ambiguous these days.
Yes, 8 character, uppercase, alpha+alphanumeric. But again, whether
that restriction is reasonable is open to debate. We need feedback on
that.
> > By default, one job queue will exist (similar to the currently existing
> > job queue). This queue is called SYSJOBQ. This queue cannot be purged
> > or renamed.
>
> Yuck. How 'bout the default JOB and SESSION queues are indicated simply
> by the omission of any name at all? If you want a default, it should be
> a default, not an arbitrary special case name that can't be distinguished
> from a user-named entity. I fyou must have a name, make it obvious that
> it is diferent: $SYS $SYSQ $SYSJOBQ or something like that.
Point taken.
> > 2. User class - Set of users who can stream jobs in this queue.
>
> Others may feel differently, but I don't want this extra level of security
> and the configuration/maintenance nightmares it will cause (are you going
> to enhance BULDACCT to save and restore these things?)
The main purpose of having a set of users as members of a particular
queue is just to have some mapping between the jobs and the queues, in
absence of explicit specification, as I explained above. It is not
there as a security feature. We wanted the system to somehow determine
which job should go into which queue, and this is the way for system
to do that.
> > This file will be read at system startup time. The name of this file
> > will be known to the operating system (say HPJOBQ.PUB.SYS) and this
> > file cannot be purged by the users. If any changes are done to the
> > file, :INITJOBQS command should be invoked for these changes to take
> > effect.
>
> Sounds like a pain to implement, along with the ability to have this file
> out of sync with the running system if changes are made and :INITJOBQS is
> not done. This also makes the file *useless* for programatically
> determining what are the valid job queues on the system.
>
> If you must have a configuration file, decide whether you are going to do
> it the MPE way or the Unix way. The MPE way would be an opaque binary data
> structure, preferably hidden away somewhere. The Unix way would be a flat
> ASCII file that can be maintained with an ordinary editor and is read
> every time its information is needed (the Unix way eliminates all the
> special purpose software and commands to maintain the file, and there is
> no synchronization issue, since the file is always the last word on the
> subject).
If we don't have an ASCII file editable by the users, we need to
provide commands to do that, which boils down to cost vs. ease-of-use.
But, the ACD approach suggested in another mail is an interesting
suggestion, and we can see if it is feasible. Sounds like a workable
idea.
> > Each user can be a memeber of only one Q at any point of time. Users
> > without any specified Q membership will belong to the general SYSJOBQ by
> > default. If any Q is deleted, all the members of that queue lose their
> > membership and will belong to the general job queue.
>
> Buzzz. Sorry, this is completely useless. Nobody that I know would want to
> run their system this way. What I want is the ability to have different jobs
> from the same user submitted to different queues. We will happily go change
> the !JOB records to have a ;QUEUE=LONGJOB (or ;INQUEUE= or whatever) option
> on them. Having a single user only able to submit jobs to a single queue
> does not solve any problems.
We need to know if all the customers feel the same way about being
able to change job files. But we thought it may not be the case. Will
need input there.
> > 3. Job limit - Maximum number of active jobs for this queue.
>
> This is really the only attibute that a QUEUE needs. Most of the other
> issues that could have been addressed here have been solved with the
> Workload Manager product.
True.
> > Operation:
> > ---------
> >
> > The max. number of jobs specified by :LIMIT command indicates the
> > system wide limit. At any point of time the number of jobs in the
> > system cannot exceed this limit.
>
> Ok. This means that :LIMIT 0,0 still means what it used to: Prevent all new
> (non ;HIPRI) logins on the system.
Yes.
> > The limit on number of jobs for each job queue can be set by using
> > ALTJOBQ command.
>
> Barf. What's wrong with :LIMIT 3;QUEUE=MYJOBQ
Agreed.
> > Whenever a job is streamed, the queue membership is determined by one of
> > the above mentioned mechanisms. Once the queue is determined, the
> > system wide limit and the queue limit is checked. If neither of the
> > limits are violated, then other parameters like INPRI and JOBFENCE are
> > checked. If all these conditions are satisfied, the job will be
> > streamed, else it will be kept in WAIT state.
>
> Ok, but let *us* tell you what queue we want it in. It would however be
> nice to be able to associate a default queue with each user, which can be
> done with :ALTUSER ;JOBQUEUE=COMPILE
Fair enough. We just need to know if everyone feels the same way.
> > A JOB Q can be renamed or purged by any user with SM capability,
> > provided there are no jobs waiting in the Q.
>
> I don't see the ability to 'rename' a queue as being worth the effort to
> implement.
It won't involve much effort and it may be useful if admins want to
change the queue name for whatever reason; may be company's naming
policies change and to conform with it. Ofcourse, it won't happen very
frequently, but since doesn't involve much effort (hopefully !), worth
providing it.
> > Users' membership to any user-defined queues can be added/deleted by
> > modifying the membership file.
>
> Again I think the one queue per user idea is silly, but why didn't you suggest
> implementing it as an attribute of the user (:ALTUSER) rather than introducing
> this new file?
For the reason mentioned above, that it will be easier for the
sysadmins to specify membership in one file rather than doing :ALTUSER
on thousands of users. We would like to know how much effort the users
are ready to put in to take advantage of this feature.
> > Membership for SYSJOBQ cannot be specified and will be ignored if
> > specified in the membership file.
>
> Why? Sounds like you would have to code to explicitly prevent this.
No. What I meant was, since every user has to belong to some queue,
all those who are not members of any user-defined queue will
automatically belong to this system Q. Hence, any membership defined
for this Q will be ignored.
> > SHOWJOB output:
> > --------------
> >
> > JOBNUM STATE IPRI JIN JLIST JOBQ INTRODUCED JOB NAME
> >
> > #J3 EXEC 10S LP SYSJOBQ WED 11:46A FTPMON,FTP.SYS
> > #J7 EXEC 10S LP SYSMGRQ WED 5:47P EMG,MGR.SYSMGR
> > #S81 EXEC 34 34 SESSION THU 12:17P MGR.GOPI
>
> One of the resaons I thing the name of the default queues sould be blank is
> that it unclutters this listing and makes it obvious which jobs have been
> assigned to non-default queues (displaying 'SESSION' for every session seems
> like a waste, and also makes the display difficult to read as 'SESSION' and
> 'SYSMGRQ' are the same length, start with the same letter, and share another
> common letter, wich makes them harder to distinguish visually).
Point taken.
Again, thanks a lot for your excellent feedback. Hope I have clarified
some of the points atleast.
Best regards,
mohan
|
