Hi All, Below is the beginning of a draft for the design and implementation of system level (global) and script-local CI variables. I will be working closely with Gopi in India so that we can deliver this enhancement in a timely manner. I will also be working with some folks in CSY that have already spent considerable time thinking about system variables, especially Larry Byler and Scott McClellan. I would like the *initial* feedback from the list to be somewhat general in nature so that we can focus on the basic assumptions, needs, constraints, etc. before we talk about the syntax and behavior of new commands, etc. !!!!!!!!!IMPORTANT!!!!!!!!! This is very prelimenary. There is no guarantee that we will complete this work. With that said ... here we go: ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Define a System (Global) CI Variable: ------------------------------------- A variable created by the CI or HPCIPUTVAR intrinsic that is accessible to all processes even processes outside of the job/session environment that created the variable. A system variable may outlive its creator, and, in fact, is not implicitly deleted until the system reboots. Additionally, a system variable needs some degree of security such that certain processes cannot write to or delete it. The current definition lets system variables be read by all processes (for anticipated performance gain by not needing to check for read access each time a variable is referenced). Need for System CI Variables: ----------------------------- Primary usage will be easy-to-use, efficient and simple interjob/process communication (IPC). Many customers have expressed a desire to use this type of IPC facility for security purposes. For example: a system level logon UDC creates a shared system variable that the production applications interrogate as they apply business rules. Why can't MSG files work? Perceived greater overhead, more complex, perhaps not transient enough (system variables will disappear on a reboot). Of course MSG files can hold more data than a CI variable, which is limited to 1024 bytes. (Conventions can be used to "chain" together many system variables.) Define a script local CI variable: ---------------------------------- A variable created in the context of a UDC or command file (i.e, a script) via a SETVAR command or setvar() function. This variable is accessible only within the script that created it. Specifically, it is not inherited by "children" scripts, nor is it accessible to processes run from the script. There are no permissions defined for script local variables: the script can create it, read it, write it and delete it. Script local variables are implicitly deleted when the script that created them exits. Script parameters are treated like script local variables except they cannot be deleted. This allows a script to change the value of a parameter. (Note parameters are always pass-by-value.) For the remainder of this document "local variable" will refer to a script local CI variable as described above. Need for local CI Variables: ---------------------------- There are 2 major reasons to have local variables: 1) a script using local variables will not overwrite a variable of the same name in the job/session scope, 2) the local variable cannot be changed by other processes in the same job/session. Example for 1: A well written command file saves the value of the predefined HPMSGFENCE variable, sets HPMSGFENCE to 2 and then restores the value prior to exiting. Unfortunately if the script exits prematurely (due to a code path that was not anticipated or because <break> was hit) HPMSGFENCE remains set to 2 and the user won't see any CI error messages. Example for 2: OpenView OperationsCenter (OpC) on MPE uses scripts to perform some of its automated tasks and to do system monitoring. OpC runs all of its scripts from a common job. Each script routinely sets and checks the CIERROR predefined variable. When multiple scripts are executing simultaneously they cannot determine if the command they just executed was successful, since another script from another process may have just set CIERROR to 0 or to some error. Objective of this project: ------------------------- 1) Make it easy to use CI system variables for IPC. 2) System and local variable behaviour should be intuitive. 3) Existing scripts should function the same* (backwards compatibility), meaning that some kind of explicit action should be needed to create a variable as "system" or "local". (To satisfy 2 above no explicit action, beyond today's dereferencing, should be necessary to reference a system or local variable.) 4) Some predefined variables should be "masked" (or hidden, or replaceable) by a local variable of the same name. In this case the CI will reference the local variable as if it were today's job/sesion variable. E.g., local HPPATH, HPMSGFENCE, CIERROR, HPCIERR should work. 5) Performance of existing scripts should be roughly the same as today. There should be no noticeable performance loss due to system or local variables. 6) Logon performance can be enhanced by creating all predefined variables once in the system variable table. A job/session variable table is only created if the process explicitly creates a job/session variable (e.g. executes SETVAR, setvar() or HPCIPUTVAR as defined today). 6a) Predefined "active function" variables (this means the variable's value is a procedure to call to get the real value, e.g., HPACCOUNT, HPJOBNUM, HPCPUNAME, etc.) only need to live in the system table -- no entries for these variables need to be in each job/session table. Objectives 6 and 6a are not directly related to system or local variables. However, we anticipate much faster logon time if we can accomplish these two ideas. Also this approach reduces the memory required for each job/session. (SR6/7 space is not an issue since the job/session variable table lives in "own" long pointer space.) On the other hand, to implement 6 and 6a is more difficult and we may decide to postpone it in order to meet a timely delivery to customers. * HPSYSNAME may need special consideration. This variable is thought of as a system variable, but since it is really job/session in scope it can be changed by anyone. This has annoyed many system managers. We can keep the current behavior, we can not allow HPSYSNAME to be set in the job/session scope (with a warning so jobs and scripts continue), we can create a new read-only predefined variable like HPNODENAME or HPDOMAINNAME. ?? External Behavior: ------------------ At this prelimenary stage of the design I want to keep the external specifications a little bit abstract so that we can all agree on the main objectives and the main problems that will be solved. In later postings the complete external behavior will be discussed. The order below is random. A. We need a way to create, modify, read, delete system and local variables. Additionally, I think, each system variable needs some sort of security defined. B. We need to be able to show one or more variables in two ways. We need to see which variable would be referenced if I did a !varname. Is there a local varname, job/session varname, system varname or none? We may need to see "varname" at each scope that it is defined. E.g., here is the local value for varname, here is the job/session value for the same name, etc. We may need to see more details than SHOWVAR reveals today, e.g., var type (int, boolean, string, JCW), var scope (system, job/session, local, CI-only [like HPMSGFENCE]), var security (for system variables), var creator/owner (for system variables). C. We can define properties that each var has: predefined deleteable writeable is_jcw is_ccb_var can_be_hidden { means that a var with the same name can be created in a more local table, e.g., if HPPATH were created in the system table, can_be_hidden would be true, meaning that HPPATH could also be created in the job/session table } creator* ACD or some kind of security* * applies to system variables only. D. We need to be able to set a variable to the value of a more locally scoped variable, e.g. I should be able to set a job/session var to the value of one of my local vars. I also may need to set a local var to the value of a higher scoped var. Both of these needs should naturally fall out of the var dereferencing hierarchy defined in E below. E.g., :setvar jobvar, localvar <-- assumes that localvar is defined locally and setvar is done in a script. :setLOCvar locvar, jobvar <-- assumes that jobvar is not a local var and setvar is done in a script. E. We need to define the default variable hierarchy. It seems reasonable to look first for a local var, then a job/session var and lastly for a system var before declaring that the var does not exist. F. We may need to be able to force a different scoping from the above default. That is we may want to say "reference this system var even if there is a local or job/session var of the same name defined". G. :SETVAR, the setvar() function, HPCIPUTVAR as defined today should create or modify a job/session variable, like today. H. !varname, :SHOWVAR, ![varname] should follow the agreed upon variable dereferencing hierarchy from E. I. :DELETEVAR and HPCIDELETEVAR should probably follow the dereferencing rules in E, although we may want to define a new parameter that allows the scope to be specified. J. *If* objectives 6 and 6a are implemented (all predefined vars at least start out only in the system var table and all active functions live only in the system var table) then wildcarded SHOWVAR needs to be defined and may be different from today. E.g.: a) logon <-- all predefined vars in system table b) showvar @ <-- should show all job/session vars (none) and all of the predefined vars, since that's what you see today c) setvar hppath "foo" <-- set job/ses HPPATH effectively hiding the system level HPPATH. d) setvar b 1 <-- set job/sess vars A and B setvar a 2 e) showvar @ <-- today this shows all predefined vars in alpha- betic order, followed by all user-defined vars in temporal order. Doing 6 and 6a means all but one predefined var is in the system table. Applying the var hierarchy rule in E, the job/sess table needs to be searched before the system table. Thus the output would look like: HPPATH = foo B = 1 A = 2 HPACCOUNT = xxx ... HPYEAR = xxx Note: if there were user defined system vars they would also appear in temporal order. Note: this order could be sorted any way you want but the order in the var table today is temporal. K. We probably need a way to see the security on a system var and to be able alter the security. L. Locking. I expect that all readers will lock the appropriate table shared and all writers will need an exclusive lock on the table. Note: if we implement objectives 6 and 6a we will be exercising the current locking significantly more than it currently is. Well that's a good start. Looking forwards to iterating through this design with you and then working with a beta team to make sure it all works! Thanks in advance for your valuable feedback Jeff Vance, CSY --