Not sure if this was cross posted so ..... I took the liberty...

Interesting stuff I might add... 

-------------- Forwarded Message: -------------- 
From: Pete <[log in to unmask]> 
To: [log in to unmask] 
Subject: Re: Suggestion for moving MPE systems to an OpenMPE. 
Date: Tue, 12 Sep 2006 23:38:55 +0000 


On 9/12/06, Ray Sparks wrote: 
> Hi Pete, 
> 
> Unfortunately, the list serve rejects your HTML file. You will need to 
> repost with plain text. 

I noticed that I sent the email in RTF. Maybe that was it? 

> Cheers, 
> 
> Ray 
> 

OK, let's try this again, without the HTML attachment. Kind of 
lengthy and is a bit more readable with HTML formatting though. 
-------------------------------------------------------------------------------- 
---------------------------------------------------------------- 


OpenMPE - my thoughts. 


Brief Introduction: 

I have been away from the HP3000 fold for a few years, but miss the 
productivity and simplicity of the platform when creating inhouse 
business application solutions. 

So who am I? I go aways back, having developed, enhanced, supported, 
and managed HP3000 systems for a variety of companies from the 
smallish to the Fortune 100. I took my HP3000 systems internals class 
with Rob Apgood, and showed the class how to break MPE security 
completely in 10 to 20 minutes. By that time, I was already 
decompiling SL.PUB.SYS and had broken the access code on COLOSSUS. 
After I moved up to Puyallup WA, I used to hang out with Jason Goertz 
and discuss system internals, cooking, and the merits of becoming an 
independent contractor. Which I became in 1989. Jason said I should 
publish and publish often. I always meant to, but just never seemed 
to get around to it (wish I could do a "do over" ;) ). I have recent 
training on the Oracle database (9i, 10g), became a CISSP (Certified 
Information Systems Security Professional), and have over 10 years 
experience on Linux. I am in negotiations to get out of the 
independent work with its unbillable hours, tax forms, hard to take 
time off, let alone vacation. I do have some time right now to assist 
in creating a viable OpenMPE. 

I don't think that MPE is economically viable to maintain, especially 
on its current proprietary hardware, HP-PA, even if given all of the 
code for free. Even though I think it is the best environment out 
there to develop and maintain high performance business applications 
quickly and with great reliability, it is just not going to be 
competitive in the server market. 


The overview of my solution is this: 

First, what does an application running on an HP3000 depend on? 
Generally, COBOL, VPLUS, Image, and the MPE command interpreter. You 
also need to have some 3rd party tools. Powerhouse, one or more of 
the job schedulers, ADAGER and/or DBGeneral, and a printer spooler 
manager. Off the top of my head, I think if you had these fully 
functional, you would have the next version of MPE. But, how to do it 
in an economically feasible way and stay competitive with what is out 
there? 

I saw where people were talking about emulating HP3000 hardware on 
Linux. Bad idea for all of the reasons given, monstrous job and would 
be very slow in execution. 

But, how about building MPE into Linux? Well first off, MPE and Linux 
have some fundamental differences, and you aren't going to get the 
Kernel hackers to make a major change just to accommodate MPE. But, 
what is really essential? The MPE file system? Maybe you could layer 
it on top of an existing Linux file system. But since Linux supports 
several file systems, I wouldn't rule out adding a new one to Linux 
(name it "mpefs"?). You also instantly gain the help and support of 
some of the best operating system programmers in the world by adding a 
new file system to Linux. Plus, I suspect the performance would be 
significantly better than stacking it on another file system. Yet, it 
would be easier to apply Linux utilities to the MPE file system if it 
sat on top of one of the other file systems that have extended 
attributes, and the whole Posix directory structure I think should be 
hidden from MPE view. In either case, it would be relatively easy to 
write MPE's file system intrinsics, and run any application that just 
uses the MPE file system. Maybe VESoft could lead the way here. 

Add the MPE command interpreter, and you have the foundation of MPE, 
and something to begin testing. If there is any problem with getting 
your hands on the MPE CI code, or making it work, building one after 
looking at the code is not an easy task. There are many open source 
command interpreters for Linux, "bash" being the most popular by far 
that might be hacked into a MPE CI clone. It would be really nice if 
VESoft would contribute an open source (GPL) version of MPEX to the 
effort, and be able to make it economically worthwhile with support 
fees, or selling their other security utilities. After all, if MPE 
dies, there is no income possible from MPEX. VESoft should definitely 
take the lead on this one. 

With an MPE file system and a command interpreter, you have the 
foundation for an MPE application platform. 

But, I can't think of a large application, off the top of my head, 
that isn't using Image for data storage. Once you have the MPE file 
system, Image is a user space (mode) drop in. No substantial 
modification is needed to Image Intrinsics nor Image utilities. Is 
there a problem acquiring the (Turbo)Image source? Another option is 
to write your own Image intrinsics calling one of the open source 
databases like PostgreSQL or the ever popular MySQL. Better yet, 
implement Image intrinsics using the open source library "libgda" (see 
http://www.gnome-db.org/docs/libgda/introduction.html) and have access 
to data in all sorts of databases (Oracle, PostgeSQL, MySQL, sqlite, 
etc.), XML documents, and LDAP repositories! If you can get Adager 
and/or Bradmark to port a copy/clone of Image to Linux (yet to be 
written mpefs, or any of the other current file systems), it would be 
trivial for them to add a "libgda" plugin for Image. At that point, 
all of your Image apps would run on any number of databases or data 
storage facilities. 

Most COBOL apps use VPLUS for their user interface. You could port 
the code, or Linux/Unix already has a rather decent full featured 
terminal control library called "ncurses". Some enhancements would 
necessary for HP's block mode, but it would give you access to 
hundreds of different types of dumb terminals. Under the VPLUS 
intrinsic umbrella, you could add an HTML mode, and in conjunction 
with "thttp" (see http://www.acme.com/software/thttpd/) or a hook to 
Apache and have a screen interface much more capable than VPLUS, maybe 
VSUPER? :) Since there are text mode Web clients (lynx: 
http://lynx.browser.org/, or links: 
http://artax.karlin.mff.cuni.cz/~mikulas/links/), maybe VPLUS should 
be rewritten to output HTML with a text only mode as backup, or 
default to text mode with graphic mode as an option. This should be 
done in a generic and efficient design so that Robelle's Qedit 
developers and Minisoft would be happy with it, and maybe they should 
lead the effort too. 

Finally, throw in a dozen or so intrinsics like CALENDAR, and you have 
pretty much MPE-NG (New Generation in Linux speak) or MPE/LiX (MPE on 
Linux), that your apps and essential utilities that you need (after 
compiling for whatever hardware you have, and Linux runs on just about 
every production CPU known to man!). 

So in summary, these are the essential pieces: 

* MPE file system (mpefs) written for the Linux kernel, based on 
one or more of the existing file systems with the extra MPE bits 
added, written in the same style. Designed and built this way, it is 
pretty much a certainty for inclusion as an official Linux file 
system. I think Robelle and/or VESoft could lead this effort. 
* MPE command interpreter, hopefully a GPLed version of MPEX. Or, 
anyone or group willing to hack the popular "bash" command interpreter 
up into a clone. Should be done by stripping out all the bash 
commands and then adding the essential MPE commands used in the 
popular XEQs and UDCs first. After that is working smoothly, then add 
in other MPE commands as necessary. This part really cries for VESoft 
leadership and involvement. 
* (Turbo)Image acquired from HP and tweaked to compile and run on 
Linux using the new MPE file system. Or, the (Turbo)Image intrinsics 
rewritten on top of "libgda". Adager/Alfredo, I would think would be 
an excellent choice to lead this project up. 
* VPLUS -- same as Image, or rewrite to output HTML and use with 
links to Apache or a small Web server like "thttp". Minisoft or WRQ 
should lead this project. 
* Add any miscellaneous intrinsics needed, written to call Linux 
system functions, of course. Maybe VESoft could take the lead here 
too. 

Voila, you now have a working MPE application platform (MPE/NG? 
MPE/LiX?) that is integrated into Linux with all of Linux's hardware 
support, and a wealth of functionality just beyond the great MPE 
business application platform within. Without any code modifications, 
you could have all of your users at an "MPE" prompt using the standard 
Posix "login" program, or a modified "login" to mimic the HP3000 
exactly! Using Xen and a Xen enabled Linux kernel, you could have 
an/multiple MPE application platform(s) running as a virtual server(s) 
that could be rebooted in a few seconds, instead of several minutes. 
Adding the "screen" program to the new MPE environment gives even dumb 
terminals the ability to run multiple programs at once in their own 
private virtual screens. 


For better Linux integration and control: 

As described above, you could create your own private MPE world 
running on top of Linux in its own isolated environment with only 
links to the kernel for networking, hardware drivers, symmetric 
multiprocessing, cluster and grid support, process scheduling and 
dispatch, thread support, and locking and semaphore control. In user 
mode, it would be indistinguishable from MPE to end users and 
application developers. All the "magic" would happen in privilege 
mode where the interface between user mode MPE and the Linux world 
would take place. And then there would be the Linux world that would 
be invisible and inaccessible from user mode MPE. In this world, PM 
code and users would see both the MPE world and the Linux world. Much 
compatibility shoehorning would be going on very similar to now with 
the MPE/iX world with the combining of Posix and classic MPE. 
Personally, I think this is muddy and adds unnecessary complexity that 
results in more opportunities for bugs to creep in, harder to learn 
when applications use the two environments to varying degrees, and 
conflicts between the two environments always result in some sort of 
compromise. As a result, my belief is that when HP went from classic 
MPE to MPE/iX, that although they gained functionality and a 
significant amount of shared common code between MPE an HP-UX, the 
bottom line for MPE the excellent business system solution platform is 
negative. 

Is this combined world really needed? Why not, when we cross the 
border from user mode to priv mode, we go from classic MPE to full 
Linux? By doing this, there is no conflict resolution. On one side 
of the wall is a Linux distribution, on the other is classic user mode 
MPE. On the Linux side, we build the support structure necessary to 
support the user mode of classic MPE. On the MPE side, we create the 
minimum number of PM intrinsics that communicate directly with Linux 
as any other Linux library would. No transition nether world. The PM 
world does not use MPE intrinsics to access MPE system internals -- it 
is simply a Linux directory tree and a library API. At that point, 
there is nothing to preventing you from using a database invisibly as 
the data store for all of MPE! Think about that a moment. 

It would not be difficult to sandbox a classic user mode MPE 
environment/platform in Linux to create a highly secure environment 
using various methods. Some are strong, but complex. Some are 
simple, but weak. Some are in between. Just know that it can be done 
relatively economically at any assurance level required - one method 
was originally contributed by the National Security Agency (see 
SELinux: http://selinux.sourceforge.net/) which NSA continues to work 
and collaborate on. This method is both extremely powerful, taking 
over the Linux kernel from a security point-of-view, but has the 
ability to be very fine grained -- nothing happens without explicit 
permission even if you are root. Linux also has been able to take 
advantage of the new CPU security capabilities from first introduction 
by Intel and AMD, that the classic HP3000 had 30 years ago. It should 
be the goal of OpenMPE to provide well hardened releases, using 
multiple layers of security that user mode would sit on top of with 
complete faith in the underlying security protecting it and the Linux 
system it sat on, from network and Internet exploitation. 

Essentially, the MPE file system would only be able to access parts of 
a single directory tree ( i.e FILE.GROUP.ACCOUNT == 
/path/to/mpetree/ACCOUNT/files/GROUP/FILE and USER.ACCOUNT == 
/path/to/mpetree/ACCOUNT/users/USER), hiding the underlying Linux file 
structure. Although with MPE/iX, the Posix environment is only thinly 
veiled, I think this just adds complexity to the simple, yet robust 
MPE environment. I don't mind adding links using PM intrinsics to get 
to the underlying additional Linux functionality as needed, just not 
exposing all of Linux to user mode. Having the Linux environment 
available to application programs and users at the command interpreter 
prompt, I think defeats the purpose of MPE being a simple and 
dependable business application platform. 

There are some issues with MPE's concept of jobs, sessions, and 
temporary files that don't have exact equivalents in Linux. I think 
most of this could fairly easily handled by an additional invisible 
(to MPE user mode) directory layer under the "/account/user/" 
subdirectories. For instance, a temporary file STATS in group 
UTIL.SYS for user JOE.SYS created in session #S103 from the Linux 
point of view would be 
"/path/to/mpetree/SYS/users/JOE/jobs/S103/SYS/UTIL/STATS". A useful 
MPE system table could be 
"/path/to/mpetree/SYS/users/JOE/jobs/S103/.jobinfo". A system table 
could be "/ path/to/mpetree/.jmat", or I like 
"/path/to/mpetree/yyyy-mm-dd-hh-mm-ss/.jmat" (the embedded timestamp 
being that of the MPE bootup -- could be handy where MPE has crashed, 
but the Linux host is still alive). UDCs would be a simple file 
"/path/to/mpetree/SYS/users/JOE/.udclist" that only things like 
SETCATALOG could touch. The UDCs in effect for a session would be a 
simple file of the different UDC files combined 
"/path/to/mpetree/SYS/users/JOE/jobs/S103/.udcs". Maybe a job 
recovery mode after a system crash could be made optional before 
actually clearing the MPE invisible "jobs" trees and before allowing 
sessions and jobs to logon. Delayed job/session recovery and analysis 
after a crash could be accomplished by simply renaming all of the 
"jobs" subdirectories to "jobs-yyyy-mm-dd-hh-mm-ss" directories where 
"-yyyy-mm-dd-hh-mm-ss" is the time of the previous bootup. This is 
recorded down to the second, because virtual servers could be rebooted 
multiple times in a minute. If disk I/O performance becomes a problem 
accessing MPE system tables, (invisible to MPE) symbolic links could 
be used to link files and/or directories to a (dynamic storage) RAM 
disk. Other solutions are available on Linux, but this is the 
simplest and most robust I can think of right now. 

Using a standard Linux configuration, MPE users would log on to Linux 
with the user MPE (or mpe, if preferred), which would take them to an 
MPE prompt for the MPE user name, exactly as it is now. Or with a 
minor amount of fuss, the Linux logon could be automatic for a list of 
directly connected devices SSH (secure telnet) devices, taking the 
user directly to the MPE logon prompt. 

My personal belief is that the most valuable part of MPE, and the 
reason for its longevity, is because it makes such a great business 
application platform -- simple, robust, and efficient. Linux/Unix is 
designed more for geeks, and should be safely kept away from business 
application development, IMNSHO. The Linux kernel and the GNU 
operating system that makes up a Linux distribution can be pared down 
to be a nice svelte customized foundation (a few megabytes) that a new 
MPE application platform could rest on, hidden from everybody 
(including SM and OP users which would use MPE interfaces to the 
hardware and the OS), except the system internals support geeks. Or, 
you could have a full blown Linux server power house with the new MPE 
user mode OS running in a virtual machine inside of it, and the MPE 
users wouldn't know the difference. 

All of this I think is quite doable, and economically viable. This is 
assuming that the new MPE application platform itself is open source 
with programs licensed under the GPL and libraries/intrinsics under 
the LGPL. This still allows for proprietary utilities like Adager and 
JMS, and proprietary 4GLs like Cognos Powerhouse as long as they are 
compiled for the CPU in your server. The OpenMPE organization would 
be the clearing house for all MPE patches and enhancements, and would 
be responsible for distributing cryptographically signed official 
versions of MPE versions. There are a number of ways to quickly 
verify that an official version of MPE has not been hacked or been 
tampered with after installed. Anyone could experiment and play with 
new ideas for MPE on their own Linux development system, which could 
be pretty much any modern desktop PC system with Linux. 

Gone is all of the driver code, which although complex and lengthy, 
still only supports a limited number of peripherals and storage types. 
Gone too is the bootstrap code, and the kernel code. The logical 
volume manager is replaced by an MPE link to the very robust and 
featureful LVM2 logical volume manager. Also gained is a network 
stack that can't be beat (the BSD gurus may disagree, but BSD's 
superiority is history now) -- nearly all network protocols are 
supported. MPE would now run on any hardware that Linux does, which 
is just about anything imaginable, as long as it has the power to 
support the business' application(s) and users. All this with only 4 
main modules and a handful of misc. intrinsics! Third party MPE 
utility vendor support is crucial too, and they don't have to open 
source their code (unless they want to ;) ). I am convinced that the 
platform itself needs to be open sourced (GPL and LGPL) to keep it 
from fragmenting and disintegrating, and future support is never an 
issue from that point on. 

The key I believe, is creating the MPE application platform, and not 
getting bogged down in the morass of hardware device drivers and 
kernel code. 

Sincerely, 
Peter M. Eggers 

* To join/leave the list, search archives, change list settings, *
* etc., please visit http://raven.utc.edu/archives/hp3000-l.html *