I seem to have gotten all this to work finally, thanks in no small measure
to the assistance of this list's members.
I still have some reservations about the _BLDPATMS_ variable settings for
the incoming STD fileset. Specifically, my recovered STD files, TEST@, do
niot seem to exhibit the same #X (number of extents) and MX (maximum
extents) values as the files created by the native STORE command. If
anyone has any suggestions as to how I duplicate the output of a native
STORE via the ftp ;buildparms setting then please let me have them.
:listfile /BACKUPS/HARTLYNE/@,2
PATH= /BACKUPS/HARTLYNE/
CODE ------------LOGICAL RECORD----------- ----SPACE---- FILENAME
SIZE TYP EOF LIMIT R/B SECTORS #X MX
STORE 128W FB 1978547 16776959 1 1978560 * * STDHAL.1.1
STORE 128W FB 1270579 16776959 1 1270592 * * STDHAL.2.1
STORE 128W FB 2194231 16776959 1 2194240 * * STDHAL.3.1
STORE 128W FB 2203869 16776959 1 2203872 * * STDHAL.4.1
STORE 128W FB 38076 204800 1 38080 22 8 TEST.1.1
STORE 128W FB 73808 204800 1 73808 40 8 TEST.2.1
STORE 128W FB 28479 204800 1 28480 17 8 TEST.3.1
STORE 128W FB 33765 204800 1 33776 20 8 TEST.4.1
A the risk of presenting an over long message I nonetheless feel that the
present solution might prove useful to some others so I present the
transfer job file here. This job only employs native MPE/iX CI scripting
commands and FOS utilities, no third party products are used at all.
!JOB FTPRSTOR,OPERATOR.BACKUPS,hartlyne;INPRI=9;OUTCLASS=LP,1,1
!
!COMMENT Set test parameter values in the lines above 21
!SETVAR _STDFILE_ "TEST"
!SETVAR _HOSTNAME_ "192.168.219.23"
!SETVAR _FTPPATH_ "/ftp-drop"
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!COMMENT This should be line 21 in the template job file.
!COMMENT Job file lines above this one are provided by the
!COMMENT calling process.
!
!COMMENT This filename is FTPRSTOR.JOB.SYS.
!
!COMMENT Author: James B. Byrne 2008 Sep 23 - JBB8
!COMMENT Derived from FTPSTORE.JOBS.SYSLOCAL
!COMMENT
!COMMENT Purpose: Transfer via FTP, MPEiX STORE disc filesets
!COMMENT held on a alien OS in binary format to an
!COMMENT MPEiX target for recovery.
!
!COMMENT Ensure that the target host is configured to accept
!COMMENT ftp connections (daemon running and firewall open).
!COMMENT Also ensure that security issues (SELinux) do not
!COMMENT prevent uploads. To avoid password compromise we use
!COMMENT anonymous ftp to transfer files, so enable anonymous
!COMMENT uploads and downloads as well. Be sure that the ftp
!COMMENT host is secured agains unauthorized access by the firewall.
!
!COMMENT Due to MPE/iX ftp client implementation errors
!COMMENT mput cannot be used to retrieve files with a name
!COMMENT containing two "." as this is considered a traditional
!COMMENT MPE style name form of filename.group.account.
!COMMENT The MPE ftp client strips off the leading "./" before
!COMMENT passing the file name to the host, which returns the
!COMMENT filename as sent, which causes the MPE modifications
!COMMENT in the client to treat the received file as belonging
!COMMENT to the traditional MPE namespace.
!
!COMMENT Files in an STD STORE SET are kept in the HFS namespace
!COMMENT with the form STDFIL.x.y, where the 'x' is the device
!COMMENT number for parallel backups and 'y' is the
!COMMENT reel number for that device. The STDFIL is the given
!COMMENT name for the STORE SET. Recovery involves transferring
!COMMENT the store set back onto a target HP3000 into the HFS
!COMMENT of that machine and setting a file equation:
!
!COMMENT :FILE SD=STDFIL.HPGROUP.HPACCOUNT;DEV=DISC
!
!COMMENT Passing *SD as a back reference to STORE will suffice
!COMMENT to recover from the HFS file set. The ";DEV=DISC" is
!COMMENT ABSOLUTLY required for RESTORE to find the files and
!COMMENT behave as required. Recall that all backup and restore
!COMMENT utilties were written with magnetic tape media as the
!COMMENT expected storage device.
!
!COMMENT Once the STD fileset is transfered to the target
!COMMENT and the requisite FILE equation is issued then
!COMMENT the RESTORE command my be used to recover any
!COMMENT files or directory structures as required.
!
!COMMENT :RESTORE *SD;@[log in to unmask]@;DIRECTORY;OLDDATE;PROGRESS=5;SHOW
!
!COMMENT 2008 Sep 30 - mput replaced by get in while loop.
!
!COMMENT If not provided above then set the IP address or
!COMMENT DNS hostname of the target system here:
!COMMENT (check that the FTP service is running on target and
!COMMENT that access is not blocked by a firewall.)
!IF NOT BOUND (_HOSTNAME_) THEN
! SETVAR _HOSTNAME_ "192.168.219.1"
!ENDIF
!
!TELLOP !HPJOBNAME FTP mget target is !_HOSTNAME_.
!
!COMMENT If not provided above then set the pickup directory
!COMMENT of the source host here:
!COMMENT (check chmod permissions of this directory on target host.)
!IF NOT BOUND (_FTPPATH_) THEN
! SETVAR _FTPPATH_ "/ftp-pickup"
!ENDIF
!
!TELLOP !HPJOBNAME FTPSTORE target directory is !_FTPPATH_.
!
!COMMENT If not provided above then set the store-to-disc
!COMMENT fileset name here:
!COMMENT [verify that the STD file(s) exist(s).]
!
!SETVAR _STDPATH_ "/!HPACCOUNT/!HPGROUP/"
!
!IF NOT BOUND (_STDFILE_) THEN
! IF HPGROUP = "SYSTEM" THEN
! SETVAR _STDFILE_ "STDSYS"
! ELSEIF HPGROUP = "HARTLYNE" THEN
! SETVAR _STDFILE_ "STDHAL"
! ELSE
! SETVAR _STDFILE_ "STDFIL"
! ENDIF
!ENDIF
!
!COMMENT FTPRSTOR is the complement of FTPSTORE. Therefore
!COMMENT the files that we expect to get must not already
!COMMENT exist, for if they do then the ftp transfer will fail.
!SETJCW CIERROR = 0
!LISTFILE ./!_STDFILE_@,2
!
!COMMENT If the files locally exist then the job fails.
!IF CIERROR = 0 THEN
! TELLOP !HPJOBNAME found existing STD files in !_STDPATH_.
! TELLOP !HPJOBNAME existing file !_STDFILE_.
! TELLOP !HPJOBNAME transfer job terminating. Check job listing.
! EOJ
!ENDIF
!
!COMMENT We will need two ftp sessions:
!
!COMMENT The first will get the files in the pick up directory
!COMMENT and put them into a working file that will be used to
!COMMENT construct the multiple gets now required because of
!COMMENT MPE/iX bug in ftp client.
!
!COMMENT The second session will actually perform the multiple
!COMMENT gets constructed from the work file.
!
!COMMENT First session, get the filenames to ftp get.
!
!COMMENT Create a temporary file to hold the ftp session script.
!FILE ftpcmd;temp
!
!COMMENT But never assume anything about your disc environment.
!CONTINUE
!PURGE ftpcmd
!CONTINUE
!PURGE fnlistls
!
!COMMENT echo > rewinds output file to start, >> appends to end.
!echo exitonerror > ftpcmd
!echo user anonymous [log in to unmask] >> ftpcmd
!echo binary >> ftpcmd
!echo prompt >> ftpcmd
!echo cd !_FTPPATH_ >> ftpcmd
!echo ls !_STDFILE_* fnlistls >> ftpcmd
!echo quit >> ftpcmd
!
!SETVAR FTPLASTERR 0
!
!SHOWJCW
!SHOWVAR @
!
!CONTINUE
!PRINT *ftpcmd
!
!SHOWTIME
!
!CONTINUE
!RUN FTP.ARPA.SYS;INFO="!_HOSTNAME_";STDIN=*ftpcmd
!
!SHOWTIME
!
!COMMENT Show the contents of the ls command.
!CONTINUE
!PRINT FNLISTLS
!
!COMMENT Create a message file to recieve the names.
!COMMENT This is required because INPUT < FILENAME only
!COMMENT reads the first record of FILENAME. If, however,
!COMMENT FILENAME is a message file (:FILE FILENAME;MSG)
!COMMENT then each read (INPUT) will remove the record read.
!COMMENT thereby "popping" the next record to the first
!COMMENT position for the next read (INPUT) statement.
!
!FILE FNLIST;MSG
!
!COMMENT Now copy the contents of the listfile into the
!COMMENT into the message file. This step is required because
!COMMENT MPE/iX ftp client will not accept FILE equate back
!COMMENT references. In other words (ftp> LS * *FNLIST) fails.
!
!COMMENT Trust NO ONE!!
!PURGE FNLIST
!
!CONTINUE
!FCOPY FROM=FNLISTLS; TO=*FNLIST; NEW
!
!CONTINUE
!RESET FNLIST
!
!COMMENT Set the build parms for the incoming files
!SETVAR _BLDPARMS_ ";CODE=STORE;REC=-256,,F,BINARY"
!
!echo exitonerror > ftpcmd
!echo user anonymous [log in to unmask] >> ftpcmd
!echo binary >> ftpcmd
!echo prompt >> ftpcmd
!echo cd !_FTPPATH_ >> ftpcmd
!echo dir >> ftpcmd
!echo :listfile ./!_STDFILE_@,2 >> ftpcmd
!WHILE FINFO ( "FNLIST", "EOF" ) > 0
! INPUT _FNNEXT_ < FNLIST
! SETVAR _FNNEXT_ RTRIM(_FNNEXT_)
! echo get ./!_FNNEXT_ ./!_FNNEXT_!_BLDPARMS_ >> ftpcmd
!ENDWHILE
!echo :listfile ./!_STDFILE_@,2 >> ftpcmd
!echo quit >> ftpcmd
!
!SETVAR FTPLASTERR 0
!
!SHOWJCW
!SHOWVAR @
!
!CONTINUE
!PRINT *ftpcmd
!
!SHOWTIME
!
!CONTINUE
!RUN FTP.ARPA.SYS;INFO="!_HOSTNAME_";STDIN=*ftpcmd
!SHOWTIME
!
!IF FTPLASTERR <> 0 THEN
! SHOWVAR FTP@
! SETJCW JCW FATAL
!ENDIF
!
!CONTINUE
!PURGE FTPCMD,TEMP
!
!TELLOP STORE STD file transfer FROM !_HOSTNAME_ completed.
!WARN @ STORE STD file transfer from !_HOSTNAME_ Finished.
!
!SHOWJCW
!SHOWVAR @
!
!COMMENT Leave no fingerprints behind.
!
!PURGE FNLIST
!PURGE FNLISTLS
!
!EOJ
I hope that this helps someone else at some point.
--
*** E-Mail is NOT a SECURE channel ***
James B. Byrne mailto:[log in to unmask]
Harte & Lyne Limited http://www.harte-lyne.ca
9 Brockley Drive vox: +1 905 561 1241
Hamilton, Ontario fax: +1 905 561 0757
Canada L8E 3C3
* To join/leave the list, search archives, change list settings, *
* etc., please visit http://raven.utc.edu/archives/hp3000-l.html *
|
