CDS340

FTP & SFTP File Transfer

Last Revised: 10/27/22

This called routine can be used on Unix/Linux systems to communicate using ftp (File Transfer Protocol) or sftp (Secure File Transfer Protocol).  It supports most popular ftp commands.  The remote server can be any supported operating system including Unix, AIX, Linux, Windows, etc.  CDS340 also supports the pubnet Value Added Network (VAN) which is similar to ftp.

Calling Format:

CALL "CDS340", ERR=XXXX, CHAN, CMD${, MSG$, DEBUG}

{Braces indicate optional parameters}.

ERR= CDS340 exits with an error whenever the requested command fails.  This would include reasons such as unable to login, requested file to get is not found, unable to perform a directory command, etc., or if the command timed out with no reply from the remote ftp server.  The MSG$ will contain the last ftp status message which may help determine the problem.
CHAN Used as the channel for the connection to the ftp command, assigned by CDS340 during the initial call to Open the ftp channel.  The channel is closed by CDS340 upon receipt of the Q command.
CMD$ Command  and arguments passed to CDS340.  The first character of CMD$ is the command type.  Any additional characters are arguments.  Multiple arguments are separated by commas.  Commas cannot be used in any arguments.
TypeDescriptionArgumentsNote
OOpen ftp connectionURL or IP, LoginID, PasswordThis must be the first call to CDS340.  The URL may optionally include a port number using a colon separator as in: ftp.excellware.com:9022
SOpen sftp connectionURL or IP, LoginID, PrivateKeyFile,options

The Note above for ftp connections, also applies to sftp connections.

Note that sftp does not support the ability to pass a password to the sftp server.  The options are either to manually enter the password when prompted or to store a pubic key file on the remote server which eliminates the password prompt.  The PrivateKeyFile is optional.  If not passed, then the default key file of ~/.ssh/id_rsa key file will be used where ~ represents the current user's home directory.  If the PrivateKeyFile is shared among multiple users, then you can specify the path to the PrivateKeyFile such as/u/CDI/XX/config/id_rsa.  See the Creating Key Files section below for steps to create the private and public key files.

CDS340 includes sftp command line options "-oMACs=hmac-sha1 -oStrictHostKeyChecking=no".

Although these options are typically sufficient, some servers do not support these options. In that case you can replace these options by providing the options argument, or pass the options argument as a blank to exclude the default options.

UUser loginLoginID, PasswordOnly needed when a second login and password is required.  Applies to ftp only.
CChange directoryDirectory 
MMake (define) directoryDirectoryMay not contain the / character when using ftp.
RRemove (delete) directoryDirectoryMay not contain the / character when using ftp.
LChange local directoryDirectory 
GGet (retrieve) fileRemoteName{,LocalName}File names cannot contain the / character when using ftp.  Use the C and/or L commands to change directory before sending/receiving/deleting data files.
PPut (send) fileLocalName{,RemoteName}
XDelete (erase) fileFilename
NRename remote fileOldName,NewNameAvailable in versions dated 7/22/22 or later.
DDirectory List{Pattern,Sequence}
 

Examples:

D

D*.TXT

D,d

D*.TXT,s

Pattern
nullInclude all files in directory
patternInclude file names that match pattern only, i.e., *.TXT or master*
Sequence (ignored except on linux/unix ftp servers)
nullFiles will be returned in ascending File Name sequence
DDate Sequence (oldest first)
dDate Sequence (newest first)
SSize Sequence (smallest first)
sSize Sequence (largest first)
BSet File Transfer mode to binaryNoneSend this command before sending or receiving files that contain non-ASCII characters.  The command remains in effect until the "A" command is used. 
ASet File Transfer mode to asciiNoneSend this command before sending or receiving files that only contain ASCII characters.  The command remains in effect until the "B" command is used.  Silently ignored when using sftp as all sftp file transfers are binary.
QQuits the ftp sessionNoneAlso closes the channel CHAN.
MSG$ Returned from CDS340.  In most cases, MSG$ will contain the last ftp status message.  In the case of the D command for directory list, MSG$ will be templated.  If no non-directory files match the pattern then MSG$ will contain only the CNT field and MSG.CNT will be zero.  If there are non-directory files that match the pattern specified, then MSG$ will contain the fields listed below.
CNTThe number of non-directory files matching the pattern specified.
MAXFILThe length of the longest file name returned
NAMES$[CNT]An array of file names within the directory.  Each will be MAXFIL characters long.
ENTRY$A string that can be used with CDS069 to prompt the user to select a file from the list
DISP$The string of file names followed by a 12 character date string in the same format as provided by the ftp command, and an 11 character file size masked with commas.  See example below.
DEBUG This optional parameter can be passed as 1 to request that CDS340 display all ftp status messages and prompt before exiting.  Pass as a 2 to disable prompting the operator with an error message.  This is the default for background tasks.

Creating key files

The secure ftp protocol does not allow passing the password to the remote server.  Although sftp will allow the user to enter the password, this is not very convenient when it is desired to automate an sftp process.  Instead, you can generate key files, and save the public key on the remote server.  When these key files are in place, then sftp will function without the need to manually enter the password.

Use the following command on linux servers to create both the public and private key files in the current user's home directory.  The command may differ on other operating systems.

ssh-keygen -t rsa

You will be prompted for a passphrase.  You should just touch Enter and not assign a passphrase. If you do assign a passphrase then you will need to enter it manually when making a connection.

The files created will be:

~/.ssh/id_rsa
~/.ssh/id_rsa.pub

The first file is the private key.  The permissions must be r--r--r-- or rw-------.  In the default location, the private key can only be used by the user that created the file.  This is because the permissions of the .ssh directory is rwx------ which prevents others from accessing the private key.  To share this private key among multiple users, move the private key file to a directory accessible by all users and insure the file permissions enable reading but not writing.

The second file with the .pub extension is the public key.  The contents of this file is required on the remote server in order to permit sftp connections without manually entering the password.

If the remote server is a linux server, then the content of your public key file needs to be appended to the ~/.ssh/authorized_keys file.  In some cases, you can sftp or scp the public key file to the remote server yourself.  In other cases, you will need to email the public key to the remote server administrator who will install you public key as appropriate for what ever operating system the remote server happens to be using.  The permissions of the .ssh directory need to be rwx------ and the permissions of the authorized_keys file must be rw-------.

If the remote server is a Microsoft Windows server, then create the .ssh folder in the Windows account profile folder, i.e., C:\Users\USERNAME\.ssh then define a file name authorized_keys with the contents of the contents of the public key file. Then grant read permissions to the .ssh folder. As an Administrator execute this command: icacls C:\Users\USERNAME\.ssh /grant "NT Service\sshd:R" /T

Note that the authorized_keys file whether linux or Windows, can contain multiple keys when required.

 

Example
CALL "CDS340",ERR=FAIL,CHAN,"Oftp.excellware.com,username,password",MSG$
CALL "CDS340",ERR=FAIL,CHAN,"Cupload",MSG$
CALL "CDS340",ERR=FAIL,CHAN,"Cdocs",MSG$
CALL "CDS340",ERR=FAIL,CHAN,"D",MSG$
CLOSE (CHAN)
IF MSG.CNT THEN CALL "CDS069", 1, MSG.MAXFIL, 0, 0, "+file name+", MSG.ENTRY$ + MSG.DISP$, 0, FILNAM$

CDS069 display will look like this: