Please note: All new versions will be located at GitHub under Waarp new project name :

http://waarp.github.com/Waarp/

The project still remains Open Source under GPL V3.

We describe here the logic of the protocol.

 

A simplified picture is shown here:

First we describe the different value that can be found.

• A Request (DbTaskRunner) can have several UpdatedInfo status:
  • UNKNOWN : no particular information on it.
  • NOTUPDATED : in used by other database object when they are taken into account.
  • INTERRUPTED : a request is interrupted but can be rescheduled.
  • TOSUBMIT : a request is proposed to be submited by the Commander (or other database object are supposed to be taken into account).
  • INERROR : a request is in error and can not be submited by the Commander until its status is changed explicitely.
  • RUNNING : a request is currently running.
  • DONE : a request is over and fully done.

• A Request can have several Step values:
  • NOTASK : the request has never started.
  • PRETASK : the request is currently in Pre transfer step.
  • TRANSFERTASK : the request is currently in transfer step.
  • POSTTASK : the request is currently in Post transfer step (valid transfer only).
  • ALLDONETASK : the request is fully finished (UpdatedInfo is in DONE status too).
  • ERRORTASK : the request is currently in the Error step while an error occurs (either in PRE, TRANSFER or POST step).
• A Request has two Step values:
  • GlobalStep : the current request step value
  • GlobalLastStep : this is the last valid request step value. When GlobalStep is in ERROR step, GlobalLastStep says in which step it was before entering in error. This information is used to enable restart of the Request from this valid last step.

• Each Step values and UpdatedInfo has an ErrorCode detailed information:
  • InitOk : stands for correct initialization of the Request (startup and authentication)
  • PreProcessingOk, TransferOk, PostProcessingOk : stand for correct ending of the specified step
  • CompleteOk : stands for all action are correct (ALLDONE for step and DONE for UpdatedInfo)
  • Running : stands for current Step is in Running status.
  • StoppedTransfer, CanceledTransfer : stand for a Request where an action stopped or canceled the given Request. A Stopped Request can be restart from the current status. A Canceled Request starts from the beginning of the current step. For instance for the Transfer step, Stopped will imply that restart is from the current valid transferred block, while Cancel will imply to restart from the very beginning of the transfer (first block).
  • QueryAlreadyFinished : stands for special code where the Request is in fact remotely already finished and so can be finished locally.
  • Other codes specifies different kinds of error (NotKnownHost, Shutdown, RemoteError, ...): 
  • • ConnectionImpossible : connection is impossible (local or remote)
    • BadAuthent : authentication is in error (local or remote)
    • ExternalOp : external operation is in error (pre, post or error actions)
    • TransferError : transfer is in error (for instance, uncorrect block)
    • MD5Error : transfer is in error due to a bad MD5 checking (in MD5 mode)
    • Disconnection : the network was disconnected
    • RemoteShutdown : the remote host is in shutdown
    • Shutdown : the local host is in shutdown
    • FinalOp : a final operation (not an external operation) is in error (for instance the finalization of the received file)
    • Unimplemented : the requested command is unimplemented (for instance, a task in a rule is unknown)
    • RemoteError : a remote error was received and stops the transfer
    • Internal : an internal error arrives
    • Warning : an external execution (pre or post operation) is executed well but with a warning (code 1)
    • Unknown : an unknown error arrives
    • QueryStillRunning : a required transfer during a restart is still in transfer
    • NotKnownHost : the remote host is unknown
    • QueryRemotelyUnknown : a required transfer during a restart or other commands should exist but is not identified on the remote host (for instance after a purge of log)
    • FileNotFound : the file is not found
    • CommandNotFound : the Command is not found 
    • PassThroughMode : the requested transfer (restart) is in PassThrough mode and is not compatible with the required action

A request of transfer follows a sequential logic:

1. The request is registered in the database with a TOSUBMIT status.
2. A Request that was in a INTERRUPTED status is changed in a TOSUBMIT status.
3. The Commander get some requests with the TOSUBMIT status and makes them as RUNNING status.
4. The Commander submits those requests as separates ClientRunners.
5. The ClientRunner first checks that the given Request is not a "Self Requested" request, meaning that only requester host can execute a ClientRunner, except if this request was in the POSTTASK step so that Requested Host can finalize the request.
6. The ClientRunner gets the remote requested Host address and tries to open the connection. If a network conection with the given requested host is already opened, this network connection is reused by the new ClientRunner.
7. The connection can use SSL support (different port than non SSL). This is an option of transfer. This option is selected while selecting the Host ID for SSL support for the remote Requested Host. This option is CPU and Memory consuming.
8. Once the network connection is found, a private connection (in memory connection) is opened to enable the multiplexing of this request with other requests on the same network connection. This private connection is attached to a new LocalChannelReference which references the Request, the session, the remote and private connections. A valid LocalChannelReference contains two private connection Ids, one for the local private connection, and one for the remote private connection.
   1. A Startup Message is sent to the local private connection to initiate it.
   2. The same Startup Message is sent on the remote private side (when connection occurs) to initiate the relation between them and to instantiate the same LocalChannelReference on the remote host.
9. Once the LocalChannelReference is OK, the Requester host sends an Authent Message in order to authenticate this host.
10. The Requested host sends back its own Authent Message too.
11. Once authenticated, the Requester Host sends the Request Message.
12. The Requested Host check if the authentication and the request are compatibles, check some specific options on the request itself (start, restart, ...), then it runs its Pre Task step

    • The Request is in PreProcessingOK as status.

13. If the Pre Task step is ok, it sends back the validated request to the Requester Host.
    • The Request is in InitOk as detailed information on requester side.
14. The validated Request is now running the Pre Task step.
    • Once finished, the Request is in PreProcessingOK status.
15. Now the transmission can start. From now, this is the sender that leads the communication, no more the requester (it could be the same host however, depending on the way of transfer). The sender (which could be either the Requested or the Requester host) launch its own RetrieveRunner. This RetrieveRunner sends to the other host all DataBlock Messages.
16. Each DataBlock Message can include a Hash control (MD5 or SHA or other) of the packet in it (option of transfer). This option is not mandatory and is CPU consuming.
17. Once all DataBlock are sent, the RetrieveRunner sends an EndTransfer Message to the receiver host.
18. The receiver host executes the Post actions.

    • The Request is in PostProcesseingOk status.

19. The receiver host sends back the validated EndTransfer Message.
    • The Request is in TransferOk status on sender side.
20. The Request is now on Finalize way. Sender host executes the Post actions.
    • The Request is in PostProcesseingOk status.
21. Once the PostProcessing is over, the RetrieveRunner (Sender) sends to the remote host a EndRequest Message.
22. The remote host sends back the validated EndRequest Message.
23. The Request is now totally finished and its status is CompleteOk ALLDONE.

At each step, an error can occurs and will stop the request, setting its UpdatedInfo to INERROR or INTERRUPTED status. The GlobalStep could be in ERROR status if the ERROR step action is run.