Please note: All new versions will be located at GitHub under Waarp new project name :

http://waarp.github.com/Waarp/

The project still remains Open Source under GPL V3.

Limit CPU / Connexion

With these options (usecpulimit), OpenR66 Server can limit the new requests according to a threshold on CPU global usage and on a maximum number of concurrent requests. When one of these limits is exceeded, the request is refused and postponed for a random number proportional to 30 seconds (option timeoutcon). After 3 retries, the request is cancelled.

These tests are done both on requester and requested side.

• cpulimit: Value for CPU is in float between 0 and 1 (% of CPU) where 0 or 1 means no limit.
• usejdkcpulimit: CPU is computed either with native support from JRE (but not all JRE support this) or either from Java Sysmon library.
• connlimit: Value for connexion is starting from 0 where 0 means no limit.

 

Check of IP on servers and clients

By default, real IP address used by the remote host is not compared against the IP (or the resolution given from DNS) stored with the remote HostId. If for security reasons this is required, you can enable this check to be done. When activated in checkaddress:

• For a server, the test will always be done.
• For a client, it depends if you enable it through the specific option checkclientaddress, then 2 cases occur:
  • If the client has an address of "0.0.0.0", then no test is done (this special address is to be used when a lot of remote clients are to be used without high security challenge).
  • If not, then the check is done as for a server.

 

Host as Client

In version 2.0, a new property is attached to the Host definition: isclient. This property stands to recognize remote client to prevent a server to try to initiate a request to this client. Since this client is not a server, it is not listening to incoming requests and therefore cannot be the target of an incoming request.

The special address "0.0.0.0" can also be used to specify a client but it extends it to not try to check its IP address. It is useful in the situation where you have a lot of clients and you don't want to declare them all in the Host table. All those clients will have to share the ID and the password (and the SSL key in case of strong authentication).

 

Cryptographic support

In version 2.0, many improvements were done on cryptographic side.

1. SSL: You can now have simple SSL support or even using string authentication of clients using the trustuseclientauthenticate option.
2. Password: all passwords are crypted using a private DES Key (private to each server but can be the same if you want). The passwords are crypted both on files, database. This key is locally referenced by cryptokey option. You have to use the GoldengatePassword tool (see GoldenGate Commons) to crypt your passwords (at least for the administrator password) or to use the administration web interface.
3. On the network, a single way crypto key is used, common to all OpenR66 hosts when they need to exchange passwords.

 

Store Task saving on XML file for Thin Client

The special option taskrunnernodb allows you to have a persistent view of the transfer (the task) for Thin Client without database by using a XML file. It allows to restart a stopped or canceled transfer smoothly without starting a new transfer from the beginning.

 

Control on restart transfer

Although an interrupted request restarts and each received packet should have been previously validated, the protocol include a backward move on the packet rank in order to ensure the quality of the transfer without doubt. When a transfer is restarted, OpenR66 first checks the receiver rank and takes it as the reference minus a gap. Then it checks the existing file (reception side) and check again against this rank minus a gap. You can control this feature by specifying the block size (blocksize) and the gap of rank (gaprestart), where the retransmitted byte size will be: block size x rank.

 

 

Usage GoldenGate LocalExec Daemon

In order to improve the efficiency of external commands execution by preventing the fork of the OpenR66 JVM process (costly regarding its memory), there is an optional support to execute those commands through a GoldenGate LocalExec Daemon (see in GoldenGate Commons). To use this support, you have to define the following:

• uselocalexec (default="False") to enable the use of LocalExec support
• lexecaddr (optional) which could contains the address (default="127.0.0.1")
• lexecport (optional) which could contains the port to be used (default="9999")

 

Usage of FastMD5 support

In order to improve the efficiency of the computation of MD5 on blocks during file transfer, 3 methods are provided.

1. usefastmd5=False => Will use internal JDK support of MD5 computation
2. usefastmd5=True but fastmd5 not declared or empty => Will use Java MD5 implementation more efficient than JDK version
3. usefastmd5=True and fastmd5=path to SO file or DLL (according to the systems) => Will use JNI C MD5 implementation suppose to be more efficient than version 2.

However note that the results could depends according to the systems and the JDK. Most of the time, Version 3 is the fastest, but sometime Version 2 is better than Version 3, and always better than Version 1. The efficiency of Version 3 greatly depends on the compilation of the C module. See GoldenGate Digest in GoldenGate Commons

 

It appears with recent versions of JVM that JDK in server mode is really efficient, almost equivalent to C JNI version. So one might use by default usefastmd5=False.

 

Note also that a new option can specify which kind of digest you want to use (it is for now a global option, not a local option): digest where values mean: MD5=0, MD2=1, SHA1=2, SHA256=3, SHA384=4, SHA512=5, CRC32=6, ADLER32=7.

 

Usage of Multiple Monitors support

In order to improve reliability of the OpenR66 File Transfer Monitors and the scalability, we propose a new option that allows to spread the load behind a Load Balancer in TCP mode (as HA-Proxy) and a shared storage (as a simple NAS).

1. multiplemonitors=1 => No multiple monitors will be supported (single instance) 
2. multiplemonitors=n => n servers will be used as a single instance to spread the load and increase the high availability 

Note that some specific attentions are needed such as to share the IN, OUT and WORK storages such that any servers can act on those files and any other storages that must be shared from the beginning of the transfer (pre-task) to the end of the transfer (post-task), and as to configure correctly the Load Balancer in TCP mode such as to spread the load and keep the connection once opened between 2 partners.

 

Usage of No Database for Server

For special project where no database is needed, but then loosing the ability to store all transfer status and associated capacity (such as automatic retry if not specified in the rules), the server code now supports to not have any database connections. However it is still possible to store the trace of the transfers within XML files, such that one can automatize some actions in regard to the status of each transfer through file analysis. Note however that some functionality like the monitoring will be limited. The option taskrunnernodb will allow to use XML files support if set to true. If set to false, no information will be retained out of memory of the process. 

 

Usage of ExecJava class

In order to facilitate the integration in application modules, OpenR66 now supports the ability to run specific Java Class through 3 ways. Note that this functionality is only valid starting in version 2.3.

• One is through pre or post or error tasks using the EXECJAVA keyword, following by the full class name which must implement the R66Runnable interface.

• Another one is through specific R66Business command, which will also execute an R66Runnable implementation, through for instance the AbstractExecJavaTask abstract class that could be extended.

• Finally, there is the possibility to associate a Business Class (see R66BusinessInterface) through a Business Factory (see R66BusinessFactoryInterface) to each transfer that will run several methods in the various steps that could occur:

  • void checkAtStartup(R66Session session): launched at the very startup of the transfer and before the pre commands

  • void checkAfterPreCommand(R66Session session): launched after the pre commands and before the transfer starts

  • void checkAfterTransfer(R66Session session): launched after the transfer is finished and before the post commands

  • void checkAfterPost(R66Session session): launched after the post commands and before the end of the request

  • void checkAtError(R66Session session): launched once an error occurs

  • void checkAtChangeFilename(R66Session session): launched if the filename is changed during the commands (pre or post)

  • void releaseResources(): launched at the very end, to release any internal resources that should be released

  • String getInfo() and void setInfo(String info): launched by programmatic (business code) to enable to set a special info (as String) and to retrieve it at any time.

• Note that to allow a host to call a Business Request, it has to be added in the configuration file as <business><businessid>hostname</businessid>...</business>. If not set, the host will not be allow. On EXECJAVA, the security is first that the rule is only local to the host, and second the rule has the possibility to limit the allowed hosts to be partner of it.