GoldenGate OpenR66

The GoldenGate OpenR66 project is a proposal of a file transfer monitor for real production environment. It supports send and receive file transfer, multiple simultaneous file transfer, transfer through support (for instance from a web server routed through OpenR66 to another server), multiplexing network connection (only one network socket is used between two servers), pre- post- or error- transfer action (several possibilities including execution of external scripts), MD5 packet validation (optional), SSL support (optional), bandwidth limitation both at global (all sockets) or local (one socket at a time) (optional), Web supervision included, simple database support (H2, Oracle, PostGreSQL or MySQL databases are supported) that allow to enhance supervision or production handling, heavy or light clients where clients can only initiate transfers (send or receive) but do not accept a new transfer from external servers (heavy clients are using a database connection, while light clients are not using at all databases, useful for simple PC for instance).

A new possibility of High Availability and horizontal scalability are proposed, based on an external Load Balancer on TCP basis (as HA-Proxy) and a Shared Storage (like a NAS). The idea is to load-balance the R66 file transfer monitors as a same server (sharing both database, storage and in front through the load balancer, the same IP and port). It does not prevent a transfer to be in error if one of the servers goes down, but does allow a quick restart.

Dependency

It is based mainly on the NETTY framework (NIO great framework support at http://www.jboss.org/netty and is tend to be really efficient, both in term of memory, threads and network bandwidth.

Specificities

Bandwidth limitation can be configured both in store and retrieve, per session (although only one value is set by default, but it can be changed dynamically if needed) or globally to the server and of course with no limitation at all if wanted. Limitation should be enough to change the bandwidth behavior for instance depending on the time in the day, so as to allow to limit bandwidth usage when users are at work and in contrary to allow more speed when only batchs are running.

The specificity of this project is you can adapt this software to your particular needs by changing the pre or post action (even error actions) on transfers (not only transfer).

GoldenGate Main Project Logo

See global GoldenGate project at http://openr66.free.fr/ and particular OpenR66 project at http://openr66.free.fr/OpenR66.html

Benchmarks

Some pre benchmarks were done on PC and a Power5 server (12 CPU). In every benchmarks, the java used memory was less than 200 Mbytes, the CPU was always less than 4 CPU by JVM (server or client). The tested file has a size of 1,12 Mbytes.

  • 100 synchronous clients using one multiplexing network connection: 32 transmissions by second or 286 Mbits (with or without MD5)
  • 1000 asynchronous transfers (submitted work) using one multiplexing network connection: 59 transmissions by second or 527 Mbits
  • 1000 synchronous clients using 10 multiplexing network connection (100 synchronous clients by network client): 70 transmissions by second or 627 Mbits