|Software :: crcp — cooperative remote file copy|
The distribution of a file to a large number of hosts is a time consuming task, even if the size of the file is modest. Consider, for instance, the scenario of a company that needs to distribute a virus update of 4 MB to all of the 100'000 machines in its Intranet. When using a single server with a bandwidth capacity of 100 Mb/s and 90% link utilization, file distribution takes almost 10 hours.
Using cooperative methods where each client that already has downloaded the file serves another client, this time can be reduced to approximately 1 minute (even less with elaborate distribution protocols). Unsurprisingly, the improvement factor becomes significantly higher as the size of the file increases to tens or hundreds of megabytes.
Cooperative remote file copy (crcp) is a cooperative version of the well-know
rcp program. It was developed as part of a research project and must thus be considered
as unstable software. It provides middleware that can be used not only to quickly replicate
files on a set of remote machines, but also to test and deploy new distribution protocols. At the
end of the distribution process, statistics are returned to the source to allow developers to
evaluate the performance of their protocols. The most efficient protocol is called
For more information on program usage, use the
Documentation is currently very minimal. Some information is available in the
Before installing crcp, make sure you have installed the
./configure --prefix=<installation-dir> --with-libevent=<libevent-dir> make make install
The program is statically linked against
The crcp program can be started in source or peer mode. The source drives the whole content distribution process. The peers listen to the source commands and actively participate to the distribution process.
The source can optionally launch remote peers via
You can test crcp by executing the following steps:
The command-line options, as of version 0.8.4, are listed below:
crcp -- cooperative remote file copy Usage: crcp [options...] local remote... crcp [options...] local @file crcp [options...] Arguments: local Local path name of the form `file://<path>' remote Remote path name of the form `ssh://<user>@<host>:<port>/<path>' or `tcp://<host>:<port>/<path>' @file File containing a list of remote path names <no argument> Wait for incoming file Options: --silent --fatal --error --warn --info --debug Print no, little, or much information (default=warn) -C, --program <string> (default="crcp") Path to the crcp program to start remotely using ssh -H, --host <string> (default="localhost") Local host -P, --port <int> Local port (default=9999) -S, --ssh "option" Option passed to ssh, e.g., -S "-2" (one ssh option per -S) -T, --timeout <int> Timeout before shutting down when there is no activity (default=60 seconds) -V, --version Print version number -d, --daemon Run program as deamon (peer only) -h, --help Print this message -p, --protocol <string> Protocol to use for content distribution (default="parallel") -s, --size <int> Define chunk size (default=16384 bytes) -x, --extra "args" Extra protocol-specific arguments Valid protocols: parallel linear tree ptree
If you any have questions, suggestions, or comments, please do not hesitate to contact us.