A Scalable Deployment Service for PlanetLab
Built on the CoDeeN content distribution network
CoDeploy provides a means to
efficiently and scalably distribute content from one source to many
receivers. In practice, what this means for PlanetLab is that it
allows you to push content to hundreds of PlanetLab nodes without
having to consume lots of bandwidth at the source. In general,
these techniques can be used for efficient peer-to-peer hosting of
arbitrary content. CoDeploy uses a
number of techniques to perform this distribution efficiently, such
CoDeploy is intended to be simple to
use, requiring very little infrastructure and installation. It
requires only two programs on the node initiating the push, and all
of the other tools are already in place on PlanetLab. It works as
CoDeploy consists of a number of components that work together to accomplish the tasks described above. The components, in brief, are described here.
Follow these steps to install CoDeploy:
CoDeploy is a fairly simple program to use. It takes three arguments, which are basically the local directory name, the URL corresponding to that directory, and the remote directory name.
For example, assume you have a local directory ~/public_html/program that is also available via your web server as http://www.example.edu/~tupac/program and that you want to put this directory (and all subdirectories) into your PlanetLab slice in the directory test. You would execute the following command on a single line
codeploy ~/public_html/program http://www.example.edu/~tupac/program test
If you want to optimize performance on the first deployment, specify the -a flag to codeploy. This flag will cause the directory to be tar'd and compressed. This optimization may be useful for the initial deployment. However, on future updates, when only a few files may be modified, using CoDeploy without flags is preferred.
Low-level details: You may find that CoDeploy creates temporary directories in your directory tree. This approach is necessary to handle several cases. When executable files are in your web tree, some web servers are configured to execute the file instead of transferring the file. CoDeploy uses the temp directory to make a non-executable copy of the file for download. Likewise, files that have a very recent modification time may be uncacheable by the CDN, so CoDeploy makes a copy of the file with an older modification time.
The CoDeploy package also includes two other programs, multiquery and multicopy. These can be used independently to perform other operations on the machines in the node list. Multiquery can be used to perform simple shell operations, and multicopy can be used to transfer files.
For example, to remotely create a directory and execute a program
in the background, you can issue the following command:
multiquery 'mkdir results; directory/program &'
Note that the quotes around the commands indicate that all of them should be executed on the remote node
The multicopy command can be used to copy files to nodes, or to
copy files from nodes. In this program, the @ parameter gets
expanded to the name of a remote machine if specified alone. If
specified with a colon suffix, it gets replaced with the slice name
and machine name. The command
multicopy filename @:directory
copies the specified file to the specified remote directory on all nodes
To grab files from a remote subdirectory and copy them locally to
directories corresponding to the remote machine names, execute the
multicopy newdir @
multicopy '@:directory/*' @
The first multicopy creates directories corresponding to the remote machine names. The second multicopy command copies the files from the remote machines to their corresponding local directories.
More details regarding the options for multiquery and multicopy can be found in the comments at the top of the multiquery.c source file. These options allow various behaviors, such as changing the degree of parallelism in the number of copies, the delay between launching successive queries, the timeout, etc.
CoDeploy and all related tools are currently regarded as being in beta test. While we strive to ensure availability of the service 24 hours a day, 7 days a week, we may encounter outages when the service is being upgraded. CoDeploy does involve a number of new components, and while we are confident enough in their development to allow open usage, we may still encounter bugs. We ask for your understanding, and equally importantly, your feedback about the service.
with help from Aki Nakao
We may collectively be contacted at
princeton_codeen at slices.planet-lab.org