General Info

By: James Reynolds - Revised: 2006-05-24 devin

General Info

Learn the basics of Xgrid including parallel vs. serial tasks, Xgrid plumbing, security, and components, and details about Xgrid's agent, controller, and client.


What is Xgrid?

Xgrid allows you take a task and execute it on many many computers. It is similar to Pooch and AppleSeed.

Pro's of Xgrid.
  • Easy to setup
  • Easy to secure
  • Easy to create custom jobs
  • No C required
  • Easy "workstation" or "dedicated node" setup
  • Free (can't say that about the other stuff)
Con's of Xgrid.
  • Can't do MPI (very common for other grids) for now (3rd parties can create plug-ins that will allow communication with other computers, but Xgrid does not offer that)
  • Mac OS X only for now (3rd parties can create other Unix versions)
  • Too basic for some jobs

Parallel vs. Serial Tasks

Xgrid works best with parallel tasks. The other type of task is a serial task. Xgrid does not do those well. An example of a serial task is the Fibonacci sequence.

The Fibonacci sequence is composed of a series of numbers. Each number is a sum of the 2 numbers preceding it. To compute a value, you must have the results of the last 2 jobs. To have the results for those jobs, you must have the results of the jobs before it. In other words, you have to start from the beginning and just work it out serially.

Parallel tasks, on the other hand, can be split up. Computer graphics is a perfect example. If you have a black background and a blue square that starts at points (5,5) and ends at points (10,10), you know that (5,5), (8,6), and (5,10) are all inside of the square, and thus would be colored the blue and not black. But (4,4), (5,11), and (11,6) are outside of the square and are the color of the background, black. You don't need to calculate (1,1) to (5,5) to know any of that.

A few other parallel tasks include Monte Carlo calculations and genome sequencing. Apple provides a few job examples, including Mandelbrot & BLAST. It is also easy to create tasks if you know how to use the command line or write scripts.

Xgrid Plumbing

Xgrid's role is the plumbing, not any actual math. It does not calculate anything, know anything about calculating, have content for calculating, or even know that you are calculating anything. All it does is send jobs (that you create or download) to the grid and return the results. The actual job is composed of software that either is on the computers already (like perl or another shell interpreter), can be installed before running the job, or the required software can be transferred to the computers by Xgrid.

The Xgrid plumbing uses the BEEP protocol for communication. Apple wrote their own Objective-C Cocoa framework. Here is a list of BEEP features:
  • Standard application layer that supports dynamic, pluggable application "profiles" (protocols).
  • Peer-to-peer, client-server, or server-to-server capabilities.
  • Multiple channels over a single authenticated session.
  • Support for arbitrary MIME payloads.
  • Java, C, and Tcl (Xgrid uses Tcl).

Xgrid Security

By default, Xgrid is setup to use passwords so that nothing happens without a password. Only allowed people can run jobs on the grid and grid computers only accept jobs that have the proper password. Of course passwords can be turned off, but that isn't the greatest idea in the world. Currently, there is no way to restrict which computers can accept jobs (hopefully this will change).

Xgrid uses a 2 way random mutual authentication protocol that includes MD5 hashes. There is no encryption on data transfer, but it may be added in a future release if there is demand. Actually, the underlying BEEP implementation does support SSL/TLS for encryption, but currently there is no way to activate it.

Also, there is only one computer on a grid that has a listening port.

Xgrid Security for Users

If you are not a grid manager, but are considering allowing your computer to be a part of someone else's grid, the next few paragraphs apply to you. But you should know how Unix users and permissions work or else it wont make sense.

Xgrid jobs run as the user "nobody" on grid computers. That means they can read/write/execute where the "world" is allowed. That means that Xgrid jobs can execute all commands in /bin, /sbin (except init), /usr/bin, and /usr/sbin. The jobs can write to places like /tmp and /Volumes. And jobs can read files like /etc/authorization, /etc/hostconfig, /etc/hosts.allow, and /etc/httpd/httpd.conf.

None of this is a major security risk, but this isn't something you would like to allow anyone to do. Information can be obtained to help perform a serious attack against the computer. Or a bogus Xgrid job could be used to bring the computer to a crawl, or a job could just fill up the computer hard disk making it slow and eventually unusable without some serious fixing.

So, Xgrid job security is excellent if you want to allow your computer to run jobs from a trusted associate, coworker, computer administrator, or even 3rd party developer (think Seti@Home). But the job security is not good enough (and never can be, really, and it shouldn't have to be) if you are going to run jobs of a complete stranger from a country where computer crimes are common and unpunished.

In a sense, running an Xgrid job is very similar to downloading and running a 3rd party application off the web. There must be a level of trust that you have with a 3rd party application developer. You would not run an application from an untrusted source (at least you shouldn't). Likewise, you shouldn't ever allow your computer to be a part of a grid that you don't trust. Application developers must be accountable for what their programs do on your computer, and grid managers must be accountable for what they run on your computer.

Xgrid Components

There are 3 Xgrid roles. A computer can play one role, two, or all three types at once.
  • Agent - Or the workers. They get jobs from the controller (one task per CPU).
  • Controller - Or the boss. Remembers jobs, gives them to agents to perform, hands results back to client.
  • Client - Or the customer. Submits jobs to the controller, views progress, gets results back. Also can create jobs.

Agent Details

  • GUI: System Preferences, Xgrid prefpane, Agent tabs.
  • Executable: /Library/StartupItems/GridAgent/GridAgent.
  • Finds controller via Rendezvous; or uses specified IP.
  • There can be many agents (over 500 at least), all over the world (local subnet probably best though).
  • By default requires a password from the controller before it will run jobs.
  • Two availability modes: 1). always available or 2). only available when idle for 15 minutes or if Xgrid screensaver is active.

Controller Details

  • GUI: System Preferences, Xgrid prefpane, Controller tabs.
  • Executable: /Library/StartupItems/GridServer/GridServer.
  • One per grid. There can be more than one grid on a subnet, and if that is the case, if agents use Rendezvous to find controller then they bind to the first controller to give the right password.
  • Listens on port 4111 (might change in a future release) and other non-privileged ports and broadcasts via Rendezvous.
  • Can be set up to give a password to the agents.
  • Can be set up to require a password from the client.

Client Details

  • GUI & Executable: /Applications/Xgrid.app or /Applications/Xgrid BLAST.app.
  • Can be set up to run in local mode, which will turn on the GridAgent and GridServer on your local computer.
  • Finds controllers via Rendezvous; or uses specified IP.
  • Asks user for password (to give to controller).
  • Creates "Plug-ins", saved as tar format (in a plist file), which are uploaded to the controller, which are then downloaded to the agents.
  • Receives results back.
  • Can monitor job status.