Distributed Rendering: Life on the Farm

I have an old friend from college who grew up in a farm house built during Shakespeare’s lifetime in Shrewsbury (Shropshire, England). A long time (over a decade) ago I was on the phone with him during my workday, explaining that I had time to chat because I was rendering. His reply was that rendering consisted of recycling pig and poultry carcasses. I wonder what he would make of a “render farm.”

A question came up about the excellent Nucleo Pro, which distributes renders across the processors in your multi-proc (and/or multl-core) machine. What about when you have procs distributed across a network and want your own render farm, a place where images grow on their own while you continue to work (or even sleep)? Before Nucleo there was X-Factor, which shipped with After Effects and created RAM Previews with network machines (an idea which turned out to be impractical, leading the product to be discontinued).
As many readers know there are several third-party solutions to help distribute an After Effects (or other 2d compositing or 3d animation) renders across a network. This has been possible in After Effects since version 6.0, which added automated rendering as part of the Scripting feature set. But Adobe doesn’t provide the means to do it; instead, they and other animation software developers provide the command-line “hooks” needed to let third-party software do it.

Thing is, this software, at least initially, isn’t so easy to install or maintain, and it tends to be larger studios with dedicated IT support who have the knowledge that it even exists, let alone how to implement it (hint: it’s not generally a matter of double-clicking an installer, instead involving setting up a domain controller on a centralized server and optimizing the network for speed and load-balancing).

This article from over a year ago lists some of the options available at that time and describes some of what is involved with implementing them. Since that time, Deadline from Frantic Films has appeared on the scene, which among other things apparently obviates the need for a server, but which is, alas, Windows-only. Also not listed from the Mac-only side is Qmaster, developed by Apple specifically for distributing Shake renders (but allegedly capable of distributing other applications’ renders as well – well, Maya, at least).

Thinking about this stuff revives a fantasy I had when the Mac Mini first appeared: get a bunch of those things with a little extra RAM and a gigabit switch and build a little farm right in my own studio.

I’ve heard others express this idea as well but haven’t encountered an actual implementation, and I’d love to hear (in the comments, perhaps?) from anyone who has created a cheap and easy (sub-$10K) render farm in a small studio, whether Mac or Windows based.

18 January 2007 | after effects | Comments

4 Responses to “Distributed Rendering: Life on the Farm”

  1. 1 Jonas Hummelstrand 18 January 2007 @ 2:53 pm

    Hi Mark,

    A year ago I managed to secure three dual-proc Windows workstations that I figured I would setup for Maya and After Effects renderings. After failing to get the IT department to open up the internal firewall, I gave up and we bought 12 seats of Nucleo Pro instead. Your link led me to look at Dr. Queue which claims it supports AE and also runs on OS X. Perhaps your stack of Mac Minis could look like this:

    – Jonas

  2. 2 Jonas Hummelstrand 18 January 2007 @ 2:59 pm

    Ahh, the lost link in the last comment showed a nice screen shot of about 20 macs rendering the same project…

  3. 3 Jonas Hummelstrand 19 January 2007 @ 12:12 am

    Here’s a new link to your renderfarm-to-be:
    http://www.fxguide.com/article323.html&mode=nested

    :-)

  4. 4 David Miller 9 April 2007 @ 11:31 pm

    Hi Mark,

    I used GridIron XFactor to drive 4 matching IBM x-series rackmountable e-servers I purchased used on eBay for an After Effects 6.5 render farm. Although I rarely used the setup for RAM previews, it was extremely useful for long project renders, such as those you might attempt using the techniques described in Stu Maschwitz’s recent book. Renders that had taken days were often accomplished overnight–plus, I was able to use my workstation for other things while the render farm was at work.

    In fact, this setup was so successful that I purchased 4 additional eservers on eBay, based on Dave McClelland’s assurance that a version of XFactor that would be compatible with After Effects 7 would soon be released (my $495 XFactor license supported 8 machines). The e-servers were purchased for a few hundred each, making the total cost well under your $10K theoretical limit. Sadly, GridIron pulled the plug on the product shortly thereafter without actually releasing a version that was compatible with AE7.

    Although Nucleo Pro is nice for RAM previews, the ability to send a long project out for final rendering was a godsend. The Deadline software you mention from Frantic Films is an intriguing alternative (and it’s compatible with AE7), but it’s over twice as expensive as the GridIron license was. Still, it may be worth a look.

    David

google

google

asus