How to render a Blender-Movie on a supercomputer with 150,000 cores

Helmut Satzger

Sunday 27 October Theater 30

I work at the Leibniz Supercomputing Centre in Munich, Germany, and have access to SuperMUC, currently still the 9th fastest computer in the world, with more than 150,000 processor cores, 300TByte RAM and 10 PByte harddisk - and it runs the standard Blender from blender.org.

In 2011, we started working on a stereoscopic animation "What is supercomputing?" (there was a talk about it at last years Blender Conference). The animation has about 27,000 frames and we are still tweaking it, which means we constantly have to render parts of it. Since I do not have exclusive access to SuperMUC, I cannot run the usual renderfarm software or network rendering solutions.

I will show the current status of the animation, but mostly talk about obvious approaches for "massive parallel rendering with cycles" and why they do not work and will present a very elegant "massively parallel-complete-auto-renderer" that:

  • is based on a very simple Google document (one line per .blend-file)
  • uses a redis database to distribute rendering-jobs to an arbitrary number of worker nodes (thats the cool part!) and
  • needs just three python scripts with less than 200 lines of code (combined) to manage everything

This approach:

  • is totally cool: You just start, add or remove generic workers; they register themselves and get to it right away
  • is fault tolerant, i.e. it doesn't matter if some of the Blender instances crash (which they do, more often than I like)
  • doesn't care if the frame-list or list of scenes changes at any time - it updates the database every couple of minutes
  • is inherently parallel, i.e. works with any number of workers from 1 to 100,000
  • can add/remove worker nodes at any time, i.e. you can add resources at will
  • has been used on SuperMUC to render the movie with thousands of parallel workers (and 32 threads per worker)