working still image renderfarm script.

The interface, modeling, 3d editing tools, import/export, feature requests, etc

Moderators: jesterKing, stiv

Posts: 0
Joined: Tue Feb 25, 2003 2:37 pm

working still image renderfarm script.

Post by ideasman »

Now blender has AO, raytracing etc, some renders are taking longer (though still fast)

To yse run the folling scripts from the command line with the latest CVS bf-blender since this relys on a python function I added for this script. ( context.setBorder() )

blender myscene.blend -P

This script is working but needs some intergration.
All the tiles need to be composited, I can do this using PIL but need to make a script that composites a directory of images.

It renders an image as many tiles (128px x 128px default)
Each tile is indexed and blender only renders if there are mo existing image of that index (a dummy is created before the render so computers dont render the same tile)

This requires all PC's to be able to accsess a networked filesystem.

if the file has a num at the end eg.
blender myscene_256.blend -P
Then a 256 x 256 tile will be used.

Here is

Code: Select all

import Blender
from Blender import sys, Scene, Noise
from Blender.Scene import Render
import string

NUM = '0123456789'

# Clamp
def max(number):
	if number > 1.0:
		number = 1.0
	return number

def isfile(file):
		return 1
		return 0

def randList(list):
	randList = []
	lenList = len(list) 
	while lenList != len(randList):
		randIndex = int( Noise.random() * len(list)  )
		randList.append( list[randIndex] )
		list.remove( list[randIndex] )
	return randList

# Strip all chars after a '.'
def stripExt(text):
	return text[:text.index('.')]

def getParts():
	name = Blender.Get('filename')
	name = stripExt(name)
	# Check that there is a number at the end of the filename.
	if name[-1] not in NUM:
		# Work out the tile size from the file name.
		tileSize = ''
		while name[-1] in NUM: # Keep pulling in the numbers as long as they are nums.
			tileSize = name[-1] + tileSize
			name = name[0:-1]
		tileSize = eval(tileSize)
	# OK either way we have a tile size.
	# Get the data
	scn = Scene.GetCurrent()
	context = scn.getRenderingContext()
	xPix = context.imageSizeX()
	yPix = context.imageSizeY()
	xParts = int(xPix / tileSize)
	yParts = int(yPix / tileSize)
	# Incase the tile size is larger then the render size.
	if xParts < 2:
		xParts = 2
	if yParts < 2:
		yParts = 2	
	return xParts, yParts

# Makes a list of rects that setBorder will pass.
def makeBorderList(xparts, yparts):
	borderRectList=[] #We store all the rects here and then return them. 
	xlen = 1.0 / xparts
	ylen = 1.0 / yparts
	xPoint = 0.0 # This stores the current X value, and incriments xlen each iteration until its equel to 1.0 
	yPoint = 0.0
	counter = 1 # Inde each border
	while xPoint < 0.999:
		while yPoint < 0.999:
			# Write the rect to the list
			borderRectList.append( (counter, max(xPoint), max(yPoint), max(xPoint+xlen), max(yPoint+ylen)) )
			counter += 1 # Keep a tag of which index this one is.
			yPoint += ylen
		# Reset yPoint for the next colum.
		yPoint = 0.0
		xPoint += xlen
	return borderRectList

scn = Scene.GetCurrent()
context = scn.getRenderingContext()
context.enableRGBAColor() # Save RGBA
context.setImageType(Render.PNG) # Save RGBA

# Make image name
imageName = Blender.Get('filename')

# Remove .blend
imageName = stripExt(imageName)
renderName = imageName + '_' # frameNum.png will be added.

xParts, yParts = getParts()

randBorderList = randList( makeBorderList(xParts,yParts) )

# Set the start and end frame to the current frame.
curFrame = Blender.Get( 'curframe')

# Keep track of frames rendered, only for a report.
renderedFrames = 0

for border in randBorderList:
	# Set the new file name WITH X/Y parts
	# blendfilename_partnum_framenum.ext
	# eg. render_01_0001.png
	partNum = str(border[0])
	while len(partNum) < 4:
		partNum = '0' + partNum
	uniqueRenderName = renderName + partNum + '_' # We add 1 so that the first image is 1, not 0
	frameNum = str(curFrame)
	while len(frameNum) < 5:
		frameNum = '0' + frameNum
	fileToRender = uniqueRenderName + frameNum + '.png'
	# Chech that the file isnt alredy there
	if isfile(fileToRender) == 0:
		#Create a dummy file so no other nodes try to render the image.
		file = open(fileToRender,"w")
		context.setRenderPath('//' + uniqueRenderName) # // is the currentdir
		# Set border
		context.setBorder(border[1], border[2], border[3], border[4] )  
		context.renderAnim() # This saves the pics.
		renderedFrames += 1

# Print report
print 'XParts', xParts,' YParts', yParts
print renderedFrames, ' of ', len(randBorderList), ' rendered'

# Quit
Last edited by ideasman on Tue May 25, 2004 1:23 am, edited 1 time in total.

Posts: 0
Joined: Thu Apr 08, 2004 4:17 pm

working still image renderfarm script

Post by anders_gud »

Has anyone tried this script with Xgrid? For stills...

Posts: 500
Joined: Wed Oct 16, 2002 5:47 am
Location: Montreal

Post by theeth »

Can you edit your first post to put the [ code ][ /code ] tags properly please?

Life is what happens to you when you're busy making other plans.
- John Lennon

Posts: 0
Joined: Tue Feb 25, 2003 2:37 pm

Post by ideasman »


I was just thinking...

It would not be hard to do this (mabe time consuming though)

Have a button/menu item that activates a python script.

This script copys a blend file to a remote location that other PC's on the internet can accsess. (FTP/WEBDAV/RSYNC)

Computers all over the internet render tiles of the pic.

As each tile as compleated it is downloaded and removed from the net and composited into the base image (empty file is kept for as a place holder, so no other PC's render it.)

- This would make blender MUCH more attractive to users who wanted to do raytracing renders/preview renders.

It would also be good to use some DIFF type uploading system (like RSYNC??) So if the user makes changes to their blend file, it can be updated quickly and re-rendered without re-uploading all the data in the file that hasent changed.

Posts: 0
Joined: Wed Jan 07, 2004 12:54 pm

Post by scourage »

Another Idea-

The master computer has a screen that shows the status of all the computers in the the renderfarm. The status would be something simple like a line with each nodes address, frame number that it's working on, time elapsed, and ping time (for internet use). The lines would update as each node completes a frame through a message system between nodes and master.

I don't know if it could be entirely run in python, it might require some code to be written in blender for the messaging system.

just an idea

here's a mock-up picture


Also, Do the particle generators use random number generators? If they do, do the random number generators need to be synced with the master?


Halfway down the trail to hell....

Posts: 0
Joined: Tue Feb 25, 2003 2:37 pm

Post by ideasman »

Teah this could be done but Id would be be done after intergrating the script into blender.

Posts: 0
Joined: Tue Nov 18, 2003 3:37 am

Post by macouno »

Are you going to keep this python or are you truly integrating it with blender?

Posts: 0
Joined: Tue Feb 25, 2003 2:37 pm

Post by ideasman »

I think I'll just keep it python. It could be added into a python menu- Or a gui could be made so users can easerly activate it.

Posts: 0
Joined: Thu Mar 13, 2003 11:29 pm

Post by jd-multi »

Can you post a tutorial how to get that renderfarm working? Like: Do I have to open blender on 2 or more computers? Or do I need to run the script on 2 computers? Do I have to enable sharing over network to share the .blend file? Please give some more info on how to set up a working renderfarm. I think many people would be interested in that. :P

Posts: 0
Joined: Tue Feb 25, 2003 2:37 pm

Post by ideasman »

Tonight I will add some features which allow the user to configre the renderfarm from within blender, than a tutorial.

Posts: 0
Joined: Sat Mar 22, 2003 10:45 pm

Post by JA-forreal »

This is great ideasman! Thanks.

It would also be nice if were possible to render a still file by assigning this job out to as many cpu's that you have in your computer for AO renders.

Posts: 0
Joined: Tue Feb 25, 2003 2:37 pm

Post by ideasman »

Good point. Do you know how to get python to know how many CPU's the PC has?

Failing that it could be configured within blender, this would also be good for hyperthreading.

- Cam

Posts: 0
Joined: Sat Mar 22, 2003 10:45 pm

Post by JA-forreal »

ideasman wrote:Good point. Do you know how to get python to know how many CPU's the PC has?

Failing that it could be configured within blender, this would also be good for hyperthreading.

- Cam
Sorry, I don't know how to get python to access CPU threads in a PC. I like the speed increase that we get with this option for rendering in Yafray in Blender. I guess that it is just a matter of time before someone gets this working for Blenders raytraced renders.

Posts: 0
Joined: Sat Nov 13, 2010 1:43 pm

Post by tarapozan »

In what language that code is written ? I familiar with c++ but it do not like a c++.
Last edited by tarapozan on Sun Jun 12, 2011 12:28 pm, edited 6 times in total.

Posts: 0
Joined: Tue Aug 05, 2003 7:58 am
Location: 45N 86W

Post by stiv »

Blender uses an extended version of Python for its scripting language.

Post Reply