working still image renderfarm script.

The interface, modeling, 3d editing tools, import/export, feature requests, etc

Moderators: jesterKing, stiv

ideasman
Posts: 1000
Joined: Tue Feb 25, 2003 2:37 pm

working still image renderfarm script.

Postby ideasman » Fri May 21, 2004 3:26 am

Now blender has AO, raytracing etc, some renders are taking longer (though still fast)

To yse run the folling scripts from the command line with the latest CVS bf-blender since this relys on a python function I added for this script. ( context.setBorder() )

blender myscene.blend -P render_client.py

This script is working but needs some intergration.
All the tiles need to be composited, I can do this using PIL but need to make a script that composites a directory of images.

It renders an image as many tiles (128px x 128px default)
Each tile is indexed and blender only renders if there are mo existing image of that index (a dummy is created before the render so computers dont render the same tile)

This requires all PC's to be able to accsess a networked filesystem.

if the file has a num at the end eg.
blender myscene_256.blend -P render_client.py
Then a 256 x 256 tile will be used.

Here is render_client.py

Code: Select all

import Blender
from Blender import sys, Scene, Noise
from Blender.Scene import Render
import string

NUM = '0123456789'
DEFAULT_TILE_SIZE=64 # px

# Clamp
def max(number):
   if number > 1.0:
      number = 1.0
   return number

def isfile(file):
   try:
      open(file)
      return 1
   except:
      return 0


def randList(list):
   randList = []
   lenList = len(list)
   while lenList != len(randList):
      randIndex = int( Noise.random() * len(list)  )
      randList.append( list[randIndex] )
      list.remove( list[randIndex] )
      
   return randList

# Strip all chars after a '.'
def stripExt(text):
   return text[:text.index('.')]

def getParts():
   
   name = Blender.Get('filename')
   name = stripExt(name)
   
   # Check that there is a number at the end of the filename.
   if name[-1] not in NUM:
      tileSize = DEFAULT_TILE_SIZE
   else:
      # Work out the tile size from the file name.
      tileSize = ''
      while name[-1] in NUM: # Keep pulling in the numbers as long as they are nums.
         tileSize = name[-1] + tileSize
         name = name[0:-1]
      
      tileSize = eval(tileSize)
   
   
   # OK either way we have a tile size.
   # Get the data
   scn = Scene.GetCurrent()
   context = scn.getRenderingContext()
   
   xPix = context.imageSizeX()
   yPix = context.imageSizeY()
   
   xParts = int(xPix / tileSize)
   yParts = int(yPix / tileSize)
   
   # Incase the tile size is larger then the render size.
   if xParts < 2:
      xParts = 2
   if yParts < 2:
      yParts = 2   
      
   return xParts, yParts
   

# Makes a list of rects that setBorder will pass.
def makeBorderList(xparts, yparts):
   
   borderRectList=[] #We store all the rects here and then return them.
   
   xlen = 1.0 / xparts
   ylen = 1.0 / yparts
   xPoint = 0.0 # This stores the current X value, and incriments xlen each iteration until its equel to 1.0
   yPoint = 0.0
   
   counter = 1 # Inde each border
   
   while xPoint < 0.999:
      while yPoint < 0.999:
         
         # Write the rect to the list
         borderRectList.append( (counter, max(xPoint), max(yPoint), max(xPoint+xlen), max(yPoint+ylen)) )
         counter += 1 # Keep a tag of which index this one is.
         
         yPoint += ylen
      # Reset yPoint for the next colum.
      yPoint = 0.0
      
      xPoint += xlen
   return borderRectList




# SETS UP DEFAULTS NEEDED FOR OUTPUTTING AN IMAGE THAT CAN BE COMPOSITED.
scn = Scene.GetCurrent()
context = scn.getRenderingContext()
context.enableBorderRender(1)
context.enableRGBAColor() # Save RGBA
context.setImageType(Render.PNG) # Save RGBA
context.enableExtensions(1)

# Make image name
imageName = Blender.Get('filename')

# Remove .blend
imageName = stripExt(imageName)
renderName = imageName + '_' # frameNum.png will be added.





xParts, yParts = getParts()

randBorderList = randList( makeBorderList(xParts,yParts) )


# Set the start and end frame to the current frame.
curFrame = Blender.Get( 'curframe')
context.startFrame(curFrame)
context.endFrame(curFrame)

# Keep track of frames rendered, only for a report.
renderedFrames = 0

for border in randBorderList:
   
   # Set the new file name WITH X/Y parts
   # blendfilename_partnum_framenum.ext
   # eg. render_01_0001.png
   partNum = str(border[0])
   while len(partNum) < 4:
      partNum = '0' + partNum
   uniqueRenderName = renderName + partNum + '_' # We add 1 so that the first image is 1, not 0
   
   # CREATE THE REAL NAME OF THE OYTPUT FILE
   frameNum = str(curFrame)
   while len(frameNum) < 5:
      frameNum = '0' + frameNum
   fileToRender = uniqueRenderName + frameNum + '.png'
   
   
   # Chech that the file isnt alredy there
   if isfile(fileToRender) == 0:
      
      # TOUCH FILE SO NOBODY OVERWRITES IT.
      #Create a dummy file so no other nodes try to render the image.
      file = open(fileToRender,"w")
      file.close()
      
      
      # SET RENDER NAME AND PATH.
      context.setRenderPath('//' + uniqueRenderName) # // is the currentdir
      
      # Set border
      context.setBorder(border[1], border[2], border[3], border[4] ) 
      
      # RENDER THE IMAGE
      context.renderAnim() # This saves the pics.
      
      renderedFrames += 1
      

# Print report
print 'XParts', xParts,' YParts', yParts
print renderedFrames, ' of ', len(randBorderList), ' rendered'

# Quit
Blender.Quit()
Last edited by ideasman on Tue May 25, 2004 1:23 am, edited 1 time in total.

anders_gud
Posts: 24
Joined: Thu Apr 08, 2004 4:17 pm
Contact:

working still image renderfarm script

Postby anders_gud » Mon May 24, 2004 3:07 pm

Has anyone tried this script with Xgrid? For stills...
http://www.blenderman.org/tutorials/xgrid/

theeth
Posts: 1184
Joined: Wed Oct 16, 2002 5:47 am
Location: Montreal
Contact:

Postby theeth » Mon May 24, 2004 3:44 pm

Can you edit your first post to put the [ code ][ /code ] tags properly please?

Martin
Life is what happens to you when you're busy making other plans.
- John Lennon

ideasman
Posts: 1000
Joined: Tue Feb 25, 2003 2:37 pm

Postby ideasman » Tue May 25, 2004 1:35 am

done-

I was just thinking...

It would not be hard to do this (mabe time consuming though)

Have a button/menu item that activates a python script.

This script copys a blend file to a remote location that other PC's on the internet can accsess. (FTP/WEBDAV/RSYNC)

Computers all over the internet render tiles of the pic.

As each tile as compleated it is downloaded and removed from the net and composited into the base image (empty file is kept for as a place holder, so no other PC's render it.)

- This would make blender MUCH more attractive to users who wanted to do raytracing renders/preview renders.

It would also be good to use some DIFF type uploading system (like RSYNC??) So if the user makes changes to their blend file, it can be updated quickly and re-rendered without re-uploading all the data in the file that hasent changed.

scourage
Posts: 92
Joined: Wed Jan 07, 2004 12:54 pm

Postby scourage » Thu Jun 10, 2004 4:24 am

Another Idea-

The master computer has a screen that shows the status of all the computers in the the renderfarm. The status would be something simple like a line with each nodes address, frame number that it's working on, time elapsed, and ping time (for internet use). The lines would update as each node completes a frame through a message system between nodes and master.

I don't know if it could be entirely run in python, it might require some code to be written in blender for the messaging system.

just an idea

here's a mock-up picture

Image

Also, Do the particle generators use random number generators? If they do, do the random number generators need to be synced with the master?

Cheers,

Bob
Halfway down the trail to hell....

ideasman
Posts: 1000
Joined: Tue Feb 25, 2003 2:37 pm

Postby ideasman » Thu Jun 10, 2004 4:54 am

Teah this could be done but Id would be be done after intergrating the script into blender.

macouno
Posts: 257
Joined: Tue Nov 18, 2003 3:37 am

Postby macouno » Sat Jun 12, 2004 1:10 pm

Are you going to keep this python or are you truly integrating it with blender?

ideasman
Posts: 1000
Joined: Tue Feb 25, 2003 2:37 pm

Postby ideasman » Sat Jun 12, 2004 5:33 pm

I think I'll just keep it python. It could be added into a python menu- Or a gui could be made so users can easerly activate it.

jd-multi
Posts: 123
Joined: Thu Mar 13, 2003 11:29 pm

Postby jd-multi » Tue Jun 15, 2004 6:00 pm

Can you post a tutorial how to get that renderfarm working? Like: Do I have to open blender on 2 or more computers? Or do I need to run the script on 2 computers? Do I have to enable sharing over network to share the .blend file? Please give some more info on how to set up a working renderfarm. I think many people would be interested in that. :P

ideasman
Posts: 1000
Joined: Tue Feb 25, 2003 2:37 pm

Postby ideasman » Wed Jun 16, 2004 1:32 am

Tonight I will add some features which allow the user to configre the renderfarm from within blender, than a tutorial.

JA-forreal
Posts: 187
Joined: Sat Mar 22, 2003 10:45 pm

Postby JA-forreal » Wed Jun 23, 2004 1:35 am

This is great ideasman! Thanks.

It would also be nice if were possible to render a still file by assigning this job out to as many cpu's that you have in your computer for AO renders.

ideasman
Posts: 1000
Joined: Tue Feb 25, 2003 2:37 pm

Postby ideasman » Wed Jun 23, 2004 2:42 am

Good point. Do you know how to get python to know how many CPU's the PC has?

Failing that it could be configured within blender, this would also be good for hyperthreading.

- Cam

JA-forreal
Posts: 187
Joined: Sat Mar 22, 2003 10:45 pm

Postby JA-forreal » Wed Jun 23, 2004 11:33 pm

ideasman wrote:Good point. Do you know how to get python to know how many CPU's the PC has?

Failing that it could be configured within blender, this would also be good for hyperthreading.

- Cam


Sorry, I don't know how to get python to access CPU threads in a PC. I like the speed increase that we get with this option for rendering in Yafray in Blender. I guess that it is just a matter of time before someone gets this working for Blenders raytraced renders.

tarapozan
Posts: 1
Joined: Sat Nov 13, 2010 1:43 pm

Postby tarapozan » Sat Nov 13, 2010 1:45 pm

In what language that code is written ? I familiar with c++ but it do not like a c++.
[size=0]astigmatism[/size]
Last edited by tarapozan on Sun Jun 12, 2011 12:28 pm, edited 6 times in total.

stiv
Posts: 3646
Joined: Tue Aug 05, 2003 7:58 am
Location: 45N 86W

Postby stiv » Sat Nov 13, 2010 3:49 pm

Blender uses an extended version of Python for its scripting language.


Return to “Interface & Tools”

Who is online

Users browsing this forum: No registered users and 0 guests