Blender integration with LSF cluster and DrQueue

The interface, modeling, 3d editing tools, import/export, feature requests, etc

Moderators: jesterKing, stiv

grimey
Posts: 0
Joined: Tue Nov 15, 2005 7:55 pm

Blender integration with LSF cluster and DrQueue

Post by grimey »

Anyone interested/know anything about Blender working on an LSF cluster? I'm trying to get Drqueue setup now. I'm a windows/dos guy so its happening quite slowly, so any help there would be great. But I heard that Maya has an LSF implementation.

I thought Blender might have a script or something to do this from the command line that would work with the LSf platform with modification. Maybe even install belnder on the cluster and run it command line...I'm still working on understanding the LSF cluster stuff myself.

BTW, the LSF cluster I have is 128 node, dual core, dual cpu (2.8 GHz) = 512 cpu's (effectively)...not too shabby :shock:

an-toni
Posts: 0
Joined: Wed Mar 17, 2004 5:20 pm

Post by an-toni »

what is that platform? we are using an xseed, works nicely - and drqueue in the local lan at the studio.

we control the farm (which is on another continent) on irc :)

~Toni

grimey
Posts: 0
Joined: Tue Nov 15, 2005 7:55 pm

Post by grimey »

We using platform LSF HPC http://www.platform.com/Products/Platfo ... m.LSF.HPC/
Still getting up the learning curve on that!

The Maya LSF implementation looks very nice
http://www.alias.com/eng/partners/condu ... _lsf.shtml
It would be nice to have soemthing like that for Blender. Seems like a scriptable kind of thing.

I don't know much about Xseed. How fast is your system? Ours is something like 2.2 Tflops, hoping to play a *quick* game of BF2 or sumthing.

grimey
Posts: 0
Joined: Tue Nov 15, 2005 7:55 pm

Post by grimey »

Just to keep you developers posted on this matter...

If I understand LSF correctly, it appears that the LSF platform is SSI (single system image) with automatic process migration. This means that using the supplied compiler, the program is automatically distributed across the cluster.

I thought maybe some of you had some experience with this platform and could give me some tips. So apparently, then DrQueue is not needed. I don't have to copy Blender across multiple nodes...blahblahblah. I will keep you posted as how things develop. Be looking for my posts over in the Coding forum! HaHa, I've programmed C/C++ in Windows but this is my first linux and definitely my first cluster.

an-toni
Posts: 0
Joined: Wed Mar 17, 2004 5:20 pm

Post by an-toni »

does it need multiprocessing or *gasp* multithreading the things you run to be able to share the load among the nodes in the cluster, or can it do something intelligent with a single-threaded app (i doubt, but you never know, and i did not look!)

~Toni

MicWit
Posts: 0
Joined: Thu Jan 19, 2006 10:59 am
Location: Australia

Post by MicWit »

I dont know much about clustering etc, however I have some P4's with hyperthreading. When using blender it has been a pain trying to use the entire cpu, as blender doesnt support multi-cpu rendering (as far as I can tell), and although yafray is supposed to support it, it seems to work some times but not others (weird). This means that it may be a pain trying to get the full cpu usage out of each computer, but the render farm should still work fine in dr que, but then it depends on the cluster support, as an-toni said. I havent had a chance to use dr que yet, but it is a program that as soon as I find time, I will get running, prob in windows, but maybe with 1 or 2 computers on linux. Let me know how you go anyhow.

Sorry I couldnt help you more,
MicWit

LetterRip
Posts: 0
Joined: Thu Mar 25, 2004 7:03 am

Post by LetterRip »

Blender is multithreaded for rendering you just need to turn it on. The button 'Threads' in the Output panel of the Render button.

LetterRip

an-toni
Posts: 0
Joined: Wed Mar 17, 2004 5:20 pm

Post by an-toni »

is that still limited to a low number of threads now, though? our machines have only two processors each.

~Toni

LetterRip
Posts: 0
Joined: Thu Mar 25, 2004 7:03 am

Post by LetterRip »

an-toni - limited in the interface, you have to change it to a higher thread number in the code right now, should be changable in the future via preferences or better interface in the render dialog.

LetterRIp

MicWit
Posts: 0
Joined: Thu Jan 19, 2006 10:59 am
Location: Australia

Post by MicWit »

LetterRip,

Thanks heaps for that, I never knew the feature was added, now dont need yafray for now.

Thanks,
MicWit

grimey
Posts: 0
Joined: Tue Nov 15, 2005 7:55 pm

Post by grimey »

LetterRip,

You don't happen to know where in the code do you? Or perhaps a link?

My plan is to implement DrQueue for the jobs I need to get done now. The pain with DrQ is you need blender on each node. Well, LSF has options so that you put blender in one directory and it takes care of the rest, that's the process migration feature.

This is my first cluster programming. So far, it's like my admin got me a date with Pamela Andersen (or, insert favorite model's name).

BoomBadaBing!

LSF is amazing.

LetterRip
Posts: 0
Joined: Thu Mar 25, 2004 7:03 am

Post by LetterRip »

4 threaded rendering achieved by setting maxthreads=4 in source/blender/render/intern/source/pipeline.c

http://bebop.cns.ualberta.ca/~cwant/multi-render.html

LetterRip

LetterRip
Posts: 0
Joined: Thu Mar 25, 2004 7:03 am

Post by LetterRip »

grimey,

you using 2.41 or CVS?

LetterRip

MicWit
Posts: 0
Joined: Thu Jan 19, 2006 10:59 am
Location: Australia

Post by MicWit »

well, I am just about over trying to get dr queue running in windows XP. I have tried various hacks etc Ive found in the dr queue forums, but just cant get it running. Would it be easier in suse? Is there a version that is installable in windows without as many issues? else is there a different render farm that will be easier to get running?

thanks,
Michael

grimey
Posts: 0
Joined: Tue Nov 15, 2005 7:55 pm

Post by grimey »

LetterRip,
I'm planning to use 2.41 on DrQ implementation. Does CVS offer something better?

MicWit,
yeah, winxp is problematic with DrQ, some have got it running though. That forum is not too active though so it may be soem time before you get any feedback or assistance. I would switch to linux if you can. If you only have a few machines to parallel, you can always manually program them using command line. I've done that until now.

Post Reply