renderfarms... well

General discussion about the development of the open source Blender

Moderators: jesterKing, stiv

macouno
Posts: 0
Joined: Tue Nov 18, 2003 3:37 am

renderfarms... well

Post by macouno » Wed Apr 14, 2004 8:14 pm

Ok

I just want to get it clear in my head. There's been lots of talk about renderfarms and opengl 2.0 hardware acceleration and all that. And I'm trying to get my head round it, but it's giving me headaches.

First off... how about a list of working network rendering software. I heard about drque but that's non windows I believe... and how about mosix??? and what works for blender internal and what for yafray? Does anyone actually have a working renderfarm for blender? (lemme guess... on linux?)

Also in most projects I encounter I only hear talk about distributing sepparate frames to be rendered across a network.

We have a Parts function in the blender renderer... how about distributing these parts over the network so that people can benefit from this development when rendering stills as well??

leinad13
Posts: 192
Joined: Wed Oct 16, 2002 5:35 pm

Post by leinad13 » Thu Apr 15, 2004 11:41 am

Because using the parts system would mean messing around with the Blender source code: To do what your syaing would invovle this:

Part of the render pipeline needs to be passed to another computer, this would mean in big scenes movings hundreads of mb's of data, which isnt a good idea.

OR

You could send the .blend file and then have the other computer just render the parts you want it too. This sounds like the best idea, but im stil not sure exactly how the 'parts' bit of the renderer works.
-------------
Over to you boffins

L!13

macouno
Posts: 0
Joined: Tue Nov 18, 2003 3:37 am

Post by macouno » Thu Apr 15, 2004 11:52 am

well you might be able to make blender on the networked puters access the file through the network... not sending it or anything. Just a straight access... might be heavy on the network but since most home networks are at least 10mbps if not 100 that should be ok.

IoN_PuLse
Posts: 212
Joined: Wed Oct 16, 2002 6:05 am
Location: Canada, BC
Contact:

Post by IoN_PuLse » Thu Apr 15, 2004 5:48 pm

OpenMosix can take any app (this is how I understand it) and split it's processing over a bunch of computers. To get it done faster.

I have NO idea what OpenGL 2.0 would have to do with this.

jesterKing
Site Admin
Posts: 207
Joined: Fri Oct 18, 2002 12:48 pm
Location: Finland

Post by jesterKing » Thu Apr 15, 2004 11:43 pm

IoN_PuLse wrote:OpenMosix can take any app (this is how I understand it) and split it's processing over a bunch of computers
This only provided that the app is using threads. (more than one :)

/jesterKing

macouno
Posts: 0
Joined: Tue Nov 18, 2003 3:37 am

Post by macouno » Fri Apr 16, 2004 12:33 am

This is what it says on the openmosix site:

openMosix is a Linux kernel extension for single-system image clustering which turns a network of ordinary computers into a supercomputer.


Which is nice but linux... though there's quite a few linux users that use blender... there's still more that use windows.

I mentioned OpenGL 2.0 because both renderfarms and hardware rendererers like the Art Pure card are ways to speed up your rendering... thus related topics in my book. Whilst one may choose to look in the direction of renderfarms another may want to upgrade their one system. of course maybe we can do both at the same time in the future.

I'd just like to know what my choices are right now... which seems to be... if you're using blender on windows... forget renderfarms and hold your breath for at least a year in the hope that OpenGL 2.0 comes out... right? wrong? I do hope I'm wrong.

joeri
Posts: 96
Joined: Fri Jan 10, 2003 6:41 pm
Contact:

Post by joeri » Fri Apr 16, 2004 4:49 pm

So I think it's about time that someone or someones write a document stating what a blender renderfarm should look like.

Thoughts should go about:
Rendering in background of current frame while working. (in parts or not)
Rendering an animation on multiple machines (& OS's)
Rendering over a intranetwork and over internet. (buddies? trustees?)
What to do with packed / unpacked files.

macouno
Posts: 0
Joined: Tue Nov 18, 2003 3:37 am

Post by macouno » Fri Apr 16, 2004 5:51 pm

Sounds nice.

What I've seen so far from the drque system seems promising, though it'll only work for animations.

it would be nice if something were integrated in blender internally though I doubt that's really practical, whilst perhaps it would be nessecary to adapt some of blenders internal code to make data appropriately available to external programs.

The way things seem to work well is... a small application that you run on "all" your systems which basicly listens for jobs, and fires up blender in command line fashion when a "master" version of that software tells them to. This seems to now respond to placing a blender file in a certain folder. Perhaps a window in which you can start/stop/que jobs as in the blender file renderer would be best.

Perhaps even the blender file renderer program could be extended itself to work in a networked environment??? (that may well be the best direction to think in since the code already works)

Another idea perhaps??? but more a question... Would it be possible to port blender's render data to RIB (renderman) so blender could be made to function with the Art Pure card and such??? But there's probably licensing involved in that, so I doubt that's an option for an open source app.

dj2
Posts: 0
Joined: Thu Jan 02, 2003 3:41 pm

Post by dj2 » Fri Apr 16, 2004 11:20 pm

There is also the network-renderer project in the projects section.

macouno
Posts: 0
Joined: Tue Nov 18, 2003 3:37 am

Post by macouno » Sat Apr 17, 2004 7:51 pm

I think I see 2 of them actually, but 1 has 0 work done and the other very little... are people still actually working on them???

dj2
Posts: 0
Joined: Thu Jan 02, 2003 3:41 pm

Post by dj2 » Sun Apr 18, 2004 9:40 am

i know there has been work done on the network renderer. as i've done some of it. Goofster actaully just did a re-org of its directory structure the other day and changed some of the code around. i don't know about the other renderer tho.

gloume
Posts: 0
Joined: Tue Jan 28, 2003 1:44 am
Contact:

Post by gloume » Sun Apr 18, 2004 7:57 pm

I actually read through a site describing OpenMosix + Blender once. It seems that mosix has little to do with multiple threads, and more to do with multiple processes. It was described like this:

You have a render farm of mosix enabled linux boxes and a master server. You start up multiple blender processes (from the command line)on the master server, each rendering a single frame. What mosix does is monitor which processes are taking up a lot of cpu time and migrates them to other nodes on the mosix network that are idle or aren't doing much. I guess each node has access to a networked filesystem and they all drop frames in the same directory. What is really neat is that, if cpu cycles are freed up on one node because it has finished rendering its frame, other nodes that have a heavy load can migrate processes to it. So you can mix fast boxes and slow boxes and the network will balance itself out.

Statik
Posts: 0
Joined: Fri Apr 16, 2004 2:05 pm
Location: Nova Scotia Canada
Contact:

Post by Statik » Mon Apr 19, 2004 12:53 pm

Maybe I'm misunderstanding something, but if you click on the Verse link on blender.org (should be to the left of this post for most/all of you, doesn't that link you to working network rendering software?

Statik

macouno
Posts: 0
Joined: Tue Nov 18, 2003 3:37 am

Post by macouno » Mon Apr 19, 2004 1:54 pm

As I understand it verse would allow multiple people to work on the same file... so you add a cube to a scene... then someone else on another workstation can add something to that same scene, whilst he gets your cube... that sort of thing... it's not network rendering as I understand it.

And mosix... if I was running linux, I'd have installed it.

macouno
Posts: 0
Joined: Tue Nov 18, 2003 3:37 am

Post by macouno » Thu May 06, 2004 4:00 pm

Something else I'd like people to consider.

I can get my hands on a really top of the line P4 3.4 gigaherz machine for around 1200 euros.

But I can purchase 3 crappy Celeron 2 gigaherz machines for the same money. (rough calculation)

If we had renderfarm capacity then which do you recon is quicker? Also... expanding the farm later on, 1 system at a time, would be much cheaper than upgrading your initial expensive monster pc. This is really why I believe network rendering is the way to go... and the sooner the better! I rechecked the progress of the existing bits in here but there seems to be little or nothing being done.

Post Reply