Blender-Python Board, gurus & coders needed

Scripting in Blender with Python, and working on the API

Moderators: jesterKing, stiv

ton
Site Admin
Posts: 525
Joined: Wed Oct 16, 2002 12:13 am
Contact:

Blender-Python Board, gurus & coders needed

Postby ton » Sat Nov 02, 2002 3:19 pm

In the past, with each new developer coming in, we were confronted with a new vision on how Python should be implemented in Blender.

This resulted in incompatibility among releases, and a non-uniform API (game engine vs. rest of Blender). Causing quite some frustrations with our users...

We're slowly catching up with development now, and one of the issues is how we will organize the total complexity in Blender. Anticipating on the general discussions on this topic, I think Python can be organized itself as a separate project in Blender.

We can establish an official Python Board, overseeing the project and defining targets, functionality and general architecture. People in the board should at least include experienced users of Blender as well as hard boiled Python coders. Issues to look at:

- keeping it cross platform (new python library versions don't always run well at all platforms)
- how to solve backwards compatibility issues? Are we going to live with the current situation, slowly improve, or radically (again!) go for a change?
- a complete new API? A feasible one, to target for a 2.5 for example? That should be done in cooperation with the other projects of course.
- sandboxed version for web plug-in
- installation issues. When functionality moves out of Blender, into Python modules, it will need a more sophisticated installation procedure

One of the concepts that were lying around, is using Python to replace the 'plugin' system in Blender (textures, sequence effects). Try to imagine a 'shading language' in Blender!
We also wanted to have import/export fully implemented in Python, including integrated (pulldown) menus to access modules. This work has been nearly completed.

Anyone? Ideas, feedback? Thanks!

-Ton-
Last edited by ton on Wed May 07, 2003 1:07 pm, edited 1 time in total.

markluffel
Posts: 13
Joined: Sun Oct 20, 2002 12:16 am
Location: Georgia Tech, Atlanta
Contact:

Ideas

Postby markluffel » Mon Nov 04, 2002 6:31 am

To prevent incompatible releases of the Python/Blender API we could store past version of the interface in sub-modules. The top-level "Blender" module could store all current implementation and the sub-modules could simply be wrappers for the current functions that made sure the parameter count was correct, the current scene was passed, or any other issue that changes warranted. These sub-modules could appear in the form "Blender.v225". Then a simple change of "from Blender import *" to "from Blender.v225 import *" would protect against (some) change. (A quick regex could also change all "Blender.*" into "Blender.v225.*" references.) It should be a priority to ensure that all scripts written in this way continue to work from one version to the next. A number of test scripts containing all previously existing functionality should be tested each release so that any breaks can be found before they affect the end user.
Some future releases may have their python interface placed in a vXXX sub-module before the actual release, allowing developers to see where the interface is headed and to give the python developers a visible indication of the gaps they need to bridge in the next month.

Ton, what do you think of the discussion about "embedding vs extending"?

ton
Site Admin
Posts: 525
Joined: Wed Oct 16, 2002 12:13 am
Contact:

Re: Ideas

Postby ton » Mon Nov 04, 2002 3:24 pm

markluffel wrote:
Ton, what do you think of the discussion about "embedding vs extending"?


Can you elaborate that?

xitnalta
Posts: 68
Joined: Tue Oct 15, 2002 11:52 pm
Location: Romanshorn (TG), Switzerland

Re: Ideas

Postby xitnalta » Mon Nov 04, 2002 7:43 pm

ton wrote:
markluffel wrote:Ton, what do you think of the discussion about "embedding vs extending"?

Can you elaborate that?

There's some discussion going on (in another thread) about having most of Blender's functionality just be a Python module (which would lead to somehow wiring the interface to Python, but I'm not sure whether and where I've read about that).

This link contains some more discussion on this topic: http://twistedmatrix.com/users/glyph/rant/extendit.html (seen in the thread where it got discussed).
Felix

slikdigit
Posts: 213
Joined: Wed Oct 16, 2002 3:52 am
Location: Northampton, MA (US)

Postby slikdigit » Mon Nov 04, 2002 8:55 pm

Some feedback as to doing a 'shading language' in blender, using python functionality:
This would be fabulous! It would be a huge benefit to be able to wire parameters into shaders, via python. Many cool things that are not possible today in the renderer would be enabled (like faking translucency by wiring light angle/intensity into a ramp (blend) shader)
As for creating a new vs. slowly improving the existing api: its probably good to do both, i.e. keep existing api with a new one next to it, so existing scripts can continue to run while the new api is developed (is this similar to the blender 210 api?), thus postponing the breaking of existing scripts untill the future has been stabilized.
I'm not a hardcore python programmer, so I don't know how the current state of things is, and what needs improving. Therefore I can't say whether its better to be radical or evolutionary at this point.

theeth
Posts: 1184
Joined: Wed Oct 16, 2002 5:47 am
Location: Montreal
Contact:

Postby theeth » Tue Nov 05, 2002 3:46 am

About keeping old API for compatibility issues, I think that the solution adopted with Jan's API (better known as Blender210) would really be appropriate in this situation. What I mean is, let the module there, but under a different name (probably keep the name scheme name + version number), that way, it is still accessible and easy to adapt old scripts (example: import Blender210 as Blender) while having the advantage of not importing duplicated functions (Publisher 2.25 API).

As for Python replacing the current plugins API, I'm all for it, even if that should make some unhappy. What would be best is a dual C/Python api, although that could be achieved with only a Python module, using the Python/C interface to speed math oriented peice of code. All in all, I think it's a really good idea, as Python is IMHO, a lot more flexible and easily codable than C (a really debatable topic).

If ILM can use it in their in house project, why can't we? :wink:

If you really need people to form some kind of board, you can count on me. I have directly access to a Win XP comp and daily (at school) access to a Linux (Debian) and OS X box, so I could test cross platform issues directly.

Martin
Life is what happens to you when you're busy making other plans.
- John Lennon

markluffel
Posts: 13
Joined: Sun Oct 20, 2002 12:16 am
Location: Georgia Tech, Atlanta
Contact:

GameLogic incorporation

Postby markluffel » Wed Nov 06, 2002 8:01 am

I've been thinking today about the division between the GameLogic and Blender API's - looking for similarities and what differences remain when the similarities are merged.
The most obvious similarities are the Blender.Draw module and GameKeys modules, then the Actuators and DataBlock accessors. Actuators and Sensors are both just high-level abstractions of DataBlock access and Draw.Register routines.

I hope we create a single API, splitting it into modules containing high and low level functions, both available in GameBlender and NormBlender.
Modules:
/User Interaction
\Sensors
/Logic
-Dynamics
\IPO
/Actuators
\DataBlock Manipulation

I imagine that this synthesis would make animation playback more similar to running a predetermined GameBlender scene. The possible collision between IPO data and physics/python constraints would require some toggle between the two (or is that three) modes. I suggest expanding the role of the constraints tab to function as the determinant of how an object's data is handled. The animation buttons, realtime buttons, and dynamics section of the materials buttons should be called what they properly are: ipo constraints, logic constraints, and physical constraints. Some sort of conversion between these, especially logic to ipo would be incredibly useful. Imagine it as baking animation onto an object from a script defined ruleset. It would need to do this by sampling at frames and fitting the samples to some ipo curve, but is quite possible.

The big thing here is imagining the three constraints as all being possible sources of control in GameBlender and NormBlender.

Also some grumbling:
I don't like the way the Blender API has been divided into modules that enclose classes of single DataBlock type. Though it mirrors the way that Blender stores the objects internally, a python class can be similar enough to a module. Why not just make the "Blender.Camera" module into a class and make functions like "New" (which should be "new") and "get" static members of the class? If there is a good reason for this please explain it.

bootsch
Posts: 11
Joined: Mon Nov 11, 2002 10:40 pm

Love to join - need more Blender experience

Postby bootsch » Mon Nov 11, 2002 11:02 pm

Hi Ton,

I'm currently researching the RIB export from Blender - on OSX.
Running in to bunch obstacles - I guess the incompatibility between
version.

(my 2.23.b-01 doesn't know about Scene and currentScene for
example - can't build a new 2.25 version on OSX yet)

But I do have experience coding python and 3d, and the
Renderman shading language b.t.w.

I would love to see such a thing for Blender - I can recommend
it to all hardcore texturing users.

A complete I/O scheme sound perfect too! I'd love to try it.

Again - I'll work and try to get things running before I'll
give my opinion on how to implement python

:wink:

Good luck,
too y'all

Bootsch

ton
Site Admin
Posts: 525
Joined: Wed Oct 16, 2002 12:13 am
Contact:

Postby ton » Wed Nov 13, 2002 5:48 pm

cool!
Just use this forum to communicate on the topics for a while, in the meantime I am discussing with our volunteers the integration of sourceforge at this site.

wahn
Posts: 15
Joined: Fri Nov 15, 2002 10:27 pm
Location: London
Contact:

Re: Blender-Python Board, gurus & coders needed

Postby wahn » Fri Nov 15, 2002 10:57 pm

Hi,

If I would have more spare time I would definitively help with a new Python API. I took it very serious at the time I was working for NaN. But I think NaN did also a great job to confuse the people about the Python API :(

I don't have the time anymore to do the coding (because I earn my money now with writing shaders and plugins for commercial packages). Anyway if someone would be interested to take over the Python coding (or better integrating a Python API) I would help to do the first steps. I allways had the idea that the user should see an object-oriented view of Blender. But most of the code is plain C (except the game engine and the collision detection - Thanks to Erwin & Gino). But if you read the manuals and if you look at the user view of the data in Blender a object-oriented API would be the best. Anyway that's what I tried to do ...

Unfortunately the Python API is also not consistent between the game engine and the rest of Blender. Erwin and me wanted to work on that but there was just no time left before NaN went bancrupt the first time. And I had other problems to survive in London :twisted:

A few days after I left the API was gone (or hidden behind "import Blender210 as Blender") and I was just tired of explaining why the new (old - whatever) API was not working anymore. Now it's a good time to have a new start. But as I said: I don't have the time for it ...

Maybe it would be also a good idea to make the Python integration version (the version of Python) independent. Maybe link shared libraries from a local installation? This would be a good thing to ask at the Python user group how to achieve this ...

Anyway, if someone has questions and wants to take over to create a new (better) Python API feel free to email me: wahn@acm.org

I wish you all a lot of fun with Python and the source should be with you ... :wink:

Cheers,

Jan

3djones
Posts: 5
Joined: Tue Nov 12, 2002 5:56 pm
Location: Las Vegas, NV
Contact:

the role of python

Postby 3djones » Sat Nov 16, 2002 2:12 am

three interesting links that have me wondering about C/python integration:

Interview with Guido
http://www.onlamp.com/pub/a/python/2002 ... guido.html

Interview with Bruce Dawson
http://www.onlamp.com/pub/a/python/2002 ... nnews.html

psyco
http://psyco.sourceforge.net/

Guido mentions that python in the future will generate machine code instead of byte-code.
Bruce discusses a game engine that uses python extended with C instead of C extended with python.
psyco is a python performance booster - very cool - try it!

this all boils down to python scripts approaching or exceding machine code speed.

so i say the more python the better, although i admit i haven't much experience with compiled languages like C - and frankly when i read about the future of python and use psyco, i don't feel like i missing much.

if the Bruce Dawson paper is any indication then large chunks of Blender can be writtin in python with the real performance intensive stuff handled by C/C++. the interface, animation system, plug-ins, FX, etc could all be coded in python...

just a thought - gotta run now but i'd like to do what i can for the effort - i've been lucky enough to use blender all day every day at my job for the past year so i can certainly test python code. i have access to win2k and os X

:)

mmelo
Posts: 17
Joined: Sun Oct 20, 2002 11:20 pm
Contact:

Re: the role of python

Postby mmelo » Mon Nov 18, 2002 10:30 am

3djones wrote:this all boils down to python scripts approaching or exceding machine code speed.


Well, I don't know about that bit "exceding machine code speed". :lol:

By its definition code can only go as fast as "machine code" will. And very seldom does compiled code go faster than hand-coded assembly, anyway. Unless it makes cache/alignment/latency optimisations that the asm coder did not think about.


Miguel

3djones
Posts: 5
Joined: Tue Nov 12, 2002 5:56 pm
Location: Las Vegas, NV
Contact:

Re: the role of python

Postby 3djones » Tue Nov 19, 2002 7:26 pm

mmelo wrote:
3djones wrote:this all boils down to python scripts approaching or exceding machine code speed.


Well, I don't know about that bit "exceding machine code speed". :lol:

By its definition code can only go as fast as "machine code" will. And very seldom does compiled code go faster than hand-coded assembly, anyway. Unless it makes cache/alignment/latency optimisations that the asm coder did not think about.


Miguel


i see your point - :oops: somewhere in the links i listed was a blurb about exceding machine code speed - perhaps i was a conceptual point; i can't seem to find it now.

machine code speed or not, python is certainly fast enough for many routine operations - see the bruce dawson article. i have to wonder how python would do under intense pixel computations, but PIL seems to run nicely...and setting up psyco bindings can seriously increase performance...

on a conceptual note:

i would like to see a flowchart style data system implemented where all datablocks can be "wired" through each other in any way - so i could connect object scale to shader RBG to UV coords to object position to armature bones to particle effect, etc.

of course this is sortof possible with the existing python API, but it's not a primary ability. perhaps keyed data structures could be used to allow easy python sunc with data attributes like:

object['scale'] = shader['RGB'] = UV['coords'] = object['position'] = armature['bones'] = particle['effect']

or offsets could be used:

object[0] = shader[1] = UV[2] = object[4]

multiple input and output pipes would be nice too:

import file -> readline -> set scene -> render
import file -> readline -> set scene -> export RIB
import file -> readline -> set scene -> export data
import file -> readline -> set scene -> email
import file -> readline -> set scene -> run game engine

a visual way to "wire" data in this way would be extremely kickass - much like the gamelogic bricks. i guess this is a bit like a procedural shading language, but for everything.

...still thinkin

Knowmad
Posts: 4
Joined: Wed Nov 20, 2002 7:02 pm
Location: Deutschland

Re: Blender-Python Board, gurus & coders needed

Postby Knowmad » Wed Nov 20, 2002 7:30 pm

One of the concepts that were lying around, is using Python to replace the 'plugin' system in Blender (textures, sequence effects). Try to imagine a 'shading language' in Blender! :idea:

-Ton-


Is this 'Shading language' idea in use, perhaps in anouther 3D.Env.
or, should we be reading that as a this a referance to an on-going, or proposed project?

Bluejay
Posts: 4
Joined: Thu Oct 31, 2002 7:57 pm

Postby Bluejay » Thu Nov 21, 2002 8:05 pm

If you’re looking for a reference implementation of a shader language you could take a look at nvidia's Cg language. I believe the spec is open (that would need to be confirmed).

http://developer.nvidia.com/view.asp?PAGE=cg_main

I am not suggesting you implement Cg in Blender (I am not saying that would be a bad idea either). But it might serve as guidance for a Blender implementation.

There are of course shader languages in most of the 3D packages, but using them for brainstorming purposes would probably result in a nice lawsuit :?


Return to “Python”

Who is online

Users browsing this forum: No registered users and 3 guests