Re: Ocean Simulator: The research continues...

[simonharvey]

Hey Simon I was just wondering if you where still working on the C/C++ plugin system??

Hi,
I have currently had a look at tutorials that others have written for Maya and 3D Studio Max to get an idea of their plugin structures and also what the acceptable "plugin-code-untidyness" is. I have decided that I am going to use C because it can be compiled on any platform which is a break from the others (they use C++, which I find incomprensible).

I have also decided that the first version isnt going to be multithreaded, but single threaded because it is simplier to write and requires less code.

Currently I am also encapsulating the FFTW3 FFT library as another BMX (this is to test out the plugin loader and also the ability to add extra 3rd party libraries without any hassle with the main developers).

Kind Regards
Simon Harvey

[antihc3]

Outstanding ,

Well if you would like any help or anything i would love to help.

My email is (my_username_here) at gmail dot com

It also looks like Owen Jancowski-Walsh has some of the same ideas.

[ton]

I'm a little concerned on the direction of the discussion here... designing a good plugin API for Blender won't be interesting if it doesn't reflect how Blender currently works itself.
As original designer of Blender, I really get confused by the proposals here... I miss general objectives or a compliant & functional structure. How the API technically is structured is really a decision for later.

Also keep in mind that a program like 3DS was designed from scratch as a plugin system. Maya as well, although it has MEL as glue too. Blender was not *at all* designed that way.
Second to keep in mind that plugin APIs also serve as a means for closed source applications to allow third parties to extend functionality. We don't need that really... within a production environment, anyone can quickly add the temporal features in Blender they need themselves.
Third to keep in mind that a plugin system for Blender cannot be made "commercial" with closed source, only GPLed code is allowed to be a plugin in Blender. I doubt it will become an incentive for third parties to make commercial Blender extensions with GPL only.

If you look at how the python API developed - apart from some pending issues that were caused by too "un-blenderish" design decisions - it does a pretty good job in reflecting Blender, how Blender actually works, and allowing to extend it with new tools. The current team spends long and long design hours on keeping that quality, and improving possibilities to extend Blender.
Already a very long time ago, it was decided to choose Python as the #1 extension/plugin language for Blender. It has many benefits over a plugin API (like accessibility, compliancy) and it allows anyone to build scripts under any license (e.g. allows closed source scripts too). You can even extend Blender - via python - with a closed source library/plugin.

So... all in all, my guess would still be to stick to one system in Blender, and help out designing better Python methods to extend Blender in ways that would be acceptable on a level for each plugin developer.
For those specific "plugins" that would require raw speed (like for textures or image plugins) we can use Python potentially still, just a matter of smart code work here.
Think of a Python based shading API, like renderman!

My 2 cents...

[joeri]

Third to keep in mind that a plugin system for Blender cannot be made "commercial" with closed source, only GPLed code is allowed to be a plugin in Blender. I doubt it will become an incentive for third parties to make commercial Blender extensions with GPL only.

Is this really true?
Plug-ins are not compiled with blender, so they don't fall under the blender GPL, just as fonts that blender uses don't fall under the blender GPL.
Right?

[simonharvey]

Third to keep in mind that a plugin system for Blender cannot be made "commercial" with closed source, only GPLed code is allowed to be a plugin in Blender. I doubt it will become an incentive for third parties to make commercial Blender extensions with GPL only.

Is this really true?
Plug-ins are not compiled with blender, so they don't fall under the blender GPL, just as fonts that blender uses don't fall under the blender GPL.
Right?

I have been thinking about this and I have come up with the following precedent:
NVIDIA who make graphics cards have a cross platform driver aritecture, one of whos targets are linux (which is a GPL piece of software), now by your reasoning they should open source their drivers but they dont. The reason is that their crossplatform drivers wernt specifically written for linux and their is a super-thin layer of LGPL code between their drivers and the vendors linux kernel.

Secondly, commercial vendors who write plugins for Maya and 3D studio max arnt going to port their codes over to python, (they could, if they had a large amount of motivation) but writing a thin C/C++ LGPL layer is much more trivial and easier to do.

Thirdly all major 3D modellers have (at some level) a native C/C++ plugin interface, so by adding one your software package, it becomes like one of the pack and therefore has a higher chance of commercial outfits writing code for it because it is seen with the same "mod-ability".

I'm a little concerned on the direction of the discussion here... designing a good plugin API for Blender won't be interesting if it doesn't reflect how Blender currently works itself

What I am really doing is allowing external binaries to access the internal blender API (i.e. the MEM_* and ui* APIs), except I have chosen to respect blenders low resource/fat approach by making the modules reloadable, so resources are only being used when they are required. also these APIs are all wrapped and can be accessed consistantly through the same mechanism.

Also keep in mind that a program like 3DS was designed from scratch as a plugin system. Maya as well, although it has MEL as glue too. Blender was not *at all* designed that way.

Blender isnt really that far from being plugin based, it has many APIs that are easy to wrap, the hard bit has just been writing the plugin loader and registeries. Im still several months away from completing the prototype version, so I cant tell you about how hard it was to add the code to get a module to shade a piece of ocean, or to add ocean waves to a grid mesh but I have already done this before and it isn't difficult.

Second to keep in mind that plugin APIs also serve as a means for closed source applications to allow third parties to extend functionality. We don't need that really... within a production environment, anyone can quickly add the temporal features in Blender they need themselves.

I have found that it is not a trivial issue to add functionality to the blender codebase, furthermore because it is being activly developed it is highly plausable that their additions may not work in a later version because the code that they based their modification was completely rewritten (this has already happened to me), How I am designing it is that the code base behind the exported API can change completely, but as long as it shows the some consistant behaviour then everything is fine.

As original designer of Blender, I really get confused by the proposals here... I miss general objectives or a compliant & functional structure. How the API technically is structured is really a decision for later.

I am sorry for keeping you all in the dark, and not giving people an idea of where I am going with it, or asking others what they think would be a good idea. This is selfish of me. My aim is to make something to startoff with and then share it with everybody else for C&C later on. In this way their is no risk to any of the core blender development team, so their time and resources arnt wasted.

What I would really like is about 2-6 months of grace so as to get something ready for other developers to evaluate. It would be cool to get permission on the project website to get some space for a fork:

We're open for developers to create experimental Blender trees or even complete forks. Just make an account at the projects site, a link for proposing projects can then be found at 'My Page'.

I just submitted a project request, it should be there for you right now.

Kind Regards
Simon Harvey

[kencanvey]

Simon good luck with this project, I for one look forward to seeing the fruits of your labour, and I urge others to give support to Simon in any way possible, as I belive it will add another level of power to an already powerfull tool.


Ken

[ton]

Sure, I've approved on your project submission.

But you have a couple of issues not examined well.

1) It is 100% *sure* that any plugin talking to a GPL-ed program like Blender has to be GPL too. Check the gnu site for that.
This is different (as exception mentioned in GPL) for drivers and operating system libraries. Here the issue is inversed, these drivers or libraries don't link to GPL, but GPL has to link to them.

2) Commercial vendors for Maya or 3DS plugin can use the Blender Python API legally as thin wrapper for their plugins. Not a GPL-ed plugin API you would make...

I know it's not "nice"... but alas, it's the 'copyleft' principle..

[ton]

Simon,

The mail address you submitted bounces...
you can contact me personal with a valid address of course.

This is what I mailed you together with the project approval:

We are very open to experiments with Blender, so you work is welcome anyway! The legal (GPL) issues we can keep as a minor problem... the functional/technical side of the work is interesting research to see happen.

We currently don't have active system administring/moderating for the projects site... I hope the things work (like cvs, mailing lists, tracker, etc). If you have problems, just poke me!

All the best,

-Ton-

[simonharvey]

Here is a release schedule:

[1] 30th June => Basic functionality:
1. Basic design
2. Binary loading
3. Splash screen with a moving status bar at the bottom
4. Core module API - enough to load a skeleton BMX

[2] 1st July - 30th July => Design Analysis
1. Other developers look at the design & offer suggestions
2. Ideas are refined and taken into account for the next release

[3] 1st August - 30th August => Prototype Recoded
1. Prototype is updated to reflect the updates from the analysis
2. Interfaces are added to allow other blender developers to export apis
from other parts of Blender.
3. The OceanFFT deep swell simulator is updated to the new design

[4] 1st July - 30th July => ModBlender release
1. Modblender is released to the internet community (as alphaware)
2. [optional] OceanFFT BMX is released as an example extension

Beyond this I just can't say but it would involve some kind of a
graphical viewer to browse the loaded modules, their resource
allocations and whether their contents & binaries have been swapped
to disk. I am not going to attempt this in Blender 2.24 since I would
rather just add an addition to the Profiler that came out in 2.35 - 2.36.

Currently I am slowly finishing up [1], and need to add a bit more code
(the total amount will be between 1500-1600 lines) as well as getting
rid of the 450 - 500 compile time errors & warnings that have popped up.


Kind Regards
Simon Harvey


Anything mentioned here is tentative only and is not to be taken as an
absolute, dependable guide and is subject to other agendas and events
ocurring in my life

[jesterKing]

Tentative or not, I'm sure this roadmap is appreciated. And do drop by in IRC if you get stuck or so.

/Nathan

[topy]

I hope that this ocean simulator will born soon...I have to use it!
^_^
topy the sage

[levon]

and like i said before, if you need help beta testing it, i want to help.

it is good to see more 'trees' of blender, instincitve and tuhopuu are the only two, and instinctive is a bit hard to get hold of on windows.

[DYeater]

and like i said before, if you need help beta testing it, i want to help.

it is good to see more 'trees' of blender, instincitve and tuhopuu are the only two, and instinctive is a bit hard to get hold of on windows.

Tell me about it. I had a project in which I would have loved to use IB's boat wake effect, but the import code wasn't working in the build I had. The bug had been fixed, but no Window's build had been made.

Pity.

[LetterRip]

Instead of more trees, I much prefer to get the code into T3. Going into T3 gives a much better long term chance of being accepted into the Blender core and it will get far more people using it than another tree.

LetterRip

[levon]

hey simon hows this all going? i know its not the 30th yet, but its such a cool project