Introduction
This document will hopefully set the new programmer on a road map for how to do development work for the blender application. It will introduce what I think you need to know, as well as tie in all the other documentation out there to hopefully provide you with the ultimate guide to developing blender. I've also added a small section to the end of this document that talks about using gprof to do profiling.
In the blender source there are basically two applications the blender player and blender. There are a couple of differences that you need to be aware of. They both have separate python API's so you'll need to be aware of which one you want to work with. The game engine is mostly coded in c++ while blender is mostly coded in c. We will get into the specifics a little later on in the document.
Python
Python scripting is probably the easiest place to start programing with blender. All you need to get started is blender, python and associating blender with python. If your not sure how to do this the following link will help you out:
www.elysiun.com/forum/viewtopic.php
A lot can be done with python just take the Make Human project for example. So whats the downside to programming in python? Your stuck with the python API, which does not expose all of blender's functionality. If it doesn't address what your after, then your stuck waiting for someone to program that part and or you'll need to get more involved in the development process, at which point it is fairly simple to add a wrapper around the internal blender functions to expose them to the python API.
Python like other scripting languages tends to be easier to do rapid development you don't have to worry as much about memory issues and other things which you do in c/c++. A good example of this is there are some people currently working on refactoring booleans in blender using python and later once they have everything working they will port it back over to c/c++ for speed. This is probably the only drawback to python scripting: Computationally intensive things maybe be slower in python. However there are lots of optimizations you can do in python, the difference between optimized python code and non optimized python code can be huge compared to other programming languages. It is also fairly easy to code the stuff you need to in c and do the rest in python.
List of python resources
www.blender3d.org/cms/Python_Scripts.3.0.html
Contains various API's for blenders python scripting, for both blender and the Game logic.
Mailing list: projects.blender.org/mailman/listinfo/bf-python
C/C++
Getting further into development requires a bit of homework. The first thing you need to do is learn the basics of cvs. There is a great cvs resource at:
www.blender.org/modules/documentation/intranet/conventions/index.html
The basics you will need to know about cvs:
How to initially checkout the source.
Which is on the cvs page. You may want to checkout multiple tree's for different projects.
cvs update
Keeps you up to date with current tree and also shows you a summary of what you have changed in your tree compared to the current cvs stuff.
cvs diff (file(s))
shows you the actual differences between you tree and cvs for the file(s) you have listed. You should always look this over before making a final patch and or committing code to cvs.
A cvs diff is also what you should submit as a patch to the cvs tree if you do not have commit rights.
cvs commit (file(s)
Commits specified files to cvs. You should make sure if you have any local changes to your tree that you don't commit them by accident as part of a commit/patch.
.cvsignore files
.cvsignore allows you to flag certain files/directories so they do not show up when you do a cvs update can make your cvs update a lot easier to read need to be careful with this though.
.cvsrc file
.cvsrc allows you to setup some basic flags for cvs functions, I use the following in my .cvsrc
cvs -q -z3
diff -u
rdiff -u
update -Pd
Setting up cvs so you don't need a password:
www.blender3d.org/cms/CVS_SSH.383.0.html
Deciding on a Tree to use
There are two major tree's for blender.
bf-blender which is the tree releases come from. Because of this the commits to it are considered production quality. To get developer access to this tree you have to provide a number of patches that are accepted to the tree and basically get a feel for how things work with the tree.
The project page for bf-blender is: projects.blender.org/projects/bf-blender/
tuhopuu is the experimental tree. The access to this tree is similar but less strict. The tree may sometimes be in a less stable state but you will also get a chance to see some of the more experimental projects going on. The idea is basically to experiment with things get them polished up and eventually migrate them to the bf-blender for a release.
The project page for tuhopuu is: projects.blender.org/projects/tuhopuu/
Installing Blender
Once you have decided on a tree to work with and have gone through the initial checkout the next step is getting blender to compile. You need to get this working before you do anything else. It will give you a basic understanding of how the code is laid out and you really need a working version of blender to start from. What method of installation you use will depend on a couple of things, your experience and what platform your on.
There are two Main ways of building blender Scons and Makefiles if you are on the windows platform you can also use Microsoft's Visual C if you have it. To be a top notch developer it would be great if you could build blender on as many platforms as you can with as many build systems as you can. Everyone needs to start somewhere though. If your familiar with Makefiles and like them, I would use them. If you have and like Microsoft's compiler I would use it. For everyone else I would use scons. Scons is very straight forward and will give you less of a headache if you're not familiar with the other tools.
blender/INSTALL has detailed information on all 3 build systems as well as other information you will need to know. Another good source of information for building on specific platforms is in the developer forums:
The following page has a list of external libraries blender needs:
www.blender3d.org/cms/Getting_Dependencies.135.0.html
The biggest issue with building blender tends to be that your missing one of the required libraries and or you don't have the configuration setup to point to them. For example if your first error message is: "cannot find jpeg.h" You need to install libjpeg, and or modify the build to find it. If your using scons you would edit blender/config.opts and look for: JPEG_INCLUDE and JPEG_LIBPATH
If you're using the Makefiles you want to set the environment variable NAN_JPEG to point to the installation point of where ever you have libjpeg installed.
It is also important to know how to clean your tree for a new build. In theory the build systems are setup so that it will rebuild anything it needs to when you do a build, in practice though things sometimes get messed up. It's always a good idea to go for a clean build when your having problems.
On Scons to do this:
scons clean
you should also mv config.opts config.old
This way you can get updates to config.opts and still be able to look at your old file to make any changes you may need.
With the Nan Makefiles do this:
rm -rf obj/*
With Visual c++ do this:
Just right click on the project and choose 'clean'
Getting Help
Now is a great time to introduce various resources that are at your disposal.
Developers Forums,documentation
www.blender3d.org/cms/Development.81.0.html
www.blender.org/modules.php version of docs)
IRC
Server: irc.freenode.net
Channel: #blendercoders
Mailing list(s)
projects.blender.org/mailman/listinfo/ has all the lists
projects.blender.org/mailman/listinfo/bf-python
projects.blender.org/mailman/listinfo/bf-blender-dev
projects.blender.org/mailman/listinfo/tuhopuu-devel
Getting a working build of the current cvs will allow you to keep up to date on the latest features that have been added to blender, and keeping an up to date cvs tree will make it easier for you to develop. Some days over 30 files will be modified. It's a lot easier to modify your code as things change than say grabbing the 2.34 source code and a month later submitting a patch that doesn't match up with any of the code. Submitting a patch against the latest cvs will also increase the odds of your patch
being accepted.
Before getting into specifics about blender programming I'd like to go over a small set of hopefully helpful hints I have on programming in general.
Kent's Top 10 Programming hints
1. Keep it simple. If you add say 200 lines of code its going to be a lot more complicated to debug than say 10 lines of code. After you have been coding for awhile you'll probably add 200 lines or so with no problems but when just starting out this is important.
2. When debugging problems you need to look at the first problem. Computers
are stupid. Usually if they get messed up, things will only get worse not better. Concentrate on the first errors you see. The rest might just be because the computer is confused.
3. Different compilers on different machines give different error messages. If you have access to more than one machine and or more than one compiler on a single machine, it is sometimes helpful to look at the different error messages. The differences in the error messages can sometimes give you the key to solving your problem.
4. It's good to try and remove all warnings from your code. Usually it leads to cleaner, more readable code.
5. Pick an editor that does syntax highlighting.
6. When compiling its a good idea to use script or a similar tool to record the output so you can go over it later.
7. When things aren't working try doing a cvs update and a clean build from scratch.
8. Check for simple syntax errors using = instead of == by accident etc.
9. When stumped show someone the code, its generally something stupid you're not seeing because in your mind the code is correct. Similar to the problem of reading text and not noticing a "teh" instead of "the".
10. Pay attention to the blender style guide. (listed below)
Great Blender Developer Documentation
There is a lot out there, and its all pretty much related to this subject. I'm going to use this section to highlight important ones and hopefully bring up some that may be somewhat hidden from the new developer. This document tries to fill in the gaps and maybe mention some important parts listed elsewhere as a reminder. So I won't go into too many details on topics covered elsewhere.
At the very least there is the following documentation which you should read:
Overview of blenders data structures and file structure:
www.blender3d.org/cms/Blender_Architecture.336.0.html
Style Guide for developers:
www.blender3d.org/cms/Coding_Style.141.0.html
There is all kinds of great additional documentation at:
www.blender3d.org/cms/Development.81.0.html
Some highlights:
A similar document that is quite a bit smaller than this one but covers some of the same topics:http://www.blender3d.org/cms/Coding_Blender.144.0.html
Real world coding examples:
www.blender3d.org/cms/Coding_Tutorials.308.0.html
Intro to developing the python API for blender (focus is on the non game mode API)
blender/doc/python-dev-guide.txt
Some docs that are in the code itself.
blender/doc/interface_API.txt
Documentation on blender's GUI and how to use it.
blender/source/imbuf/readme.txt
How to add another image format to blender.
blender/doc/python-dev.guide.txt
Details on the blender specific python API as well as an example of how to add something new.
The .blend file format
One of the biggest questions that gets asked is what is the .blend file format. The answer to this question is in the Architecture document, and the answer is basically that the .blend format evolves and is basically a dump of the data structures in blender with some information on the "version" of the .blend that tells blender how to deal with things. While there is not a heck of a lot of detail in the documentation on this your best bet is to look at: blender/source/blender/makesdna/intern/makesdna.c It is heavily commented and should have all you need to deal with changes to it. Ton has written some additional notes on it here:
www.blender3d.org/cms/Ton_s_Notes_on_SDNA.436.0.html
Getting around in the code
It's important to be able to find things in the code. If it is a function, a variable or even just an error message. There are a number of tools that can help you do this. Here are a few I would recommend.
cscope
One thing with cscope is it only works on c files can't use it on c++ files.
Source Navigator
ctags
www.computerhope.com/unix/uctags.htm
grep -r "string" (files and or directories)
Hooks into blender
Since blender is a user interface to 3d programming I'm going to list some basic places to tie modifications you want into the blender interface. These are some of the most visible places so they should be fairly straight forward to work with. The best idea is too try to find something similar to what you want to do and start by copying it and working your way up from there. Start with a small project so you won't get frustrated. A good example would be just adding an extra button to the GUI and link it to some function that already exists.
Python
blender/source/python/api2_2x
Need to mention blender/doc/python-dev-guide.txt
Python Game Mode
blender/source/gameengine/
Command line
blender/source/creator.c
Command line Game Mode blender/source/gameengine/GamePlayer/ghost/GPG_ghost.cpp ???
GUI
blender/source/blender/src/header_*.c
blender/source/blender/src/buttons_*.c
and some other files in same directory
Stubs
Stubs can be an effective way to work on programming. Stubs are Pieces of programs, usually functions or procedures, that provide the correct interface but not the correct implementation of some other piece of program. Basically what you can do is write a small little set of dummy functions/structures so you could work on your code without having to worry about the rest of the blender code base, once you have it working then you can drop it in to the blender code. You can also use them to test out a function and make sure its working correctly.
Profiling
This section is for advanced users, that would like to do some optimization. Since the majority of users use gcc this section will use gprof with is gnu's profiler. I'm guessing visual c++ has something similar but the details will be different. The idea behind a profiler is you setup the program to collect statistics, run the program focusing on the things you want to work on. Then look at the statistics to see how much time the
application spent in each function. This will give you an idea of what the "problem" functions are in a program and give you an area to focus on.
To compile blender with profiling you need to add -pg to the compile options. Probably the easiest way to do this would be to add it to the CFLAGS, CPPFLAGS, and LDFLAGS. If your using scons you can do this in config.opts, if your using the Nan Makefiles you can do this in:
blender/source/nan_compile.mk
You'll want to make sure you do a clean build after adding the -pg (and or removing it when your done) Once it has compiled it is time to run your application. You should think of a good test set of things to do with blender to focus in on what your trying to work on. If you're working with the sequencer, using radiosity in your test suite would not be a good idea.
Once you have run through your tests its time to look at the data. Once you exit blender it will save statistics to gmon.out. It might be a good idea to move gmon.out to another file so you don't accidently overwrite it by starting blender again. To view this data you run gprof (application) (gmon.out) Where application is the path to blender and gmon.out is the path to your gmon.out file. I would redirect this to a file by appending "> wah.txt" to the end of that command example: gprof blender ggg.out > wah.txt There are lots of options for gprof but this is the basic idea behind it.
One thing to note, you may need to look at the log from building blender to find missing -pg's and modify your build environment to get them added if you do not get a gmon.out when you run it. I had this issue because it wasn't adding -pg to the final linking stage with the Makefiles after adding -pg to LDFAGS that fixed it.