The Blender node system basically consists of three main components, which will be described in detail below. I've chosen to follow the convention of prefixing struct names in Blender with a lowercase letter 'b', this to prevent potential conflicts with using system libraries (a "struct Node" might very well exist already).
The list of nodes and the list of links between nodes is stored within this struct. Node trees can be of a specific type, for example shading or compositing. Currently the kernel code handles hardcoded, type-specific exceptions like for adding nodes or removing/freeing nodes. You can find comments in the code that this is a candidate to become "handlerized" by using function pointers. (BTW: I strongly advise "handlerizing" this before adding more NodeTree types.)
A Node Tree can exist in any location within the Blender database, though for simplicity, it is advised to limit it to adding to existing Library structs, such as Material (shading) or Scene (compositing). Node Trees used this way are a static part of this Library Data (in the Blender Architecture doc called "direct data"), and has to be read/written or duplicated/freed like regular direct data as well (meaning no other users of such Node Trees will exist).
Groups
Using Node Trees as Library data is also possible. In such a case, the Node Tree becomes available as a "Group", and is stored in the Blender Main Library, allowing reuse and linking among files. You cannot, however, link a Node Tree Group directly to, for example, a Material. Such Groups can only be added by inserting a "Group Node" within the static Material Node Tree. The current implementation doesn't allow Groups within Groups, I prefer to first stabilize the Node system fully before considering this.
This design decision - having the same Struct for static Node trees and for dynamic Node trees - was made to simplify the API for node manipulation.
It is good practice to ensure that re-linkable Node Trees (i.e. Groups) restrict themselves to nodes where it makes sense that they be re-used. The nodes you will typically add in the static Node Trees will define how the input and output is handled. This is mostly evident in the Compositor now, where you need the "Render Result" nodes based on what the Scene itself has defined, these Nodes cannot be inserted in a Group
Type definition
Another important Node Tree feature is the built-in "type definition". Each node type, as allowed within a tree, has a generic type definition array, which is either stored in the Blender binary (built-in nodes) or can be generated on the fly (for example, Group nodes or Python defined nodes).
This type definition ensures that the functionality for a specific Node can be improved or altered without creating conflicts with saved data in Blender files. It will allow the addition or removal of input sockets, or even the complete removal of node types altogether.
Type-definition checks are done on each Blender file read (or Library append/linking). This is a nasty piece of code, especially for restoring changed nodes within Groups, and the most important reason why Groups cannot contain Groups yet.
To allow multi-user and multi-threaded execution of Node Trees, each Node Tree has its own local "stack". In the stack, all potentially changing data is copied from all nodes in the tree. When a Node Tree includes Groups, the nodes inside the Group will also use this stack, ensuring multiple instances of a Group can be safely used within a tree.
An attempt has been made to make Nodes as generic as possible, so most of the kernel and UI editing functions can be applied without checking for node types.
The most important Node feature is, of course, its 'sockets'. Any Node can have an unlimited amount of inputs and outputs, as stored in a list.
The Node's type-definition (pointer to a static bNodeType array) defines the number of input and output Sockets and the socket types. On each file read, this type definition is used to verify the stored sockets.
Each Socket stores its own local "stack data", a generic description of the data as used while executing a Node. Typically the inputs of a Node then contain user-defined or previously calculated data, which then gets evaluated and written to the output sockets.
When an Input Socket has no link, a Node will only evaluate its own user-defined input data. In the case an Input Socket has a link, the Node Tree stack system ensures that the data from the linked Output entry is used, so copying of data is not required.
In short, Input Sockets are read-only, Output Sockets are write-only, and based on how it all is linked, a stack is used ensuring that the reading and writing results in the proper end result.
Currently in this stack data, Shader Nodes only use a 4 component float vector, and Composite Nodes either use the same, or store a generic "CompBuf" in the available data pointer. After execution, the Compositor also stores the resulting buffer(s) of a Node in its output(s) again, this to enable faster recalculating when only a single Node changes on editing.
A Node currently has three ways to store type-dependent data;
- using the built-in 'custom1' and 'custom2' shorts (for lazy quick coding)
- using a Library ID pointer (to Material, Texture, Image, bNodeTree)
- using the void *storage pointer (is part of Blender DNA, the struct name is part of the Node type definition)
For drawing Nodes at a certain location, the only user-controlled data in a Node is its (locx, locy) coordinate. Based on the various drawing settings in the node flag, the bounding box 'rects' for the Node are calculated on each redraw again. (Note: since these rects are stored in a Node, only a single Group Node can be drawn editable at the moment. It should become part of the drawing API).
Drawing Nodes happens in a fully generic manner in src/drawnode.c. The optional Preview is added if set (and rendered). In case a Node requires a GUI (buttons), a callback will have to be set in the Node's type definition. This happens in the initialize function of src/drawnode.c too.
All links between Nodes are stored in the Node Tree. Whether or not links are allowed is based on a couple of rules:
- The socket 'limit' value defines the maximum number of allowed connections. This is part of the type definition too, with value "0" denoting no limit. The current implementation (Shaders, Composite) only uses "limit = 1" for all inputs, and "limit = 0" for all outputs. That's the default for how execution with the stack works as well.
I'm still uncertain if it should be possible for Input Sockets to have multiple links, like for Logic editing. In case such requirements exist, it might be better to allow a Node to have a variable number of inputs (like for an OR node).
- The socket 'type' can enforce links to be established only between equal sockets, this is to prevent confusing situations (like a Normal output connected to a Value input). This however has proven to become a usability bottleneck. The first implementation automatically inserted converter Nodes in that case, which easily made Node Trees grow unnecessary complex and slow. The Compositor in Blender already has integrated, automatic conversion (based on logical defaults), something that should be added for Shaders too.
However, for future expansions (like Python Scripting Nodes) it might still be required, if it allows completely incompatible socket types.
On each update of Node Tree links, the "ntreeSolveOrder()" function is called. This is the basic dependency graph for Nodes, which sorts the list of Nodes in a way that it can be correctly executed in order.
Creating cyclic situations is still possible, and will draw links in red to denote this conflict. In that case the evaluated outcome is undefined.
Since links define which sockets are n use, they also define which sockets can exposed for use externally in Group Nodes. By default, only unused sockets are exposed in Groups. Changing links internally in Groups also changes the Group 'type definition' and therefore can destroy existing usage of linked Groups.
When the Node Tree type already exists, adding a new Node type is quite straightforward.
source/blender/blenkernel/BKE_node.h
Here you define a unique identifier for the Node. This is a hardcoded integer define which cannot change ever (unless an exception is coded on file reading). For future/past compatibility, always use new numbers for new nodes, so if you remove a node the old define should not be re-used.
Currently, the range 1-100 is used for common Nodes, 100-200 for Shader Nodes, and 201-300 for Composite Nodes. These ranges are likely too limited though...
In the top of this include file you can find the type definition structs for Sockets and Nodes. This definition must be hardcoded in a C file.
source/blender/blenkernel/intern/node_shaders.c
source/blender/blenkernel/intern/node_composite.c
These are the two C files currently storing the Node definitions and their execution. For each node three chunks of code can be found:
1) Socket definitions
- The input and output Sockets require a unique name. This name is used to verify correct type definition, so cannot be user-defined or changed easily.
- For now, always use "limit = 1" for input sockets, and "limit = 0" for output sockets. This may disappear at a later stage.
2) Execution callback
The execution callbacks are independent of Node Tree types, so you may need to cast the stack data input to the required types.
The callbacks must remain fully threadable, so do not initialize data nor use static data in the functions.
Two types of execution now exist; all Shading nodes are evaluated on every Node Tree evaluation call, whilst Composite nodes are evaluated once sequentially (or in parallel when threaded).
3) Node definition
- The Node name may be changed by a user, the type definition name only provides a default.
- The "node class" is used for defining drawing type (theme-able) and for detecting on what level the new Node will be inserted in the "Add" menu.
To make the Node 'existent', you have to add a pointer to its definition in a global bNodeType array (in bottom of C file). This will become later a dynamic list, to allow Python defined Nodes.
More information can be found as a tutorial in the wiki; mediawiki.blender.org/index.php/BlenderDev/AddingANode