crafter-blender integration

General discussion about the development of the open source Blender

Moderators: jesterKing, stiv

matt_e
Posts: 410
Joined: Mon Oct 14, 2002 4:32 am
Location: Sydney, Australia
Contact:

Post by matt_e »

bfvietnam, the yafray shader language is already made - take a look over at www.yafray.org, in their wiki, and try downloading crafter and making some test XML files. I get the feeling you're wanting to re-invent the wheel...

bfvietnam
Posts: 0
Joined: Wed Apr 21, 2004 8:54 pm

Post by bfvietnam »

wumamor wrote: bfvietnam: to you i put this.....
HELP WAVEK INTEGRATE CRAFTER!! if you can and would be so kind because it seems to me that you have a very good understanding of how visual shaders and the languages associated with them should work.

well anyway ty for the quick responses :D
Well I know how languages in the computer work, I've programmed many years, and I had some courses in digtial electronics and made A's while many programmers I knew made C or D's, but that's about where my experience is sort of limited.. Programmers work on a level of abstraction, or should.. Rather than thinking about how to do things, I think about hwo to structure the way of thinking about how to do things, and I do this myself while I hack.. there was a time when I thought you do lots of design before coding, but you need to hack stuff before you can design it right. So at best I can give wavk good ideas, but I don't know how well I could do to help him code the C++.. I code on paper, that what is driving me to suggest this graph-language. Coding on paper doesn't get anywhere without the implementation, in a text-based language. I might do well to work as a project manager but I hate project management. Thanks for the encouragement though..

I've been trying to get a development environment for C++ together, but it seems so much more compelx than it used to be.. Damn my 2-bit unix upbringing.. When all you needed was a makefile and a handful of libraries.. Now there is so much more to learn.. Whatever happened to that simple C64 style of coding, where you just draw dots and lines.. Anyhow, I like to focus on simplicity and the design of programs for users not for machines. So I wouldn't be much help to wavk other than offering suggestions..

Well time to go to bed..
Last edited by bfvietnam on Mon Aug 23, 2004 10:27 am, edited 1 time in total.

bobthevirus
Posts: 0
Joined: Wed Jun 25, 2003 7:11 am

Post by bobthevirus »

wavk: I've been wanting for a while to get into the coding side of things, and as you will be going severly OOP, I'd love to help. Email me bobthevirus@orcon dot net dot nz if there is something relitively easy that I can start off with... In the meantime I will look through the matarials code and see if I can figure out how this all will fit in.

Also, In my (comletely uninformed) opinion, extending the yafray shading language, but implementing it in the node-based structure as you guys have been talking about, (ie: making it so that the nodes convert directly into xml) might be an easier way of doing things. (Unless of course you already have object designs and everything sorted out)

bfvietnam
Posts: 0
Joined: Wed Apr 21, 2004 8:54 pm

Post by bfvietnam »

broken wrote:bfvietnam, the yafray shader language is already made - take a look over at www.yafray.org, in their wiki, and try downloading crafter and making some test XML files. I get the feeling you're wanting to re-invent the wheel...
Yeah that's text based, but I'm talking graph-based languages..

How many graph-based languages are there? At least we could play around with the concept and see what comes of it?

postlogic
Posts: 0
Joined: Thu Jan 30, 2003 1:05 pm

Post by postlogic »

Hmm.. I was just wondering, and you're more than welcome to flame me for not reading up on this, but does Blender and/or Crafter and/or Yafray utilize GLSL? It's a great language, easy to use, and a pretty decent standard.

bfvietnam
Posts: 0
Joined: Wed Apr 21, 2004 8:54 pm

Post by bfvietnam »

postlogic wrote:Hmm.. I was just wondering, and you're more than welcome to flame me for not reading up on this, but does Blender and/or Crafter and/or Yafray utilize GLSL? It's a great language, easy to use, and a pretty decent standard.
Sorry.. This is not a flame..
I'm sure the language you have here is quite ncie and all, but its going to evolve and other people will make things like it with different syntax, do you really want to deal with all that learnign an relearning of ways to express how to make a shader?

Look here..

http://oss.sgi.com/projects/ogl-sample/ ... df#Page=68

This is where GLSL's grammar starts.. I haven't looked at it in detail but it looks like C. So why is it a seperate language and not C? I figure it to make things simpler for the coder and maybe simpler for the users if the users know C.. But you can generate code for this language from tree-based graphical language, like the one I'm trying to describe.. Because the grammar is tree based..

For instance.. I just randomnly took something from the GLSL
language and put it here:

Code: Select all

function_call:
     function_call_generic

function_call_generic:
     function_call_header_with_parameters RIGHT_PAREN
     function_call_header_no_parameters RIGHT_PAREN
How this reads is "function_call" is really a "function_call_generic".
And "function_call_generic" is really either a
"function_call_header_with_parameters" and a "RIGHT_PAREN"
or it is a "function_call_header_no_parameters" and a "RIGHT_PAREN".

Somewhere else in that pdf file, function_call_header_with_parameters and function_call_header_no_parameters are defined, as well as RIGHT_PAREN.. Hmmm..

The thing that is funny with this grammar is that it implies some knowledge about a garammar for C, like what is a "RIGHT_PAREN". Somewhere there needs to be

Code: Select all

RIGHT_PAREN: 
                  ')' 
But I couldn't find it in that grammer..

Because the way this grammar is defined in this document, RIGHT_PAREN
can be '}' , or "<end>", or pratically anything you want it to be, and it would still be GLSL compliant. I think the implication is that you assume its C. And all the examples in the document are of C like syntax. But one could take it to that extreme, and use a translator to convert it from symbols one language uses and what GLSL uses.. I would not suggest this, I'm just saying what defines a language for computers is often redundant and resembles a tree when relating the parts of the language's grammar.

One purpose I think, for this recreation of languages, is just to keep people from entering C programs into GLSL compilers. But then what do you do if you want to combine C with GLSL, even if they are fundamentally the same? You then have to do something funky like interface C to GLSL, which results in writing C interfaces for GLSL and GLSL interfaces for C, and then linkers that integrate the two. How about if you want to use more languages? Well then you might get something like CORBA which was
made to make legacy technologies (programs written in arcane languages like COBOL, Pascal, FORTRAN, etc.) look like graphs of node that can connect to each other, called a distributed integration.

This is the kind of things computer programmers spend all their lives doing.. Making programs that do useful things.. And then trying to figure out hwo to connect them to other programs. And a commercial vendor, like NVidia for instance, will create a new language to make programmers lives easier, then it will make their lives harder as they try to cope with the muck of other shader languages.. And it never ends, because its not in these companies best interest to stick to one language. If that were the case, they would have to deal with competition. GL developers would have to deal with NVidia..

Its this inability to deal with others that perpetuates the seperation of languages. I mean, why do we have German, and French, english, spanish if fundamentally you can say the same things in each? And what do we do when we try to communicate with someone who doesn't speak our language? We use our fingers, and we point to things, and we make actions..

This creation of yet another programming language forces people to understand syntax rules, when really underneath the computer generates a chart. And the pattern of that chart is like the grammar.

Here are some other grammars..
C: http://www.lysator.liu.se/c/ANSI-C-grammar-y.html
PASCAL: http://www.moorecad.com/standardpascal/pascal.y
SCHEME/LISP: http://www.htdp.org/2001-01-18/Book/node43.htm

(lisp and scheme are close, I couldn't find a lisp grammar probably because its too easy its pointless to define a grammar)

PERL: http://home.uninet.ee/~ragnar/yacc/perl/perly.y


I was hoping to finds some grammers in the CTAGS library, what text editors like VIM use to hilight code (and what I use a lot), but it seems the author doesn't know how to use lex/yacc.. That's no biggy, you can describe a grammar programatically, just it will be a heck of a time trying to get anyone else to figure out your syntax. See you can take a grammar like the ones above and compile them with some or no modification with lex (a program that turns a lexer into a C program that parses that grammar). Then it up to you to put that into some form internally, and that form is usually pointers and structures that represent a tree. (given that the language is not too arcane and can be reduced to a grammar easily). I think perl and Fortran, unlike I previously thought, are not context-free grammars.

Lex is often used with Yacc
to make compilers and interpretive languages..

So given that you know how to approach making your own language..

I'm saying, why even bother.. Why not go direct to something that is graphical, like the graphs, which is what data structures tend to look like (and are, if the coder is a good one) that are produced intermediate to lex and yacc.. Note I'm giving you a high-level course in computer science in an american college, I probably shouldn't say anything, its almost taboo!!

To write a language, first make a grammar, then define functions from that grammar that Recursively define
how to Parse text following that grammar. Then write functions that create a data structure to keep track of the relationships of the code parsed. That data structure can be reduced, first, to machine code (that can be executed on a computer) or , second, it can be executed directly. The first is called a compiler, and second kind is called an interpreter.

Perl and Basic are interpreters. C and Java are compilers, or executables that turn text into executables.. C produces Machine Code (by way of a machine specific assembly language) and Java produces byte-codes specific to a virtual machine called JVM (Sun was going to make a CPU that ran Java bytecode directly, but I don't know what happened to it, ah, its being put in cellphones now).

After giving you this short introduction on making more useless languages..

My point is there is a way to end this insanity.. We can try making these languages graphical in form, using boxes and lines.. Graphs.. Like with a flow-chart. No I don't think its too simple an approach.. I mean, how much do you need to make a computer language useful?

You need something like a C++ class.. Then some way to express math. Then a way to define functions. Those are the methods of a class. Then a way to store variables in the class. Now you have an object oriented structure. You can make a entire language from just that.. If you need global variables, just make another class that stores variables with methods to access those variables. And every object that needs those variables can access that class (or an instance of that class).

It may not be possible, however to get away without defining some kind of text based language, but its better that the text is encapsulated into a graph-based environment than bypassing the graph altogether.. But then you run into the problem that this is only encouraging otehrs to make more useless languages. If you keep it graphical you will avoid the pitfalls of the gibberish computer languages can produce, as each programmer tries to simplify the representation of the language, which just produces more languages. When really the programmer is too focused (as programmers usually are) on using text when the best model is not to use text but to use graphs. If this were not the case, we wouldn't use graphs to describe how to design a program.

By the way, this wikipedia page is quite interesting:
http://en.wikipedia.org/wiki/Algorithm

Its the super-set and origin of computer languages.. But notice off to the side, the reference to "flowcharts". I'm suggesting, we make a flow-charting language that is compileable, thus bypassing a symbolic language.. And have the artists use this to make their own shaders with, and overtime make things like modelling tools, and who knows what else..

Also Buzz Tools is a bit like crafter, but for music instead of textures.. Imagine a language where both could be connected just by virtue of them being objects and not different languages. I think this can be done making something internally one-to-one with what you see on the screen. I believe that this is the only way it can be done.

Also found some sites that associate flow-charts with storyboards:
http://www.google.com/search?sourceid=m ... +flowchart

To make the look of this article intresting :) , I will put a reference to an image from the buzzmachines site showing what buzz's graph-interface looks like, note this is a configuration of connected virtual machines not so much something like a script or a program, but it might be possible to someday make the machines in this graph from graphs like this of smaller components. It gives one some idea what the interface of future versions of crafter might look like:

Image
Last edited by bfvietnam on Mon Aug 23, 2004 9:16 pm, edited 5 times in total.

TorQ
Posts: 29
Joined: Wed Jan 29, 2003 2:03 am

Post by TorQ »

Some very interesting comments here... I just wanted to let everyone know how excited I am about the prospects of a better material creation interface. As far as I am concerned Blender's material interface is its most confusion and underpowered part. Admittedly I think that even MAX's material interface is better. Can you imagine it? :) Well anyway good luck and I will be looking forward to trying out whatever you come up with.

TorQ

Koba
Posts: 0
Joined: Thu Apr 15, 2004 9:48 am

Post by Koba »

Hi

I thought Crafter was dead (nothing has been updated on the site for ages). If it really is going to be integrated into Blender, this is really, really good news.

Koba

wavk
Posts: 126
Joined: Wed Oct 16, 2002 9:58 am
Location: The Netherlands
Contact:

Post by wavk »

Please don't get too high hopes! I'll try to work on it. But I'm very limited on time, so it could take a lot of time :(

cekuhnen
Posts: 0
Joined: Mon Jan 13, 2003 11:04 pm

Post by cekuhnen »

am not a code and so no help here but i just want to let you know i would be happy to see a node based system in blender that works together with yafray and or rendman compatible systems!

claas

bfvietnam
Posts: 0
Joined: Wed Apr 21, 2004 8:54 pm

Post by bfvietnam »

well I would be interested in working on it, but I don't have too much time either.. I did over the weekend ha! I might be able to implement some
of the idea I had.. Is there a favorable environment for developing this in?

bobthevirus
Posts: 0
Joined: Wed Jun 25, 2003 7:11 am

Post by bobthevirus »

Proposal for implementing crafter like features in blender.

1. Create a new window type called matarial editor. This will be a tree, like in crafter, with an "output" node on the right hand side as in crafter. The renderer will traverse back up the tree, and into all leafs as needed. The excact shape should probably be decided at some later stage, because I'm not sure how to integrate it into the existing renderer.

2. Implement the node system as a gui feature, preferably so that it can also be used in the OOPs window too.

3. Implement the aspects of the current shaders as nodes, initially at a coarse level: Blinn with Spec, Hard and Refrac sliders, ray reflection with sliders for Raymir, fresnel, frac, depth, etc etc.

4. Then split those up so that instead of sliders there are inputs, and add/sub/mix/mul/div combiners etc. Also have ramp combiners, and others... lots of options.

5. ...

6. Profit!

Probably an XML file format could be used, and converters written to get it to yafray/renderman/etc... Also c or pytyhon plugin nodes could be used in place of any of them...

Lastly, maybe we could allow for volumetric matarials, for hair, fur, fog, beer, etc.

If at least the first item in my list is agreed upon, I'll see if I can figure out how to the 1337 h4x0rs can implement it...

bfvietnam
Posts: 0
Joined: Wed Apr 21, 2004 8:54 pm

Post by bfvietnam »

bobthevirus wrote:Proposal for implementing crafter like features in blender.

1. Create a new window type called matarial editor. This will be a tree, like in crafter, with an "output" node on the right hand side as in crafter. The renderer will traverse back up the tree, and into all leafs as needed. The excact shape should probably be decided at some later stage, because I'm not sure how to integrate it into the existing renderer.
It might be good to see pictures of these interfaces.. Anyone know a good program for diagramming? I have Openoffice..

kidb
Posts: 0
Joined: Wed Jul 23, 2003 4:31 pm
Contact:

Post by kidb »

dia

jesterKing
Site Admin
Posts: 207
Joined: Fri Oct 18, 2002 12:48 pm
Location: Finland

Re: How about a material compiler..

Post by jesterKing »

bfvietnam wrote: It would expose the reality of what computer languages are, and why we shouldn't ever use text to represent something that is fundamentally visual!!!
just FYI: http://wouter.fov120.com/aardappel/ a language with a graphical representation and editor.

/Nathan

Post Reply