I'm a newbie in realtime 3D, but I've maneged to use Blender with some impressive results (for a newbie). I work with Video Production and I've seen many realtime engines for integrating realtime live video with realtime rendered 3D graphics, but most of them were not as powerful or easy as blenders engine. The features in it have so many possibilities in the area i'm working on, so I would just like to know how difficult would it be to implement a layer where we could colourblit live video into the rendering engine. I'm certain you understand the potencial for this in video production area, because it is so simple to track a camera movement and have blender calculate the virtual scenario on the background...
Can anyone just give me a hint of what it would take
Not a bad idea, it would definately take some source code to pull off though. One of the limitations currently of the blender game engine is you can't load textures on the fly, in other words, you are limited to what textures you have on hand. Piping in a texture constantly updating with live video is just not going to happen without some work in the source.
Also, it would be hard to make this feature cross-platform compatible because of the differences in video interfaces between platforms, it would be best to allow the textures to be retargetted in realtime, then on top of that have a plugin layer where you can target the texture to a file or videostream.
As all the developers (all, what, 3 of us) are all working on different things, and not much being done as it is, this will likely be a feature you'll have to either add yourself or wait a long time for.
Your idea seems even more interesting than mine... You were thinking in having live video for textures, I was just thinking of having a 2D layer with colorblit live video with no 3D calculation whatsoever...
Unfortunatly, my programing skills are close to zero, so I don't think I can try to implement this myself, although I've read somewere that someone has looked at blender's rendering engine and wrote some detailded explanation of how it works, and also had some ideas for new features, just for the rendering engine... I'll try to dig some more into it. Thanks anyway.
I will still be using blender to interact with realtime video, although I have to chromakey the video over the CG background, and therefore, I can never get 3D objects in front of the video layer.
Blender is indeed very powerfull, so keep up the excelent job... Hope to see future releseases of the game engine!
Hmm, the only way I could see to do that would be to have a texture. I think if you do a 2d layer, it goes over the top of everything, which isnt what you want. I'll look into it eventually if I ever get that far