i recently rediscovered the idea of face-tracking for lipsyncing. i had to try it, so i put some markers on my lips/face and made a quick shot with my old Hi8 cam. i loaded the sequence in blender as a bg-image and set up some control objects to match the positions of my markers. then i thought - hey, it would be nice to have my face-markers tracked and the motion assigned to my control objects. the image below illustrates this basic setup.
i'd basically like to know if other ppl are interested in such features and how hard it would be to implement as a py-script or core blender feature.
I dont think it would be too hard. You could adjust the contrast and brightness of an image to get a black and white image with just the markers on.
Then you could write an algorithm to group the pixels together (so the image shows single dots for the markers, rather than blobs)
Beyond that, setting a control to each point could be done manually or automatically could be done quite easily by knowing the coordinates of the marker.
Then each frame move the nearest control to each marker to the position of the marker (sorry if that doesnt make sense)
I have recently searched all possible softwares which are capable of motion tracing from video, usually they aren't implemented in the 3d package(which would be great, becaouse you wouldn't have to convert the motion data). There's 1 free tool for motion tracking. Dgeeme from geeware.com. This software is unfinished, i had problems with exporting bvh files, but it works quite fine. I tried to contact the author, but he didn't respond.
these sites are about motion tracking(algorthms etc.)>
other links at geeware, but the server is often unaccesible, if anybody wants this utility(dgeeme), i could maybe post it. I also could write something about algorithms. It would be wise not to implement this only for lipsync/face animation, but for the movement of the entire body. this goes hand in hand with the implementation for bvh files...
A motion tracking would be also very useful for us doing special effects.
It would help us put a 3D object in the right place inside a real movie.
Eventually, it could also be used to recreate the mouvement of the camera using markers.