Voice Over Assistant, for recording audio over video

Scripting in Blender with Python, and working on the API

Moderators: jesterKing, stiv

Post Reply
Posts: 0
Joined: Sat Apr 20, 2013 7:02 am

Voice Over Assistant, for recording audio over video

Post by iND » Wed Apr 24, 2013 9:01 am

Voice Over Assistant.
Shows text to be read over a video clip, and recorded.

- have the lines enter at the correct time
- manually be able to adjust the timing of lines
- set the number of characters/words visible in a line
- improve loading/playing speed
- adjust font (face and size) without the huge processor hit

Works in the Sequencer, but not in Preview mode (yet). I would like to make this into an addon, so any suggestions would be greatly appreciated.

NOTE: You have to add a file name to function:


For the time being, the filename has to be manually entered.
Uses the Header. Should be a Menu, but it's a Panel right now.

Still getting the following error. Any ideas?

Code: Select all

    Python script fail, look in the console for now...
    Traceback (most recent call last):
      File "\Text", line 264, in draw
    UnboundLocalError: local variable 'sub' referenced before assignment

    location: <unknown location>:-1
Some properties:

voice_over_is_on = on/off
voice_over_pos_x, voice_over_pos_y = on-screen position (X,Y)
voice_over_text_color = text color

voice_over_script_filename = script filename (implementation in progress)

voice_over_visible_chars = number of characters shown onscreen (not implemented)
-- probably should be the maximum number
-- words are not divided
voice_over_font_face = font face (not implemented)
voice_over_multiline = allow multiple lines (not implemented)
voice_over_font_size = font size 12-24 (not implemented)
-- huge memory hit to change font size, since all oGL chars have to be redrawn

Code: Select all

import bpy
import bgl
import blf
import re
from bpy.types import Header, Panel, Operator

# return current filename
def get_script_filename(self):
    return Put your filename here.

# font drawing function
def draw_callback_px(self, context):
    if not context.window_manager.voice_over_is_on:
    scene = context.scene

    # check for resized window/region
    # reset visible chars to the max chars in order to force it to shorten later
    if self.regionWidth != int(context.region.width):
        self.regionWidth = int(context.region.width)
        self.visibleChars = self.maxVisibleChars
        self.scriptText = self.getScriptListFromFile(self.filename)

    # get the correct text line from the list
    if scene.use_preview_range:
        minFrm = scene.frame_preview_start
        maxFrm = scene.frame_preview_end
        minFrm = scene.frame_start
        maxFrm = scene.frame_end
    # no divide by zero
    if maxFrm == minFrm:
    curPercPos = max(0,min(1,(scene.frame_current - minFrm) / (maxFrm - minFrm)))
    showStr = self.scriptText[int((len(self.scriptText)-1) * curPercPos)]

    sqc = context.scene
    font_id = 0
    # changing the font size has a huge memory hit, since all the cached text has to be re-written
    # blf.size(font_id, sqc.voice_over_font_size, 72)
    blf.size(font_id, 21, 72)
    # get text line's onscreen width and height
    lineWid, lineHt = blf.dimensions(font_id, showStr)

    # shorten the max text length to fit onscreen
    if self.visibleChars > self.minVisibleChars and (sqc.voice_over_pos_x + lineWid) > (context.region.width - (sqc.voice_over_pos_x * 2)):
        self.visibleChars = max(self.visibleChars - 10, self.minVisibleChars)
        self.scriptText = self.getScriptListFromFile(self.filename)
        # --- RECURSION ---
        draw_callback_px(self, context)
        # bgl.glColor3f(1.0, 1.0, 1.0)
        bgl.glColor3f(sqc.voice_over_text_color[0], sqc.voice_over_text_color[1], sqc.voice_over_text_color[2])
        blf.position(font_id, sqc.voice_over_pos_x, sqc.voice_over_pos_y, 0)
        blf.draw(font_id, showStr)

class VoiceOverReader(Operator):
    """Voice Over Reader"""
    bl_idname = "sequencer.voice_over_reader"
    bl_label = "Voice Over Reader"

    _fileText = None
    _timer = None
    _handle = None

    regionWidth = 0

    scriptText = None
    maxVisibleChars = 300
    minVisibleChars = 30
    visibleChars = maxVisibleChars
    filename = None

    # returns a list of lines of text, split into words, but no longer than visibleChars length
    # it will be longer than visibleChars on certain occasions.
    def getScriptListFromFile(self, curfilename):
        if not self._fileText:
            if not curfilename:
                filename = get_script_filename(self)
                curfilename = filename
            self._fileText = open(curfilename, "rt").read()
            self._fileText = re.sub('[=]','',self._fileText)
            self._fileText = self._fileText.replace('\n\n','\n').replace('\n','\n(pause)\n')
            self._fileText = re.split('\s+',self._fileText)

        tf3 = [""]
        for wrd in self._fileText:
            if (len(tf3[len(tf3)-1]) + len(wrd)) > self.visibleChars:
            tf3[len(tf3)-1] += wrd + " "

        return tf3

    def modal(self, context, event):
        # sometimes the region is empty, so this error check catches that
            return {'PASS_THROUGH'}

        # main thingy
        if event.type == 'TIMER':
            draw_callback_px(self, context)
            return {'PASS_THROUGH'}

        # to quit
        if event.type in {'RIGHTMOUSE', 'ESC'}:
            return self.cancel(context)

        # this is last because this function has to clear the screen first (I think)
        if not context.window_manager.voice_over_is_on:
            return self.cancel(context)
        return {'PASS_THROUGH'}

    def invoke(self, context, event):
        self.scriptText = self.getScriptListFromFile(self.filename)
        # if context.screen.is_animation_playing:
        if context.window_manager.voice_over_is_on is False:
            # operator is called for the first time, start everything
            context.window_manager.voice_over_is_on = True
            self.regionWidth = 0
            self.visibleChars = self.maxVisibleChars
            self._timer = context.window_manager.event_timer_add(0.05, context.window)
            self._handle = bpy.types.SpaceSequenceEditor.draw_handler_add(draw_callback_px, (self, context), 'WINDOW', 'POST_PIXEL')
            return {'RUNNING_MODAL'}
            # operator is called again, stop displaying
            return self.cancel(context)

    # remove event listeners, etc.
    def cancel(self, context):

            bpy.types.SpaceSequenceEditor.draw_handler_remove(self._handle, 'WINDOW')
        context.window_manager.voice_over_is_on = False
        return {'CANCELLED'}

# properties used by the script
def init_properties():
    sequencer = bpy.types.Scene
    wm = bpy.types.WindowManager

    sequencer.voice_over_script_filename = bpy.props.StringProperty(
        name="Script Filename",
        description="Script Filename",
        # get=get_script_filename,
    sequencer.voice_over_pos_x = bpy.props.IntProperty(
        description="Voice over text Y position",
    sequencer.voice_over_pos_y = bpy.props.IntProperty(
        description="Voice over text Y position",
    sequencer.voice_over_text_color = bpy.props.FloatVectorProperty(
        description="Voice over text color",
        default=(1.0, 1.0, 1.0, 1.0),
    # sequencer.voice_over_font_size = bpy.props.IntProperty(
        # name="Size",
        # description="Voice over text size",
        # default=21, min=12, max=24)

    # Runstate initially always set to False
    # note: it is not stored in the sequencer, but in window manager:
    wm.voice_over_is_on = bpy.props.BoolProperty(default=False)

# removal of properties when script is disabled
def clear_properties():
    props = (
        # "voice_over_font_size",

    wm = bpy.context.window_manager
    for p in props:
        if p in wm:
            del wm[p]

# defining the header items
class SEQ_Voice_Over_Reader_display(Header):
    bl_label = "Voice Over Reader Display"
    bl_space_type = 'SEQUENCE_EDITOR'
    bl_idname = "SEQ_Voice_Over_Reader_display"
    def poll(cls, context):
        return (context.object is not None)

    def draw(self, context):
        wm = context.window_manager
        sc = context.scene
        layout = self.layout
        if not wm.voice_over_is_on:
            layout.operator("sequencer.voice_over_reader", text="Voice Over", icon = "PLAY")
            layout.operator("sequencer.voice_over_reader", text="Hide V.O.", icon = "PAUSE")
            row = layout.row()
            row.prop(sc, "voice_over_script_filename", text="")
            row = layout.row(align = True)
            # row.label(text="Position:")
            row.prop(sc, "voice_over_pos_x", text="X")
            row.prop(sc, "voice_over_pos_y", text="Y")
            row = layout.row()
            row.prop(sc, "voice_over_text_color", text="")

            # row = layout.row(align = True)
            # row.prop(sc, "voice_over_font_size")
def register():

def unregister():

if __name__ == "__main__":

# if __name__ == "__main__":  # only for live edit.
    # bpy.utils.register_module(__name__)

Posts: 0
Joined: Sat Apr 20, 2013 7:02 am

Addon project locations

Post by iND » Thu Apr 25, 2013 2:12 am

Posts: 0
Joined: Sun Apr 05, 2009 7:42 pm
Location: Germany

Post by CoDEmanX » Thu Apr 25, 2013 10:14 pm

Works in the Sequencer, but not in Preview mode (yet)
There's a simple problem: you can have SEQUENCER, PREVIEW or SEQUENCER_PREVIEW - both at the same time are possible. Thus, they can't both use the WINDOW region, it's only the SEQUENCER occupying this region. PREVIEW has its own region, 'PREVIEW'. So use that instead of 'WINDOW' when you add the draw handler and it will work!
- have the lines enter at the correct time
- improve loading/playing speed
- manually be able to adjust the timing of lines
You should design this addon like this:

1 - Let user locate subtitle file (maybe allow user to select a bpy.data.texts datablock as well, so internal text as well?)

2 - When user accepts a file with file dialog, build a lookup dictionary:
{frameNumber: "First line of text", frameNumber: "..."}
That should make the addon work really fast for playback, just look for the key scene.frame_current. You can test for playback with the property screen.is_animation_playing. If it's not playback, then you need a little more difficult code to find the appropriate key (min of all frameNumbers smaller or equal to frame_current). Make sure you rebuild the lookup dict whenever needed, rather provide a button to update text if there's no way to do it automatically in a nice way.

3 - Text changing is done with a bpy.app.handler.frame_change_post (or pre?), I don't think you need to update with a timer event...

4 - Allow text timing changes either via a panel in the sidebar of the sequencer area (template_list i suppose would work out fine, and could even allow user to change text *hm...*) OR abuse timeline_markers, but note that they support up to 63 chars and nothing more, and you can't add custom data to them. So user would place markers and for every marker, the next line of text would be shown - rather ugly if you add a line of text inbetween or remove a marker by accident... (Maybe transfer from template_list to timeline_markers?). Maybe there is a nicer way, to somehow keyframe a custom property so you could work with real keyframes to adjust text?
- set the number of characters/words visible in a line
you can utilize the .dimensions() function of the blf module to find out the text bounding box, with a trial-and-error algorithm, you could shrink text iteratively until it fits maybe.
- adjust font (face and size) without the huge processor hit
Not sure what you mean, how is this cpu expensive? Is blf that slow for fancy fonts? Not sure, but blender blf/bgl is quite limited, but maybe there is a chance to "cache" the text output, so you don't need to draw everything for every frame, but only on text changes?!
I'm sitting, waiting, wishing, building Blender in superstition...

Posts: 0
Joined: Sat Apr 20, 2013 7:02 am

Post by iND » Fri Apr 26, 2013 6:41 pm

CoDEmanX, thanks for the suggestions!

I'll try out 'PREVIEW'. That seems to be the key to this whole thing, after all.

I was looking for something like the frame_change_post command when I asked the question about timer+draw event duplication. This seems to be in the right direction.

I am working on file loading, but it was not critical at this stage. Simpler is better for dev. At some point in the near future, I will probably be asking questions about it. Stay tuned!

I am leaning toward timeline markers, and pretty close to the ideas that you have suggested. It is defining a dynamic intersection between two sets (frames and words), so there are only so many ways to do that.

The font issue may resolve itself. I am putting that on the back burner for now.

I have been having most of my trouble with the Blender python docs . . . they are incredibly unhelpful. Everything unknown seems to be about 6 degrees of separation from the last thing I did not know. I can find commands, but putting those commands in use has very little documentation, and why things end up working is occasionally a complete mystery. Are there any *GOOD* (thorough, detailed, wide-ranging) guides for Blender python? For instance, where did you learn about frame_change_post, and also window vs. preview?

Posts: 0
Joined: Sun Apr 05, 2009 7:42 pm
Location: Germany

Post by CoDEmanX » Fri Apr 26, 2013 10:34 pm

i learned about app.handlers by reading the docs and scripts which use them
http://www.blender.org/documentation/bl ... dlers.html

Well, mostly by looking at this template:
http://projects.blender.org/scm/viewvc. ... hrev=53207

Also commit log:
http://projects.blender.org/scm/viewvc. ... sion=53207

But there is actually no explanation of the params, you need to know that 'WINDOW' refers to a region.type and i found out that the preview is a custom region type by using the python console.

But mostly i learned from experimenting and looking at commit diffs.
I'm sitting, waiting, wishing, building Blender in superstition...

Post Reply