Voice tracking

Normally the display scrolls at a pace you set, and you read to keep up with it. Voice tracking turns that round. The display listens through the microphone, works out which words you have already said, greys them out, and moves the script so the next thing to say is on the reading line. Stop talking and it stops. Skip a paragraph and it skips with you.

Turning it on

The microphone button sits next to the transport controls on the display, and the keyboard gets V. There is nothing to switch on first.

The first time you press it the browser asks for permission to use the microphone. If you refuse, or have refused before, the readout under the script says so and voice tracking switches itself back off; the permission has to be changed in the browser’s own site settings, not here.

Whether the room is following a voice is shared, like the pace and the type size: turn it on from the phone and the display starts listening, and both devices show it. The spoken language is the exception, and is set on the display that does the listening, because it is a property of that browser rather than of the room.

Using it

Once it is listening:

• Words you have said go dim, in the reading surface’s own muted tone, so the brightest text on screen is always the sentence you are in the middle of.
• A small orange rule sits under the next word out. It is deliberately not a highlight: a block of colour under the word you are about to say is precisely where you do not want your eye pulled.
• A strip under the script shows the last few words the display heard, and how many of the script’s words have been matched. It is there for the moment something goes wrong: it tells you whether it misheard you or simply lost the line, which look identical without it.

Voice tracking and constant-speed playback are two different things moving the same text, so only one of them ever runs. Pressing play, or Space, stops the listening. Pressing the microphone stops the clock. Pace in words per minute is ignored entirely while it is listening, because your actual delivery is the pace.

Scrubbing, tapping a line on the remote, stepping and Home all still work and all win outright. Moving the script by hand tells voice tracking to forget where it thought you were and pick you up from where you have just put it.

How it finds your place

The obvious approach (remember the last word matched, compare the next word heard to the next word written) falls apart on the first line of real use. People skip words, add words, say “and” where the page says “&”, and recognition mishears roughly one word in ten. Any of those desynchronises a simple pointer permanently.

So instead the last dozen words you said are matched as a phrase against the part of the script you were already in, allowing for insertions, deletions and substitutions, and scored. Two or three solid word matches anywhere in that phrase are enough to place you precisely, and everything else in it can be wrong at no cost.

Searching a window rather than the whole script is the other half of it. A script says “thank you” six times; a search over the whole document would send you to the wrong one. Looking only a couple of sentences either side of where you already were makes a repeated phrase unambiguous.

If nothing matches for about four seconds, it assumes you are somewhere else entirely and searches the whole script instead, at a much higher bar for certainty, because moving you to the wrong page is worse than not moving you at all. That is what lets you jump to a different section mid-take: say a few words from it and it will find you.

From the remote

The remote has a microphone button too. It does not listen; the microphone stays on the display, which is the device the person speaking is standing in front of. The button asks the display to start and stop.

The remote also greys out the spoken words on its mirror, which it can do without hearing anything: it already knows where the display has got to, and everything above the reading line has been read.

Where the audio goes

Recognition is the browser’s, not ours. Teleprompt asks the browser for a transcript and receives words back; there is no API key to obtain, no service to sign up for, and nothing added to a self-hosted deployment to make this work.

What that means in practice depends on the browser. Chrome and Edge stream the audio to their vendor’s speech service and return text. Safari does more of the work on the device. In neither case does the audio reach a Teleprompt server, and in neither case is a transcript stored; the words are matched against your script in the page and discarded.

This is the one place Teleprompt sends something you produce anywhere other than your own devices, which is why the settings panel says so next to the control, and why nothing listens until you press the microphone. Privacy and data covers everything else.

Where it struggles

• Noisy rooms and background music. Recognition quality is the whole ceiling here, and it degrades before you would expect it to.
• Languages written without spaces: Chinese, Japanese, Thai. The matching works on whitespace-separated words, so scripts in those languages will not track well. The language picker is in the display’s settings and is set per device.
• Long silences. Browsers end a recognition session on their own schedule; a new one is started immediately and invisibly, but a word said in the gap can be missed.
• Scripts that repeat themselves closely. Two near-identical paragraphs a few sentences apart can be confused after a re-sync.
• Firefox. It has no built-in speech recognition, so the switch is unavailable there and says so.

A connection is needed in Chrome and Edge, because their recognition is a service rather than something on the device. On a venue network you do not trust, set a pace and use the remote.