One JSON file in, one finished reel out: how I render short videos with code

I make short explainer videos without opening a video editor. I write one JSON file that lists the scenes. A pipeline on my Mac speaks the script, times every word, draws the motion graphics in a web page, captures the frames and encodes a finished 1080×1920 video. It makes no API calls and uses no AI tokens per video, and a 55-second reel renders in about two to three minutes. Here’s how it works, and what I’d reuse in any automation.

This is part of ContentHQ, the system behind my videos.

The input: one file

Everything about a video lives in reel.json. A scene looks like this:

{
  "accent": "violet",
  "scenes": [
    {
      "theme": "dark",
      "text": "It changed **fourteen** files.",
      "chip": "FIRST",
      "widget": {
        "type": "files",
        "mode": "count",
        "count": 14,
        "cues": ["@fourteen"]
      }
    }
  ]
}

• text is what’s said and what’s shown. **stars** make a word pop.
• widget is the visual for the scene: a file list, a chat bubble, a gauge.
• cues say when the widget acts. "@fourteen" means “the moment the word fourteen is spoken”.

That last idea is the heart of the system. I never type a timestamp. I point at a word.

The pipeline, in six steps

1. Voice

Each scene’s text is spoken by Kokoro, a text-to-speech model that runs locally. Scenes are generated as separate clips, trimmed and joined, so scene boundaries are exact.

2. Word timing

A local Whisper model listens to the finished voice track and reports when each word starts. Now every caption word has a start time, accurate to a fraction of a second.

3. Timeline

A script turns the text markup into caption blocks and resolves every cue into seconds. "@fourteen" becomes a real time. The result is written to timeline.json.

4. Frames

This is the part I’m happiest with. The video is a web page. A small player, made of a stylesheet, a caption engine and a set of widgets, can draw any moment of the video from a single function, seek(t). Headless Chrome opens the page, steps through time one frame at a time, and saves each frame as an image.

5. Audio

The voice, a quiet music bed and a few sound effects are mixed and normalised to a consistent loudness, so every reel sounds the same level on a phone.

6. Encode

ffmpeg combines the frames and audio into an H.264 file at 30 frames per second.

Why a web page makes a good video renderer

I tried thinking of this as “video editing” and got nowhere. Thinking of it as “a web page that can show any timestamp” made it simple.

• The whole style is three files. Colours and fonts in CSS, caption behaviour in one script, scene visuals in another.
• Preview is instant. I open the player in a browser with the timeline and the voice track, and watch the reel play live. If I change a colour, I refresh.
• It’s deterministic. seek(t) always draws the same frame for the same time, so a render is repeatable.
• AI is good at it. Assistants are very good at HTML and CSS. A new widget is a small function, not a plugin for a video tool.

The cheap check before the expensive one

A full render takes a few minutes. Before that, I run a “stills” pass that takes about twenty seconds and produces contact sheets, with a few frames from every scene laid out in a grid. Most mistakes are visible there: a caption that wraps badly, a widget firing on the wrong word, a scene with too much text.

A fast, rough check before the slow, real one is worth building into any pipeline. It’s the same reason my deploy pipeline starts with a dry run.

Why run everything locally

• Cost. Voice, timing and rendering cost nothing per video, so I can render the same reel ten times while I adjust it.
• Speed. There are no upload queues and no rate limits.
• Control. The voice, the timing and the look stay the same until I change them.

The trade-off is setup. The voice model and its files take a few hundred megabytes, and the first run takes some patience. After that, it’s one command.

Where AI fits

AI doesn’t make the video. It helps upstream and downstream:

• Before: drafting the script from a topic, which I then edit.
• After: nothing. The render is plain code.
• Around: building the engine itself. The player, the widgets and the pipeline script were written with an AI coding assistant, one piece at a time, using the habits I follow for AI coding.

That split is deliberate. Use AI where judgment and language help, and plain code where the result should be the same every time.

What I’d reuse anywhere

1. One input file. If a human only edits one file, they’ll actually use the tool.
2. Refer to meaning, not numbers. “When this word is said” survives a script rewrite. A timestamp doesn’t.
3. Files between steps. Voice, timings, timeline and frames are each saved. If step five fails, steps one to four don’t run again.
4. A fast check first. Twenty seconds of stills saves many three-minute renders.
5. Keep the style in one place. Changing the look of every future video means editing one stylesheet.

One of the first reels I rendered this way was about a habit worth having when you code with AI: you ask for a logout button, and the AI changes fourteen files. The script became the example above. The advice in it became a blog post.