Why You Should Store Your Videos as JSON, Not MP4

For decades, the MP4 has been the final word in video distribution. It’s the "compiled" binary that we upload to YouTube, embed in Slack, and send to clients. But if you are building a SaaS product, an automated marketing engine, or an LLM-driven content pipeline, the MP4 is actually your biggest bottleneck.

When video becomes programmatic, treating it as a static binary file is like storing your web application as a series of screenshots instead of source code. It’s heavy, opaque, and impossible to diff.

At VideoFlow, we believe the future of programmatic video isn't binary—it's Video as JSON. By shifting the source of truth from a rendered file to a portable VideoJSON document, you unlock a level of flexibility that traditional video pipelines simply cannot match.

The Portability Problem

A typical 10-second 1080p MP4 might weigh 5MB to 20MB. A VideoJSON document describing that same 10 seconds—including every layer, transition, and GLSL effect—is usually less than 50KB.

When you store your videos as JSON, your database doesn't need an S3 bucket for every version of every video. You can store thousands of complex video definitions directly in a Postgres jsonb column. This doesn't just save on storage costs; it changes how you architect your application. You can fetch, clone, and manipulate video scenes with the same latency as a user profile update.

Because VideoFlow is open source and built on a standardized schema, this JSON is truly portable. The same document you generate in your backend can be sent to the Playground for debugging, or passed to @videoflow/renderer-dom for a 60fps live preview inside your React app.

Version Control for Video

Have you ever tried to run a git diff on two versions of an MP4? It’s impossible. If a designer changes a hex code or a developer tweaks an animation easing in a binary file, you have no way of knowing what changed without watching both videos side-by-side.

When your video is JSON, it lives in your repository.

// A snippet of VideoJSON showing an animatable property
{
  "type": "text",
  "properties": {
    "text": "Welcome to VideoFlow",
    "color": "#FF5A1F",
    "fontSize": 6,
    "position": [0.5, 0.4]
  },
  "animations": [
    { "property": "opacity", "from": 0, "to": 1, "duration": 0.5 }
  ]
}

With this approach, a "video edit" is just a pull request. You can see exactly which line of the timeline changed, who changed it, and why. This brings the rigor of modern software engineering to video production. You can branch your video content, run automated tests against the JSON schema, and roll back changes instantly if a render fails.

The Three-Renderer Advantage

The real magic happens when you realize that JSON doesn't care where it is rendered. In a traditional pipeline, if you want a preview, you usually have to render a low-res MP4 on the server and stream it back. This is slow and expensive.

VideoFlow’s architecture is built around the "Three-Renderer Rule":

1. Live Preview: Use @videoflow/renderer-dom to render the JSON directly to the browser's DOM for instant, frame-accurate scrubbing.
2. Client-Side Export: Use @videoflow/renderer-browser to turn that JSON into an MP4 using WebCodecs, right in the user's tab—zero server cost.
3. Headless Automation: Use @videoflow/renderer-server in Node.js to batch-render thousands of personalized videos in the cloud.

By keeping the video as JSON until the very last millisecond, you avoid the "render-wait-fix-render" loop that plagues creative teams.

Dynamic Personalization at Scale

If you're building a Spotify Wrapped style recap, you aren't rendering one video—you're rendering millions. Storing millions of MP4s is a nightmare. Storing one JSON template and millions of small data payloads is trivial.

You can even let AI agents participate in the process. As we explored in our guide on building LLM-powered video pipelines, language models are incredibly good at emitting structured JSON. It is far easier to prompt an LLM to "output a VideoJSON document with a blue background and a fade-in title" than it is to have it manipulate a binary video file.

How VideoFlow Handles the Handoff

The @videoflow/core library is designed specifically to make this JSON-first workflow seamless. You use a fluent TypeScript builder to compose your scene, and then call $.compile() to produce the portable VideoJSON.

import VideoFlow from '@videoflow/core';

const $ = new VideoFlow({ width: 1080, height: 1080 });

const title = $.addText({
  text: 'JSON is the new MP4',
  color: '#FF5A1F',
  fontSize: 8
});

title.fadeIn('500ms');
$.wait('2s');

// This is the moment your video becomes a portable data object
const videoData = await $.compile();

From here, you can save videoData to your database, send it over a WebSocket, or ship it to a serverless function for rendering. You aren't moving pixels; you're moving instructions.

Start Building with VideoJSON

Moving away from the MP4-as-source-of-truth might feel like a big shift, but for anyone building at the intersection of code and content, it is the only way to scale. It turns video from a heavy, static asset into a dynamic, versionable, and lightweight data type.

Ready to see what VideoJSON looks like in action? Head over to our official documentation to explore the schema, or try building your first scene in the VideoFlow Playground. The future of video is human-readable, machine-writable, and entirely open source.