The Three-Renderer Rule: Why Portable VideoJSON is the Future of Automation

Most video automation pipelines today are trapped in a single environment. You either have a brittle FFmpeg shell script running on a Linux server that is impossible to preview locally, or a heavy React-based runtime that works beautifully in the browser but becomes a resource-hungry nightmare to scale headlessly. This fragmentation creates "renderer drift"—the frustrating reality where your "live preview" is just a rough approximation of the final MP4 export.

To build truly resilient video infrastructure, you need to follow the Three-Renderer Rule: a single, portable source of truth that renders byte-for-byte identically across the browser, the server, and the live preview. At the heart of this rule is VideoJSON.

The Problem with Environment-Locked Video

Traditionally, programmatic video has been a tale of two extremes. On one side, you have the FFmpeg approach. It's powerful and ubiquitous, but it's "stringly-typed." Composing a complex scene with transitions and GLSL effects requires concatenating massive strings of filter graphs. There is no "live preview" for an FFmpeg command; you run it, wait for the encode, and hope for the best.

On the other side, modern tools like Remotion have brought the power of React to video. This is a massive leap forward for developer experience, but it introduces a new kind of lock-in: your video is now a React component tree. To render it on a server, you must spin up a full React runtime. To store it in a database, you have to serialize complex component logic.

VideoJSON acts as the universal intermediate representation for video pipelines.

Enter the Three-Renderer Rule

The Three-Renderer Rule states that your video definition should be decoupled from the rendering engine. Instead of being "code that renders a video," your project should be "data that describes a video." In VideoFlow, this data is VideoJSON.

By treating video as a portable JSON document, you enable three distinct rendering paths from the exact same source:

1. The Browser Renderer (@videoflow/renderer-browser): Exports high-quality MP4s directly in the user's tab using WebCodecs. This allows for "zero-cost" rendering where the user's hardware does the heavy lifting, perfect for client-side tools.
2. The Server Renderer (@videoflow/renderer-server): Runs headlessly in Node.js, driving Chromium via Playwright to produce the same MP4. This is the workhorse for batch automation and background jobs.
3. The DOM Renderer (@videoflow/renderer-dom): Provides a frame-accurate, 60fps live preview inside your web app. This is what powers the VideoFlow Playground and the React Video Editor.

Because all three renderers consume the same VideoJSON schema, you eliminate renderer drift. What you scrub through in your editor is exactly what your server will encode at 4K.

Why VideoJSON is Resolution-Agnostic

One of the biggest hurdles in programmatic video is handling different aspect ratios and resolutions. If you hardcode a title at x: 1920, y: 1080, your video breaks the moment you try to render a vertical version for TikTok.

VideoJSON solves this by using normalized coordinates and "em" units. In the VideoFlow ecosystem, 1em is defined as 1% of the project width. A fontSize of 5 is always 5% of the width, whether you are rendering at 720p or 8K.

import VideoFlow from '@videoflow/core';

const $ = new VideoFlow({ width: 1920, height: 1080, fps: 30 });

// Position is normalized 0..1 (center is [0.5, 0.5])
// fontSize is in em (1em = 1% of width)
const title = $.addText({
  text: 'The Three-Renderer Rule',
  fontSize: 6,
  color: '#FF5A1F',
  position: [0.5, 0.4],
});

title.fadeIn('600ms');
$.wait('3s');

// This compiles to a portable VideoJSON document
const json = await $.compile();

This portability is why we argue that your MP4s should live as version-controlled JSON. You can store this JSON in a Postgres JSONB column, diff it in Git, and send it across the wire to any of the official renderers.

Headless Video Rendering: WebCodecs vs FFmpeg

A common question when building headless video rendering pipelines is whether to use FFmpeg or a browser-based approach. FFmpeg is the industry standard for encoding, but it lacks a modern layout engine. It doesn't understand CSS, web fonts, or complex GLSL transitions out of the box.

@videoflow/renderer-server takes a different path. By default, it uses a combination of WebCodecs and MediaBunny inside a headless Chromium instance. This means you get the full power of the modern web platform—SVG, Canvas, WebGL—without the overhead of managing FFmpeg binaries on your server. If you do need the specific control flags of x264, VideoFlow offers an optional FFmpeg-based pipeline that pipes frames from the browser into an FFmpeg process.

VideoFlow bridges the gap between high-level builder APIs and low-level rendering hardware.

How VideoFlow Handles the Rule

The VideoFlow toolkit was designed from day one to support this architectural split. The @videoflow/core package is a pure TypeScript builder that knows nothing about the DOM or encoding. Its only job is to produce a valid VideoJSON.

This separation of concerns is what makes VideoFlow the ideal choice for Remotion alternatives. While other tools bundle the "what" and the "how" together, VideoFlow gives you the flexibility to build your timeline in one place and render it in another.

Whether you are building a YouTube Shorts factory or an embeddable video editor, the Three-Renderer Rule ensures that your pipeline is scalable, portable, and—most importantly—predictable.

Getting Started with Portable Video

Ready to stop fighting your rendering pipeline? Here is how to start:

1. Explore the Playground: See VideoJSON in action and watch the DOM renderer handle complex GLSL effects at 60fps.
2. Read the Renderers Guide: Learn how to set up the browser and server renderers in your own project.
3. Check the GitHub Repo: VideoFlow is open-source (Apache-2.0). Star the repo and join the community building the future of programmatic video.

By adopting the Three-Renderer Rule, you aren't just choosing a library; you're choosing an architecture that treats video as the first-class data type it deserves to be.