Video Rendering Without FFmpeg: Inside the VideoFlow Server Engine

For years, the standard answer for programmatic video was simple: "Just use FFmpeg." But as any developer who has tried to maintain a production-grade video pipeline knows, "just" is doing a lot of heavy lifting. Between complex CLI flags, binary dependency management, and inconsistent environments, FFmpeg is a powerful but brittle foundation for modern SaaS applications.

At VideoFlow, we took a different path. While we support FFmpeg as an optional opt-in, our primary server-side renderer, @videoflow/renderer-server, is built on a modern, FFmpeg-free architecture. By leveraging headless Chromium and the WebCodecs API, we’ve created a pipeline that is more portable, more predictable, and significantly easier to deploy.

The Problem with the FFmpeg Shell

When you build a video engine around FFmpeg shell commands, you’re essentially concatenating strings to describe a complex, multi-layered timeline. This approach has three major drawbacks:

1. Deployment Friction: You have to ensure the correct FFmpeg binary (with the right codecs) is installed in your Docker container, Lambda function, or server.
2. Lack of Visual Parity: What you see in your local preview often differs from the final render because the rendering logic is split between your application code and the FFmpeg command string.
3. Opaque Internals: Debugging a failed render in a shell script is a nightmare of log-scraping and trial-and-error.

Why WebCodecs Changes Everything

The WebCodecs API allows us to move video encoding directly into the browser environment—even when that environment is a headless Chromium instance running on a server.

In the @videoflow/renderer-server pipeline, we use Playwright to drive a headless browser. Instead of recording the screen or piping frames to an external process, we use our internal library, MediaBunny, to capture raw frames from the canvas and feed them directly into the browser's hardware-accelerated encoder.

This means that the exact same rendering engine used in the VideoFlow Playground is the one producing your MP4 on the server. There is no translation layer, no "close enough" approximations, and no FFmpeg dependency required by default.

Building an FFmpeg-Free Pipeline

To see this in action, let’s look at how you’d set up a server-side render. First, you construct your video using the @videoflow/core builder:

import VideoFlow from '@videoflow/core';

const $ = new VideoFlow({ width: 1080, height: 1920, fps: 30 });

// Add a background image
$.addImage(
  { fit: 'cover' },
  { source: 'https://images.unsplash.com/photo-1618005182384-a83a8bd57fbe' }
);

// Add a title with a transition
const title = $.addText({
  text: 'NO FFMPEG REQUIRED',
  fontSize: 8,
  color: '#FF5A1F',
  fontWeight: 700,
  position: [0.5, 0.4]
});

title.fadeIn('800ms');
$.wait('3s');
title.fadeOut('500ms');

Then, you simply import the server renderer and call renderVideo.

import '@videoflow/renderer-server'; 

const buffer = await $.renderVideo({
  outputType: 'buffer',
  verbose: true,
});

// The buffer is now a valid MP4, ready to be saved or streamed.

By using @videoflow/renderer-server, you get a portable VideoJSON document that renders identically across our three official renderers. Whether you are exporting an MP4 in the user's browser tab or generating thousands of personalized videos on a server, the output is byte-for-byte identical.

Performance and Portability

One of the most common questions we get is: "Is it fast?" Because WebCodecs utilizes the browser's optimized encoding paths, performance is often comparable to, and in some cases faster than, traditional per-frame FFmpeg piping. More importantly, it is predictable.

This architecture is what allowed us to build features like the in-browser MP4 export, where the user can download their video without ever touching a server. The @videoflow/renderer-browser and @videoflow/renderer-server share the same core logic, just running in different host environments.

Conclusion

Moving away from FFmpeg isn't just about avoiding a binary dependency; it's about treating video like code. When your video is a portable JSON document and your renderer is a standard browser environment, you unlock new levels of automation and developer productivity.

If you're ready to stop fighting with shell scripts and start building cinematic video pipelines in pure TypeScript, head over to the VideoFlow GitHub to get started, or try it out instantly in our interactive playground.