How to Build a Video Rendering API in Node.js (Without FFmpeg)

For years, building a video rendering API in Node.js meant one thing: wrestling with FFmpeg. You’d spend days concatenating complex shell commands, managing temporary file systems, and praying that the binary was correctly installed in your production environment. If you wanted to move beyond simple clips to cinematic content—with GLSL effects, dynamic text, and frame-perfect transitions—the complexity grew exponentially.

But the web has changed. With the advent of WebCodecs and headless browser automation, we can now treat video rendering as a first-class citizen of the modern web stack. In this tutorial, we’ll build a high-performance video rendering service that turns JSON definitions into MP4 files using nothing but TypeScript and VideoFlow.

The Architecture of a Modern Video Pipeline

Traditional pipelines are "stringly-typed." You pass a string to a shell, and it returns a file. A modern headless video rendering architecture, however, treats the video as a document—specifically, a portable VideoJSON document.

This approach offers three major advantages:

1. Portability: The same JSON that powers your live preview in a React dashboard can be sent to a Node.js worker for final rendering.
2. Consistency: Because VideoFlow uses the same rendering logic across the browser and the server, you never have to worry about "it looks different in the preview."
3. Speed: By leveraging @videoflow/renderer-server, we can bypass the overhead of per-frame screenshots and re-encoding, using browser-native WebCodecs to produce the MP4 directly.

Step 1: Defining the Video Schema

First, we need to create our video composition. Unlike other tools that require you to manage frame numbers manually, VideoFlow uses a fluent builder API that handles timing and flow for you.

import VideoFlow from '@videoflow/core';

export async function createVideo() {
  const $ = new VideoFlow({ width: 1080, height: 1920, fps: 30 });

  // Add a background image with a blur animation
  const bg = $.addImage(
    { fit: 'cover', opacity: 0.8 },
    { source: 'https://images.unsplash.com/photo-1618005182384-a83a8bd57fbe' }
  );

  bg.animate(
    { filterBlur: 0 },
    { filterBlur: 10 },
    { duration: '5s', wait: false }
  );

  // Add a title with a cinematic transition
  const title = $.addText(
    {
      text: 'PROGRAMMATIC VIDEO',
      fontSize: 8,
      color: '#FF5A1F',
      fontWeight: 800,
      position: [0.5, 0.4]
    },
    {
      transitionIn: { transition: 'blurResolve', duration: '800ms' }
    }
  );

  $.wait('4s');
  title.fadeOut('1s');

  return await $.compile();
}

In this snippet, we’re using the VideoFlow core builder to define a 5-second vertical video. Notice the use of normalized coordinates [0.5, 0.4] and em units for font size—this ensures our video is resolution-agnostic.

Step 2: Implementing the Rendering Endpoint

Now, let's wrap this in a Node.js API. We'll use @videoflow/renderer-server, which uses Playwright to drive a headless Chromium instance. This is the secret to why VideoFlow doesn't need FFmpeg for most tasks—it uses the browser's own encoding capabilities.

import express from 'express';
import VideoFlow from '@videoflow/core';
import '@videoflow/renderer-server'; // Registers the server renderer
import { createVideo } from './video-source';

const app = express();

app.post('/render', async (req, res) => {
  try {
    const $ = new VideoFlow();
    // In a real app, you'd populate the project from req.body or a template
    const json = await createVideo();
    
    // Load the project into the builder
    $.load(json);

    console.log('Starting render...');
    
    // Render directly to a buffer
    const buffer = await $.renderVideo({
      outputType: 'buffer',
      verbose: true
    });

    res.setHeader('Content-Type', 'video/mp4');
    res.setHeader('Content-Disposition', 'attachment; filename="render.mp4"');
    res.send(buffer);
    
  } catch (error) {
    console.error('Render failed:', error);
    res.status(500).send('Internal Server Error');
  }
});

app.listen(3000, () => console.log('Video API listening on port 3000'));

Scaling the Pipeline

When you're building a video rendering API in Node.js, concurrency is your biggest challenge. Each render launches a headless browser, which is CPU and memory intensive.

For production workloads, you shouldn't render directly inside the HTTP request. Instead:

1. Queue the Job: Use a library like BullMQ to push the VideoJSON to a Redis queue.
2. Worker Pool: Have a pool of workers subscribe to the queue. Each worker calls $.renderVideo() and uploads the result to S3.
3. Webhook: Once the upload is complete, send a webhook back to your main application to notify the user.

This pattern allows you to scale horizontally by adding more worker nodes without impacting the responsiveness of your API.

Why VideoFlow is the Best Choice for APIs

If you've looked at Remotion alternatives, you'll notice that most are either proprietary or tied heavily to a specific framework like React. VideoFlow is different.

• Apache-2.0 Licensed: The core and all three renderers (Browser, Server, and DOM) are completely open source. You can embed them in your commercial SaaS without a licensing headache.
• JSON Portability: Because the source of truth is JSON, you can generate videos from Python, Go, or even directly from an LLM.
• Zero-FFmpeg Dependency: By default, @videoflow/renderer-server uses WebCodecs inside Chromium, making it significantly easier to deploy on serverless platforms like AWS Lambda or Google Cloud Run.

Get Started

Ready to build your own video factory? You can start experimenting right now in the VideoFlow Playground, where you can write code and see the results in real-time. For more advanced implementations, check out our comprehensive documentation or explore the official renderers guide.

If you find VideoFlow useful, consider giving us a star on GitHub—it helps us keep the project growing!