WebCodecs vs ffmpeg.wasm: Why In-Browser Video Conversion Got 10x Faster

For years, “convert video in your browser” meant one thing: ffmpeg.wasm. Download a ~31 MB WebAssembly blob, run FFmpeg in a sandboxed VM, wait. It worked, but it was slow enough that most converters just gave up and uploaded your file to a server instead.

That changed when WebCodecs shipped. Browsers now expose their built-in hardware video decoders and encoders directly to JavaScript, and the speedup isn’t 20% — on codec-heavy work it’s routinely 10x or more. If you’re building (or just curious why some browser tools are suddenly fast), here’s how the two approaches actually compare, where each one breaks, and why the right answer in production is usually both.

What WebCodecs actually is

WebCodecs is a W3C API that gives JavaScript direct access to the codec implementations the browser already ships — the same H.264/H.265/VP9/AV1 decoders that play YouTube, usually backed by your GPU or a dedicated media engine. You feed VideoDecoder encoded chunks, it hands you back raw VideoFrame objects, often without the pixels ever touching the CPU.

What it deliberately does not do: containers. WebCodecs has no idea what an MP4 or MKV file is. You bring your own demuxer (libraries like Mediabunny or @remotion/media-parser fill this gap), pull out the encoded packets, and hand those to the decoder.

Support, per caniuse:

Browser	WebCodecs support
Chrome / Edge	94+ (September 2021)
Safari	16.4+ video-only (no audio codecs until Safari 26)
Firefox	130+ on desktop; not yet on Android as of Firefox 151

So in mid-2026, full video decode support covers every current desktop browser and Chrome/Safari on mobile. The stragglers are Firefox for Android and anything more than a couple of years out of date.

What ffmpeg.wasm is, and what it costs

ffmpeg.wasm is the actual FFmpeg C codebase compiled to WebAssembly. That’s its superpower: it understands nearly every container, codec, filter, and weird edge case FFmpeg does, because it is FFmpeg. Pass it the same CLI arguments you’d use in a terminal.

The costs are structural, not bugs:

• Download size. The standard @ffmpeg/core wasm build is roughly 31 MB. That’s a one-time, cacheable hit, but it’s a brutal first-load experience on mobile data.
• No hardware access. WebAssembly can’t reach the GPU’s media engine. Every frame is decoded and encoded in software, in a VM.
• Single-threaded by default. The multi-threaded @ffmpeg/core-mt build needs SharedArrayBuffer, which browsers only enable on cross-origin-isolated pages — meaning your server must send Cross-Origin-Opener-Policy: same-origin and Cross-Origin-Embedder-Policy: require-corp headers. Those headers break embedded third-party iframes and some CDN setups, so plenty of sites can’t ship them and fall back to the slow single-threaded core. Per ffmpeg.wasm’s own benchmarks, the multi-threaded build is only about 2x faster anyway (60.4s vs 128.8s on a test conversion that native FFmpeg finishes in 5.2s).
• Memory ceiling. The project FAQ is blunt: maximum input file size is “2 GB, which is a hard limit in WebAssembly.” Files live in an in-memory filesystem, so input, output, and working buffers all compete for the same 32-bit address space. In practice things get shaky well before 2 GB, especially on phones.

The benchmarks, with sources

Numbers from people who’ve measured this, not vibes:

Task	WebCodecs path	ffmpeg.wasm	Source
Convert 1080p H.264 to WebM + resize to 320×180	804 frames/s	12 frames/s	Mediabunny benchmark, Ryzen 7600X + RTX 4070, June 2025
Iterate all video packets (691 MiB file)	10,800 packets/s	2,390 packets/s	same
Extract metadata	862 ops/s	1.83 ops/s	same
General conversion	”at least 5x faster”	baseline	remotion-dev/webcodecs-benchmark

The BurnSub writeup on building a browser-local subtitle burner lands on the same shape without exact multipliers: software encoding runs “roughly an order of magnitude slower than hardware-accelerated paths on the same machine,” and a WebCodecs pipeline gets a 60-second clip encoded in 30–60 seconds of wall time on modern hardware. The 67x gap in Mediabunny’s transcode test is a best case (desktop GPU, friendly codec); 5–15x is a more honest expectation across average hardware.

One nuance worth internalizing: the gap is in decode and encode, not everything. If your workload is mostly filters, format gymnastics, or audio resampling, FFmpeg’s breadth still wins — there is no WebCodecs equivalent of -vf filter graphs.

What this means for GIF conversion specifically

A video-to-GIF pipeline has two halves with very different profiles:

1. Decode the video. This is the codec-heavy part, and it’s exactly what WebCodecs accelerates. Decoding a 1080p H.264 clip in wasm at ~12–25 fps means a 30-second 30fps clip takes 40–75 seconds just to read. WebCodecs chews through the same clip in a few seconds.
2. Encode the GIF. GIF is a 1987 format — 256-color palettes, LZW compression. No GPU on earth has a hardware GIF encoder, so this half is CPU work (palette quantization + dithering) no matter which stack you use. It’s fast, but it’s the part you can’t accelerate away.

So a WebCodecs-based converter doesn’t make GIF encoding faster; it removes the decode bottleneck that used to dominate. In practice that turns “go make coffee” into “done before you switch tabs” for typical clips, and a 10-second 480px 15fps result still lands around 4–8 MB — if that’s too big for your target platform, run it through a GIF compressor afterward rather than re-converting at lower quality.

The hybrid architecture: fast path + fallback

Neither tool alone covers all users:

• WebCodecs-only fails on Firefox for Android, older browsers, and the occasional codec the browser refuses to decode (some 10-bit HEVC, exotic profiles).
• ffmpeg.wasm-only is slow for everyone and front-loads a 31 MB download even for users whose browser could have done the job in hardware.

The pattern that’s emerged — and the one GIF Den’s video to GIF converter ships — is a tiered pipeline:

1. Feature-detect. 'VideoDecoder' in window, then VideoDecoder.isConfigSupported() with the actual codec string from the demuxed file. Detection is cheap and per-file, not per-browser.
2. Fast path. Demux in JS, decode via WebCodecs, paint frames to canvas, quantize and encode the GIF in a worker. No wasm download at all for the ~90%+ of users on current Chrome, Edge, Safari, or desktop Firefox.
3. Fallback. If detection fails or the decoder errors mid-stream, lazily fetch ffmpeg.wasm and run the same job in software. Slower, but it completes, and the user never sees a “browser not supported” wall.

The fallback also covers inputs WebCodecs can’t: weird containers, ancient codecs, files FFmpeg shrugs at and handles anyway. The reverse direction matters too — going from GIF back to a real video format (GIF to MP4) needs an encoder, and VideoEncoder support follows roughly the same browser matrix.

A side benefit of either approach: nothing leaves your machine. Both stacks are genuinely client-side, which you don’t have to take on faith — open devtools and watch the network tab, or see how to verify no-upload claims for the two-minute audit.

Quick decision table

Your situation	Use
One codec-heavy step (decode or encode), modern browsers	WebCodecs
Need filters, subtitle rendering via filtergraph, obscure formats	ffmpeg.wasm
Audio processing in Safari < 26	ffmpeg.wasm (Safari’s WebCodecs was video-only until 26)
Files over ~1.5 GB	WebCodecs (streamed) — wasm’s 2 GB memory limit will bite
Shipping to real users in production	Hybrid: WebCodecs fast path, wasm fallback

Checklist if you’re building this

• Feature-detect with isConfigSupported() per file, not navigator.userAgent.
• Demux yourself (Mediabunny, mp4box.js, or @remotion/media-parser) — WebCodecs won’t.
• Lazy-load the wasm bundle only when the fast path fails; never on page load.
• If you do ship @ffmpeg/core-mt, confirm window.crossOriginIsolated === true at runtime before assuming threads exist.
• Budget memory: wasm input + output must fit under 2 GB combined, less on mobile.
• Process in a worker either way; a pegged main thread reads as “frozen” to users.
• Expect WebCodecs decode errors on a small tail of files and treat them as a routing signal to the fallback, not a hard failure.