New browser APIs unlock new possibilities

Published on: April 29, 2023

When new browser APIs get announced or are implemented, like the recent announcement from the Chrome team about WebGPU, there is an inevitable push back:

[Is] this yet another information leak anti-feature that we need to disable? [source]

Nobody needs new fancy features, what people really need is more reliable fingerprinting [source]

The browser truly is the new OS, for better or for worse. [source]

They couldn’t secure our OSes to run untrusted code safely, so they built a OS on top of a OS.

There is merit to ensuring standards arrive at a thoughtful specification, but I don’t like how these comments disparage the hard work that has gone into these features. In an alternate universe, an app of mine would be severely limited without these features.

So I figured I’d list out all of the new APIs that I leverage.

New APIs

WebGL2. You see that cool map? It’s generated entirely client side from a file that the user selects from their system. WebGL2 is critical to be able to create this map that can be panned and zoomed around flawlessly. Previous implementations to crunch the file and spit out the 11.5 million pixels ran on the CPU, but couldn’t exceed 4 frames a second even after optimizations. I have a separate article on how to simulate EU4’s map in the browser with WebGL

WebAssembly. The entire app relies on WebAssembly to process the selected file, whether on client side or on server side after the user decides to upload the file. This allows for almost all Rust code written to be shared between the client and server, and it is one of the main reasons why I wrote about feeling vindicated for structuring everything around Rust. Everything from CLIs, shared libraries, Wasm, to backend services share the same code.

Compression Streams API. The files that are uploaded are often embarrassingly compressible. We’re talking about a size reduction of 10-20x, which saves a significant amount of bandwidth and time. Being able to use the builtin deflate implementation is a nice option if on hand. Otherwise it can be polyfilled with libraries like pako or your codec of choice (eg: brotli / zstd) with WebAssembly. I wrote an article and a side project that benchmarks compression implementations.

VideoEncoder. The referenced map screenshot is not static. It can transform into a timelapse, showing how borders change over time. Like everything else, the timelapse is done client side, but then the question arises: what if a user wants to share a timelapse they created? Initially, the MediaStream Recording API was used and it worked by recording the canvas at a set interval as the timelapse advanced. This worked well for several months, but the result wasn’t perfect as the frames in the webm output were inconsistent. Some frames were too long, while other dates in the timelapse were skipped. The fix was to transition to the VideoEncoder API where there’s improved coordination and everything can progress in tandem due to the duration being encoded in the frames. Sprinkle in a supplementary webm muxer and the timelapse is much more consistent and faster to generate because users no longer have to sit through the entire timelapse, which includes pauses for each date.

File System Access API. The user selected file may change while the user is interacting with the site. Wouldn’t it be great to have the user opt into a feature such that when the underlying file changes, the UI is refreshed? Until progress is made on the File System Change Observer proposal, polling via the File System Access API is the only way to achieve this behavior.

Web Worker. A classic API, but it’s absolutely critical to be able to process these large files without blocking the UI thread.

Shared Array Buffer. The webm output from the timelapse isn’t usable everywhere, so users can opt into using ffmpeg.wasm to transcode it into an h.264 mp4. ffmpeg.wasm requires the use of Shared Array Buffers.

There are more APIs, like the storage APIs of local storage and IndexedDB (though I no longer use IndexedDB after finding out it does not transparently compress data), but I think the gist is clear: the browser APIs are extremely useful.

Browsers are frictionless

The best part is that users get to experience these features without installing additional software (or even signing up). How else would I have been able to distribute the app frictionlessly? I couldn’t have. These browser APIs make the impossible possible.

No app will touch the entire API surface area that browsers expose. I don’t see myself touching bluetooth or MIDI devices, but I can see how they enable new types of websites.

I know security and privacy are major concerns, and I don’t have a solution. Some want to see a browser API consent banner to go along with our cookie consent banners, but I think that the lack of persistent permissions is already a sore spot of the File System Access API, and I can’t imagine doubling down on this philosophy is better.

Having users download and install software is way more intrusive and prone to abuse, and too many people are forgetting this. New browser APIs unlock ideas that would have never been possible before.