A Bevy app entirely off the main thread

There are several options for how to structure a Bevy app for the web:

1. Run the Bevy app and canvas on the main thread
2. Run the Bevy app on the main thread but with an OffscreenCanvas
3. Run a Rust “window” on the main thread with the Bevy app and OffscreenCanvas in a web worker
4. Remove Rust from the main thread and run everything in a web worker

As we progress further down the list a bit more elbow grease is required for each one as we stray further from the easy path and walk into a land devoid of examples.

Option 4, especially, is the black sheep of the family, as Bevy manages window and input events through winit, and winit requires that it is run on the main thread.

Winit requiring the main thread makes sense – how else is winit supposed to respond to window and input events? Web workers don’t get to “see them”. However, it appears winit may be able to smuggle events from the main thread to workers.¹

The consequence of this is if we want to run everything in a web worker, we need to ditch winit. And running bevy with a different window handler is walking into relatively uncharted territory. Though I won’t be the first intrepid individual to do this², we can put a new spin by compiling against the upcoming bevy 0.16 release, writing a more simplified, idiomatic solution, and keeping all our Wasm in a web worker.

Is this even a good idea?

While it is possible to create a smooth Wasm experience on the main thread³, it is too easy to accidentally compute something intense and start causing jank in the UI⁴. This is especially true as an application grows in complexity.

Running Wasm, even a large Wasm bundle, as an orchestrator on the main thread is ok, as Wasm compilation is asynchronous, cacheable, and happens off the main thread when utilizing WebAssembly.instantiateStreaming() (though, in general, just because a function is asynchronous doesn’t mean it can’t have serious ramifications for blocking the main thread⁵).

Since web worker compute is table stakes, options 1 and 2 are off the table, as they involve running logic on the main thread. Sometimes the easy path is not the right path to take.

Option 3 is enticing if Rust can be used everywhere, including the UI. Seeing bevy embedded inside a leptos app with effective signals is powerful⁶, though with the Wasm weighing at 44 MB, the 1 second delay to first contentful paint on a high end desktop machine is unacceptable.

In Javascript this problem is solved with bundle splitting and lazy loading, but the hurdles to bundle splitting Wasm are high and thus not typical. Leptos acknowledges this and recommends one seek alternatives to bundle splitting⁷. The best UX is probably serving up a static html page with a loading spinner. I’d normally reach for server side rendering to at least take advantage of concurrent data fetches, but server side rendering a UI through Wasm might be too bleeding edge, even for me!

Another option is to manually bundle split by having two crates: a UI Wasm payload load the App Wasm payload. I’d be wary of the DX for the UI Wasm payload to reference a digested asset to maximize cache effectiveness. And I wouldn’t think that any communications between the two modules would be facilitated just because both are written in Rust when each is playing in their own sandbox.

There is an argument that writing the UI in Rust could ease writing a desktop UI and web app, but a recent survey of Rust GUI libraries⁸ leads me to believe that we are still a couple years out from a good solution, so be prepared for a potential rewrite down the road.

Despite being bearish on Rust and Wasm UI, I’ll forever defend executing business logic with Wasm⁹, it’s just that lines get a little blurred when coding a Bevy app or game, as business logic is intimately entwined with the UI.

This is why I want to explore option 4. I can stick with the battle proven JS ecosystem (a potentially ironic stance given how relatively fast moving it is), as I know how to deliver an excellent experience from an infrastructure, design, developer, and user perspective with JS. And I minimize the chance of this being compromised if all Wasm is moved to a web worker.

Enough pontificating, onto the code!

The Web Worker

Let’s start with the web worker code. The wasm app is assumed to have been built with.

wasm-pack build -t web my-app-wasm

import init, { BevyApp } from "my-app-wasm";

// Get the URL of our digested Wasm to maximize cache effectiveness
import wasmPath from "my-app-wasm/my-app_wasm_bg.wasm?url";

// Use comlink (the only sane way to use web workers)
import { proxy } from "comlink";

const initialized = init({ module_or_path: wasmPath });

export const createGame = async (
  ...args: ConstructorParameters<typeof BevyApp>
) => {
  await initialized;

  // Args contains whatever the app needs, like the offscreencanvas
  // with the current dimensions of the backing canvas element
  const app = new BevyApp(...args);

  // Since we aren't using winit, we need to drive our app 
  // (animation, timers, game logic). Beneath the hood, winit
  // also uses requestAnimationFrame
  function update() {
    app.update();
    requestAnimationFrame(update);
  }
  requestAnimationFrame(update);

  // Our web worker needs to expose functions that accept input events.
  // Below is a function for the main thread notifying the web worker
  // that the canvas has resized
  return proxy({
    resize: (...args: Parameters<BevyApp["resize"]>) => {
      app.resize(...args);
    }
  })
};

In essence our web worker will be used to drive animations and proxy main thread events to the bevy app.

The Bevy App

For our bevy app compiled to Wasm, we’ll reference bevy without winit.

[dependencies]
bevy = { version = "0.16", default-features = false, features = [
    "bevy_sprite",
    "bevy_window",
    "webgpu",
]}

This will allow us a pretty slim initialization and update loop.

#[wasm_bindgen]
pub struct BevyApp {
    app: App,
}

#[wasm_bindgen]
impl BevyApp {
    #[wasm_bindgen(constructor)]
    pub fn new(canvas: web_sys::OffscreenCanvas, canvas_size: CanvasSize) -> Self {
        let mut app = App::new();

        app.add_plugins(DefaultPlugins.set(bevy::window::WindowPlugin {
            primary_window: Some(Window {
                resolution: WindowResolution::new(canvas_size.width, canvas_size.height),
                ..Default::default()
            }),
            exit_condition: bevy::window::ExitCondition::DontExit,
            ..Default::default()
        }))
        .add_systems(PreStartup, setup_added_window)
        .add_systems(Startup, setup);

        app.insert_non_send_resource(canvas);

        BevyApp { app }
    }

    #[wasm_bindgen]
    pub fn update(&mut self) {
        if self.app.plugins_state() != PluginsState::Cleaned {
            if self.app.plugins_state() == PluginsState::Ready {
                self.app.finish();
                self.app.cleanup();
            }
        } else {
            self.app.update();
        }
    }
}

#[derive(Resource, Copy, Clone, Debug, Deserialize, Serialize, tsify_next::Tsify)]
#[tsify(into_wasm_abi, from_wasm_abi)]
pub struct CanvasSize {
    width: f32,
    height: f32,
}

A few things worth calling out:

• The BevyApp::update calling finish and cleanup is not related to app exit, but instead to initialization. I can’t tell you how long it took me to figure out that they are critical to a successful startup. Bevy does contain a custom loop example that would have been helpful if I knew what I was looking for.¹⁰

• The setup system is the same as Bevy 2D shapes setup. Copied below for convenience

const X_EXTENT: f32 = 900.;

fn setup(
    mut commands: Commands,
    mut meshes: ResMut<Assets<Mesh>>,
    mut materials: ResMut<Assets<ColorMaterial>>,
) {
    commands.spawn(Camera2d);

    let shapes = [
        meshes.add(Circle::new(50.0)),
        meshes.add(CircularSector::new(50.0, 1.0)),
        meshes.add(CircularSegment::new(50.0, 1.25)),
        meshes.add(Ellipse::new(25.0, 50.0)),
        meshes.add(Annulus::new(25.0, 50.0)),
        meshes.add(Capsule2d::new(25.0, 50.0)),
        meshes.add(Rhombus::new(75.0, 100.0)),
        meshes.add(Rectangle::new(50.0, 100.0)),
        meshes.add(RegularPolygon::new(50.0, 6)),
        meshes.add(Triangle2d::new(
            Vec2::Y * 50.0,
            Vec2::new(-50.0, -50.0),
            Vec2::new(50.0, -50.0),
        )),
    ];
    let num_shapes = shapes.len();
    for (i, shape) in shapes.into_iter().enumerate() {
        // Distribute colors evenly across the rainbow.
        let color = Color::hsl(360. * i as f32 / num_shapes as f32, 0.95, 0.7);

        commands.spawn((
            Mesh2d(shape),
            MeshMaterial2d(materials.add(color)),
            Transform::from_xyz(
                // Distribute shapes from -X_EXTENT/2 to +X_EXTENT/2.
                -X_EXTENT / 2. + i as f32 / (num_shapes - 1) as f32 * X_EXTENT,
                0.0,
                0.0,
            ),
        ));
    }
}

The setup_added_window needs a bit more of an explanation and has its own section.

The OffscreenCanvas Handle

This will be the most convoluted code, and as a graphics noob, I can’t say I fully understand and can explain it either. With that disclaimer out of the way, let’s look at the setup_added_window system.

fn setup_added_window(
    mut commands: Commands,
    canvas: NonSendMut<OffscreenCanvas>,
    mut new_windows: Query<Entity, Added<Window>>,
) {
    // This system should only be called once at startup and there should only
    // be one window that's been added.
    let Some(entity) = new_windows.iter_mut().next() else {
        return;
    };

    let handle = OffscreenWindowHandle::new(&canvas);

    let handle = RawHandleWrapper::new(&WindowWrapper::new(handle))
        .expect("to create offscreen raw handle wrapper. If this fails, multiple threads are trying to access the same canvas!");

    commands.entity(entity).insert(handle);
}

The crux of the system is that whenever a Window is added to our ECS, we add a component to the entity that can handle graphic calls. This is the bridge that allows Bevy’s renderer to actually render graphics to our OffscreenCanvas. Otherwise our app would run but have no visible output.

There’s a problem, one can’t directly attach an OffscreenCanvas as RawHandleWrapper requires the contents to be Send and Sync, but the handle for OffscreenCanvas is not thread safe!

The workaround is to introduce runtime thread safety, where one can theoretically “send” the value to other threads, but can only call methods on the thread that created the value. This might seem like a bizarre concept, so there are crates catering to this use case¹¹

Since we are dealing with a single threaded environment anyways, it’s more of a formality, so we can write the thread safety ourselves.

pub(crate) struct OffscreenWindowHandle {
    window_handle: raw_window_handle::RawWindowHandle,
    display_handle: raw_window_handle::DisplayHandle<'static>,
    thread_id: ThreadId,
}

impl OffscreenWindowHandle {
    pub(crate) fn new(canvas: &OffscreenCanvas) -> Self {
        // Equivalent to WebOffscreenCanvasWindowHandle::from_wasm_bindgen_0_2
        let ptr = NonNull::from(canvas).cast();
        let handle = raw_window_handle::WebOffscreenCanvasWindowHandle::new(ptr);
        let window_handle = raw_window_handle::RawWindowHandle::WebOffscreenCanvas(handle);
        let display_handle = raw_window_handle::DisplayHandle::web();

        Self {
            window_handle,
            display_handle,
            thread_id: std::thread::current().id(),
        }
    }
}

/// # Safety
///
/// At runtime we ensure that OffscreenWrapper is only accessed from the thread it was created on
unsafe impl Send for OffscreenWindowHandle {}

unsafe impl Sync for OffscreenWindowHandle {}

impl HasWindowHandle for OffscreenWindowHandle {
    fn window_handle(
        &self,
    ) -> Result<raw_window_handle::WindowHandle<'_>, raw_window_handle::HandleError> {
        if self.thread_id != std::thread::current().id() {
            // OffscreenWrapper can only be accessed from the thread it was
            // created on and considering web workers are only single threaded,
            // this error should never happen.
            return Err(raw_window_handle::HandleError::NotSupported);
        }

        Ok(unsafe { raw_window_handle::WindowHandle::borrow_raw(self.window_handle) })
    }
}

impl HasDisplayHandle for OffscreenWindowHandle {
    fn display_handle(
        &self,
    ) -> Result<raw_window_handle::DisplayHandle<'_>, raw_window_handle::HandleError> {
        Ok(self.display_handle)
    }
}

Main thread events

Our bevy app is up and running. It is displaying sprites and can drive animations, but the app is cut off from user input. So we must bridge the main thread world with the bevy worker. There are many events that one will need to bridge, but I’ll focus on canvas resizing to keep things short.

Below is how we handle this on the JS side (also including initialization code), by creating the worker and then using a ResizeObserver so the canvas matches the size of its container

// The canvas container
const container = document.getElementById("container")!;

const bounds = container.getBoundingClientRect();
canvas.width = bounds.width * window.devicePixelRatio;
canvas.height = bounds.height * window.devicePixelRatio;
canvas.style.width = `${bounds.width}px`;
canvas.style.height = `${bounds.height}px`;

const offscreenCanvas = canvas.transferControlToOffscreen();
const webWorker = new Worker(new URL("./worker.ts", import.meta.url), {
  type: "module",
});

let worker = Comlink.wrap(webWorker);

const initializeApp = async () => {
  const game = await worker.createGame(
    Comlink.transfer(offscreenCanvas, [offscreenCanvas]),
    {
      width: bounds.width * window.devicePixelRatio,
      height: bounds.height * window.devicePixelRatio,
    },
  );

  let resiveObserverAF = 0;
  const ro = new ResizeObserver((_entries) => {
    const bounds = container.getBoundingClientRect();
    canvas.style.width = `${container.clientWidth}px`;
    canvas.style.height = `${container.clientHeight}px`;

    cancelAnimationFrame(resiveObserverAF);
    resiveObserverAF = requestAnimationFrame(() => {
      game.resize({
        width: bounds.width * window.devicePixelRatio,
        height: bounds.height * window.devicePixelRatio,
      });
    });
  });
  ro.observe(container);
};

And on the rust side:

#[wasm_bindgen]
impl BevyApp {
    #[wasm_bindgen]
    pub fn resize(&mut self, size: CanvasSize) {
        let world = self.app.world_mut();

        // Update window resolutions
        for mut window in world.query::<&mut Window>().iter_mut(world) {
            window.resolution.set(size.width, size.height);
        }

        // Find all the cameras and update their projections if they're orthographic
        for (_camera, mut projection) in world.query::<(&Camera, &mut Projection)>().iter_mut(world)
        {
            if let Projection::Orthographic(ref mut ortho) = *projection {
                ortho.scaling_mode = ScalingMode::WindowSize;
            }
        }

        // TODO: do we send a WindowResized event here?
    }
}

Thanks to my bevy inexperience, the above isn’t perfect. You can see I’m unsure if a WindowResized event needs to be emitted, or what effect it would have. And I’m surprised about how I needed to the camera’s scaling mode or otherwise textures would stretch.

But it works! Of course, we aren’t done. Even in this example, I imagine that resizing the window could have an effect on the size, number, and positioning of the textures. I’ll leave that as an exercise to the reader.

For those questioning if there is a performance hit to proxying frequent events like mouse positioning from main thread events to a web worker, don’t worry there won’t be a hit.¹²

In the end, we’ve seen how to encapsulate Bevy and Wasm entirely to a web worker, but it’s unclear if this option is better than having winit on the main thread proxying events to the web worker.

And stepping back even further, there is a bit of a nagging thought that better solutions for the web are out there or upcoming when web is the predominant platform. Even if this hunch proves true, the core bevy logic can be taken as is, thanks to the modularity of bevy.