How QVAC actually works under the hood

You call loadModel(). A model ends up in memory. You call completion(). Tokens stream back.

What actually happened in between is more interesting than the API suggests, and understanding it explains most of the behavior that surprises people when they first ship a QVAC app — why the first call is slow, why close() exists, and why memory behaves the way it does on mobile.

Here's the architecture.

The core insight: the SDK isn't where inference happens

QVAC's SDK runs on three JavaScript runtimes: Node.js, Bare, and Expo. But its underlying components — the native addons that actually do inference — run only on Bare.

Bare is Holepunch's minimal JavaScript runtime, designed for embedding and cross-platform native modules. It's the substrate the whole QVAC stack is built on.

So when you run the SDK on Node.js or Expo, it spawns a Bare worker — a separate process where all AI operations take place. Your app talks to that worker over RPC.

When you run the SDK on Bare, no worker is spawned. Requests are handled in-process.

This single design decision explains almost everything else.

Phase 1: initialization (lazy, and once)

The worker isn't spawned when you import the SDK. It's started lazily, on the first RPC call.

The first time you call loadModel() — or any function other than close() — the SDK runs a full initialization sequence:

Initializes a runtime-specific RPC client.
Sends configuration to the worker via an internal __init_config message.
Spawns the worker process.

That worker is then reused for every subsequent call until you explicitly close it.

Practical consequence: your first call pays for process spawn plus initialization. Every call after it doesn't. If you're benchmarking, or if your UI needs to feel instant, this is the cost you want to move off the critical path — warm the SDK during a splash screen or idle moment rather than when the user taps a button.

Phase 2: model loading (singleton client, many models)

Here's the part people get wrong: there is one RPC client and one Bare worker per application, not per model. It's a singleton.

When you call loadModel():

The model is downloaded (if needed) and cached.
It's loaded from disk into memory.
It's registered with a unique model ID.

From that point it stays available for inference until you unload it.

You can call loadModel() multiple times to hold several models simultaneously — an LLM and an embeddings model and a transcription model, all live at once — and they all share the single worker.

loadModel() accepts models from three sources:

a local filesystem path
an HTTP URL
QVAC's distributed model registry

The registry path is the ergonomic one. The SDK exposes constants for preconfigured models (for example LLAMA_3_2_1B_INST_Q4_0), each mapping to a model already published to the registry, so you hand loadModel() a constant instead of managing files.

Phase 3: inference

With one or more models loaded, you call the capability functions — completion(), embed(), and so on — passing the relevant modelId. Requests go over RPC to the worker, the native addon runs the inference, and results come back (streaming, for token generation).

Multiple models, multiple concurrent inferences, one worker.

Phase 4: shutdown

Two levels of teardown, and the distinction matters:

unloadModel() releases a single model's memory. On a phone, this is not optional housekeeping — it's the difference between an app that survives and one the OS kills. Load what you need, unload what you don't.

close() explicitly shuts down the worker and releases the RPC connection. On Node and Expo, this terminates the worker process. On Bare, it's a no-op, since there's no separate process.

After close(), the next SDK call reinitializes the client and spawns a fresh worker — you pay the Phase 1 cost again.

There's a useful subtlety in the docs: unloadModel() will automatically close the RPC connection when there are no active models or providers left. But close() is the explicit, intentional way to shut down. Relying on the implicit path is the kind of thing that works in development and surprises you in production.

Why this design

Three things fall out of "native addons run on Bare, so spawn a Bare worker":

Cross-platform consistency. The native inference code is written once against Bare. Node, Expo, and Bare hosts all get identical behavior because they're all talking to the same worker implementation. This is why "write once, run on iOS, Android, macOS, Windows, and Linux" is actually true rather than aspirational.

Process isolation. Native inference code is memory-hungry and occasionally crashy. Running it out-of-process means a native-side failure doesn't necessarily take your whole app with it, and memory can be reclaimed decisively by terminating the worker.

A clean RPC boundary. Everything crossing between your app and the inference engine goes through a defined message interface. That's what makes the same SDK surface work identically whether inference happens in-process (Bare), out-of-process (Node/Expo), or — the elegant part — on an entirely different device via delegated inference. Once you've built an RPC boundary, "the worker is on your desktop instead of in your app" is a change of address, not a change of architecture.

That last point is worth sitting with. The Bare-worker design isn't just an implementation detail. It's what makes delegated inference possible without rewriting anything.

What this means for your app

Warm the SDK early. The first call spawns a process. Do it during a moment the user expects to wait.

Treat model lifecycle as a first-class concern. Especially on mobile. Loading is expensive, memory is finite, and the OS is unsentimental. Load, use, unload.

Use runtime lifecycle hooks. QVAC exposes suspend/resume for the runtime — pair these with your app's background/foreground events rather than letting a worker idle in the background.

Cancel in-flight work. You can cancel by requestId, or broad-cancel by modelId during unload or shutdown. A user who navigates away shouldn't be paying for a generation they'll never see.

Don't fight the singleton. One worker, many models. Design around it.

The honest note

QVAC is pre-1.0, with 195 releases at the time of writing. The architecture above is drawn from the current official documentation, but internals are exactly the kind of thing that changes between versions. Verify against docs.qvac.tether.io, and read the release notes before upgrading.

Part of our QVAC series. See also QVAC SDK Explained, The Complete Guide to QVAC's Capabilities, and Memory and Model Lifecycle on Mobile.