Delegated Inference: How Your Phone Borrows Your Laptop's GPU

On-device AI has one obvious problem: devices are not equally capable. Your desktop has a GPU. Your phone has a thermal budget and a battery. The model that runs beautifully on one crawls on the other.

The industry's answer to this has always been the cloud. Can't run it locally? Send it to a data center.

QVAC has a third answer, and it's the most interesting feature in the SDK that nobody is talking about: delegate the inference to a peer. Your phone hands the heavy work to your laptop, over an encrypted peer-to-peer connection, and gets the answer back. No cloud. No server. Nothing leaves your own network.

Here's how it works.

The idea

Delegated inference means one device performs AI inference on behalf of another. In QVAC, this runs over the Holepunch stack — the same peer-to-peer infrastructure family behind Keet and Hyperdrive.

The practical shape:

• Your phone wants to run a large model it can't comfortably host.
• Your desktop, on the same network (or reachable as a peer), has the model loaded and the compute to run it.
• The phone delegates the inference request to the desktop over an encrypted P2P link.
• The desktop runs it and streams the result back.

To your app's code, this looks almost the same as local inference. To your user, it looks like their phone got dramatically more capable.

Tether's own QVAC Workbench app ships this: your mobile Workbench can offload heavy computation to your desktop Workbench, with everything staying private and local.

Why this is different from cloud offloading

It is tempting to say "so it's just a server, but at home." That undersells the distinction in three ways.

It's your hardware. The peer is a device you own, on a network you control. There's no third party who now holds your prompt, your document, or your logs. The data leaves the phone but never leaves you.

It's encrypted, direct, and peer-to-peer. Not a request to a company's endpoint. A direct connection between two of your devices.

It's the same code. You don't write a client and a server. You don't deploy anything. You don't maintain infrastructure. The QVAC SDK does local inference and delegated inference through the same interface, so the decision of where something runs becomes a runtime choice rather than an architectural rewrite.

That last point is the one developers should sit with. Delegated inference turns "on-device vs. server" from a fork in the road into a dial you can turn.

Blind relays: getting through the network

Peer-to-peer connections have a famous enemy: NATs and firewalls. Two devices behind different routers often cannot see each other directly.

QVAC handles this with blind relays — relay nodes that route traffic between peers who can't connect directly, letting them establish a link across network boundaries.

The word "blind" is doing important work. A relay forwards traffic; it isn't a party to the conversation. This is the same philosophy as the rest of the stack: infrastructure that moves your data without being able to read it, rather than a service that processes your data and promises to be good about it.

This is what makes delegated inference practical outside a single LAN. Without relays, "delegate to your desktop" works at home and fails on hotel WiFi.

Where this actually matters

Mobile apps that punch above their hardware. A phone app can offer a large-model experience when a desktop peer is available, and gracefully fall back to a small local model when it isn't. Same code, same privacy guarantee, better ceiling.

Robotics. A small robot with modest onboard compute delegates a heavy planning step to a workstation on the local network — keeping latency far below a cloud round trip and keeping sensor data inside the building. (See QVAC for Robotics.)

Agent frameworks. This is the sharpest one. Local agent frameworks consistently report the same weakness: small local models handle execution fine, but the reflection and planning steps — where a model reasons about what it just did — are shallow. Delegated inference offers an answer nobody else has: run the fast execution loop on-device, delegate the heavy reasoning step to a peer. Local throughout, capable where it counts.

Households and teams. One capable machine serves the phones, tablets, and laptops around it. The family GPU. The office workstation. Compute as a shared local resource, not a rented remote one.

The honest caveats

A peer must exist and be reachable. If your desktop is off, delegation isn't available — so apps need a sensible fallback path, which is design work you have to do.

There is a trust boundary, just a much better one. Data moves from device A to device B. If both are yours, that's fine. If you delegate to a peer you don't control, you've made a different trade — one that's still better than a cloud provider, but not the same as pure on-device.

And this is pre-1.0 software. Verify behavior against the current docs before you build a product around it.

Why it matters

The framing that makes delegated inference click is this: the cloud's real product was never intelligence. It was borrowed compute.

Delegated inference says you can borrow compute without renting it, and without surrendering your data as the price. The GPU three feet away on your desk is, for most purposes, a better data center than one three states away — lower latency, zero marginal cost, and it belongs to you.

Tether's vision doc puts it as "we are constrained only by the number of devices on Earth." Delegated inference is what turns that from a slogan into an architecture.

---

Part of our QVAC series. See also The Complete Guide to QVAC's AI Capabilities and How QVAC Works Under the Hood. Verify implementation details in the official docs.