Local language models, reachable from anywhere
Language models running on my own graphics card at home, reachable from outside it. Most of the work was in what connects the parts: getting the GPU through to the models, the containers through to each other, and nobody through from outside without an identity check.
Two containers, Ollama and Open WebUI, on WSL2 running Ubuntu 24.04, with the NVIDIA container toolkit for GPU passthrough and both containers on a user-defined Docker network so they resolve each other by name. Phi-3 Mini and a 9B Qwen, chosen because they are what this hardware can actually hold rather than what a benchmark recommends.
The first decision is the one worth writing down. It started in a virtual machine, because a VM is the cleaner boundary and isolation was the instinct. GPU access across that boundary cost more than the isolation was worth for this workload, so it moved to WSL2 deliberately: a real security property traded away for hardware access, with the trade understood rather than stumbled into.
The second is smaller and matters more. The install commands circulating for this stack differ from the vendor's own documentation in several places. I stopped and worked out why before running them. That is the same habit as reading the response headers after a deploy instead of trusting that the deploy did what it said.
Access from outside runs over a Cloudflare tunnel with an identity check at the edge, so the machine has no inbound exposure and the interface is not reachable by anyone who has not passed that check. Behind that check, Open WebUI still asks for its own username and password.
What I would do differently: one Compose file with its own network from day one, instead of growing it a docker run at a time. And read what the runtime reports before trusting a model: the 9B Qwen sat on the CPU for minutes with the GPU idle, and ollama ps said so in one line.