Ollaya is an open-source runtime that lets developers run decision models — lightweight classifiers that answer structured yes/no, choice, and scoring questions — locally on their own hardware. The pitch: sub-10ms latency on a GPU, no data leaving your machine, and full API compatibility with TypeSafe's hosted service. It ships as a desktop app, CLI, and Docker image across macOS, Windows, and Linux. The architecture is notable for what it avoids. Decision models answer in a single forward pass rather than token-by-token generation, which is why a five-question request through the HTTP API clocks in at roughly 10ms on an RTX 4090 for the smallest encoder models. The largest decoder models (Decider at 2B parameters) still complete in under 200ms. These are classification-grade tasks — intent routing, safety filtering, risk scoring — not general-purpose chat. The model library draws from multiple contributors: Convai Innovations' Laya (322M-421M params, 100+ languages), Mapika's Decider on Qwen3.5 (0.75B-1.9B, described as the most accurate open decision model shipped), Moritz Laurer's NLI zero-shot classifiers, Knowledgator's GLiClass, and others including Qwen3Guard for safety and Kev from Jared Palmer. Models are open-weight and pulled on demand, following the Ollama playbook that developers already know. The TypeSafe API compatibility is the strategic play here. By implementing /v1/systemone and /v1/models with identical request and response shapes, Ollaya positions itself as a local drop-in replacement. The official TypeSafe Python SDK 0.7.1 works unchanged — you just redirect the base URL. This means any team currently paying for hosted decision-model inference can run the same code against local hardware with zero migration cost. The privacy argument is real and specific. Ticket content, customer emails, and user messages are exactly the kind of data organizations are most uncomfortable sending to third-party APIs. Running classification locally eliminates that exposure entirely. For regulated industries or teams with strict data residency requirements, this removes a genuine friction point. Platform coverage is thorough: native GPU acceleration via CUDA 13 on Linux and WSL 2 (NVIDIA only, driver R580+), CPU fallback everywhere else including Apple Silicon. No AMD or Intel GPU acceleration yet. The Docker image covers both amd64 and arm64. GGUF LLM-based decision models via llama.cpp are listed as planned but not yet available. The competitive dynamics are straightforward. Ollaya compresses the margin between TypeSafe's hosted API (236-276ms per the cited third-party benchmarks) and local inference (8-190ms depending on model), while eliminating per-request pricing entirely. Whether that margin matters depends on volume: for a team making thousands of classification calls per minute, the economics shift fast. For occasional use, the hosted API is simpler. The Apache-2.0 license and GitHub availability keep the exit costs at zero.
