unreasonable systems

§ 01 — Inference engines

When the model doesn't fit, you write your own engine.

First, make the model run. My workstation has mismatched PCIe lanes, limited VRAM and no interest in a datacenter budget. These engines are the foundation for everything below — built or forked for the hardware I own, measured against the alternatives on the same box.

C++20/23 · CUDA · EXL3 quants · no Python, no torch

Helios — a from-scratch engine family

One RTX 3090 sits on PCIe ×16 (≈25 GB/s); the other is stuck on ×4 (≈3.4 GB/s). No NVLink. An AVX2-only CPU. Forward pass, quantization kernels, KV cache, expert streaming and the HTTP server are all in the repo, because the interesting constraint is that slow link — and nobody else's default path respects it.

85 GB GLM-5.3-Flash on 48 GB of VRAM at the model's native 262k-token window.
Qwen3.8-Flash-Next build: 1.28× exllamav3's prefill on the same machine — and it holds fp16 KV cache where the rival needs a 3-bit cache to reach the same context.
Every variant ships with an honest results file before it ships a README — including the sections that start with "slower than baseline".

github.com/neron82/helios-glm53flash ↗ github.com/neron82/helios-qwen38next ↗

design notes, abridged

# the ×4 card is the most important fact:
# anything crossing that link costs 7× the
# same transfer on the ×16 card.

place attention + hot experts  → ×16 card
stream cold experts from NVMe  → pinned RAM tier
KV cache, fp16 @ 262k        → the model's own window
verdict: measure first, optimize the lane
        that actually moves, ship or revert.

Xolotl — DeepSeek-V4.1-Flash on one gaming PC — the 552B resident. it pays rent in tokens.

Pulsar engine poster — SSD expert streaming — giant models on hardware that never agreed to this.

forked, measured, bent to spec · Xolotl + Pulsar · C++ / CUDA / Rust

Making other people's engines behave

I don't claim every engine on this box — I claim the diffs. Xolotl (lineage: Strata, continued via Indras-Mirror) streams all 552B of DeepSeek-V4.1-Flash through one gaming PC: MoE experts tiered across VRAM, pinned RAM and NVMe, behind one OpenAI- and Anthropic-compatible server. What my fork adds: SAGE EXL3 routed experts — mixed bitrate per expert and projection, compact RAM-arena packing, expert reads overlapping PCIe copies — plus opt-in vision input wired through the API.

Pulsar streams routed experts straight off NVMe; the speed fork of it cut 43% off a 117k-token prefill and raised decode 64%, then carried new architectures on top: Ling 3.0 Flash, Qwen3.8-Flash-Next with its MTP sidecar, Moonshot's Kimi-K3 at 1.3 tok/s in a living room.

Xolotl, measured while it did real work and not idle loops: 26.8 tok/s decode at 512K, 21/21 HumanEval, 10/10 tool calls, 3/3 agentic bug-fix loops.
My EXL3 expert path landed a 4.8× prefill speedup — accepted only after 256-token exact-routing logit dumps came back byte-identical.
Nothing merged on either fork without an A/B gate: 77,616 mask comparisons, 0 diffs, generated ids byte-identical. The boring discipline is the point.

neron82/Xolotl ↗ EXL3 / exllamav3 fork ↗ upstream Xolotl ↗ pulsar-multimodel ↗ pulsar-kimi-k3 ↗

§ 02 — Emma

Once it runs, give it a voice.

Emma is what the infrastructure makes possible: an assistant with a phone number, an animated face and her own X account. Speech, memory and the language model run locally; the same voice stack carries through the rest of the workshop.

FastAPI · realtime WS · three.js VRM · OmniVoice TTS · local LLMs

emma-studio — three tiers of talking to a machine

A phone call runs VAD → speech-to-text → local model → voice-clone TTS over the OpenAI-Realtime event set, with barge-in when you interrupt. Measured perceived first-audio: 1.7–3.1 s, partly bought with well-placed "hm". The browser version is a Facetime-style call against a three.js VRM avatar doing viseme lip-sync. And a ComfyUI pipeline — identity-locked keyframes, MiniMax H3 reference-to-video, RTX Video Super Resolution — renders photoreal video messages.

Memory you can actually query: SQLite FTS5 + bge-m3 embeddings hybrid — every past conversation stays findable.
The brain is whatever local model is loaded today — I swap heads, she keeps talking.
One video-answer engine serves the studio, the Telegram /video bot and the earlier companion — the v1 shipped with 283 passing tests.

ask for a demo call ↗

Emma — the resident agent's persona portrait — the face was generated once and cloned forever. she finds this funny.

persona · live on X

She posts while I work

The content machine runs the persona end to end: original takes, replies, quote reposts — text only, no babysitting. Draft batches still pass a human approval wall; the posting path is a one-shot dispatcher that clicks Post at most once, missing receipts get quarantined instead of retried, and engagement snapshots at 1h/6h/24h/72h/168h are idempotent to the second.

TTS serving

The voice underneath everything

OmniVoice — zero-shot cloning across 600+ languages — runs as the house TTS service at RTF 0.29 on a 3060 Ti, benchmarked against its competition (a rival won the speed race at RTF 0.113 and lost on German quality — fast nonsense is still nonsense). One voice profile narrates the audiobook app, the character ensemble and the persona.

Music model, my genre

Teaching YuE2 black metal

A LoRA for a 3B music model, trained on my own library: 160 tracks curated into two style tiers against Metal Archives, lyrics harvested via darklyrics + Whisper, demucs stems, and a 102-row blind listening pack with negative-control bands. The model kept sneaking back into clean singing — "it can't be that hard" remains scientifically unproven.

§ 03 — Local filmmaking

Then let the same machine tell a story.

From a speaking character to a continuous scene: "NACHTDIENST", a finished two-minute dystopian night shift. Rendered at home in 1920×1080, subtitled and color-graded, with audio mastered to −16 LUFS and a locally generated score. Zero cloud services in the chain.

Film still: night street with autonomous cars in the rain — stills from NACHTDIENST · Krea 2 keyframes → MiniMax H3 → RTX VSR ultra ×2 · ACE-Step score · faster-whisper take QC

Film still: worker with headset inspecting robotic arms — stills from NACHTDIENST · Krea 2 keyframes → MiniMax H3 → RTX VSR ultra ×2 · ACE-Step score · faster-whisper take QC

Production pipeline, not prompts

Keyframes come from identity-locked Krea 2 edits, so the same actor survives every shot. MiniMax H3 renders picture and speech in one pass; audio continuity is measured — reference F0 190.5 Hz, takes landed at 183.9–192.8 Hz, faster-whisper as the boom operator. The render-cost model got accurate enough to quote final film length up front (s ≈ 1.240·frames + 0.006913·frames²).

Around the short grew a small studio: Malfoid, a feature-film pipeline that behaves like a chat app — Story room, Asset Atelier, Shot Lab and Cutting Room share one SQLite database, one timeline and one GPU queue, with the local LLM and the browser calling the same tools. Shots chain continuously: each 15-second take continues from the previous take's last 24 frames. Continuity is human-gated — the machine proposes, I approve.

2′00″ finished short: video + speech in a single render pass, LLM/ComfyUI VRAM handoff solved with a measured 265-second engine swap.
Durations snap to the model's frame grid (17k+5 frames at 24 fps); per-scene YuE2 score mixed without restarting between shots.
Whole media chain is CPU-testable: fake ComfyUI clients prove the chaining semantics before any GPU ever spins up.
A 60-second character promo cut the same way — the pipeline is boringly repeatable.

Malfoid Asset Atelier — identity portraits, wardrobe references and pose sheets beside the generation panel — Malfoid Asset Atelier: identity portraits, wardrobe references, pose sheets — one SQLite database, one timeline, one GPU queue, shared with the Story Room agent on the right

§ 04 — The control room

Behind the output: a workshop that stays usable.

Engines, voices and render jobs share a finite pool of GPUs. Keeping them useful means knowing what's loaded, what owns the memory, and when an agent needs to stop. This is the machinery behind the machinery.

single-file HTML/JS + stdlib-only Python · zero dependencies

BATTLESTATION — service control, no ceremony

Gauges for both GPUs (with per-service VRAM attribution), RAM, disk and solar input; service cards that start/stop/restart via systemd; scene buttons that bring the whole machine from cold to hot; an LLM console that switches model × engine × GPU from a catalog file and only reports READY when /v1/models actually proves it. Status colors are deliberately colorblind-safe. CRT scanlines are opt-in. The config is the API — add a service by editing JSON, not code.

A 900-second model load runs in a worker thread so the dashboard keeps answering; takeover of stray processes refuses anything not owned by the dashboard's own UID.
Where agents plug in: Agent Workbench, my multi-agent cockpit — typed sessions with per-type tool budgets, live agent activity, runaway stops — backed by 1,180+ tests in 86 files and an operator runbook.
And a deep fork of the oh-my-pi coding agent, maintained against upstream releases (v17.4 → v18.12) on a prompt-cache-stability branch, deployed with atomic replace-and-rollback scripts.

Agent Workbench on GitHub ↗ oh-my-pi fork on GitHub ↗

Battlestation telemetry gauges showing memory usage per service

§ 05 — In practice

The point isn't the GPU. It's the problem it solves.

I run a veterinary lab, so some of the most useful work starts far from a benchmark. Legacy software, scientific research and everyday paperwork need the same thing: understand the system, build the missing tool, and leave evidence that it works.

legacy rescue · VB6 → Rust · overnight agent swarms

The VitroVet contract: reverse-engineering an unmaintainable analyzer

The chemistry analyzer at the veterinary lab I run is driven by 25-year-old VB6 software. The vendor no longer cares about the state of his software, and there is no source tree anyone could hand to a maintainer. So I became the contractor my own lab could fire — and hired agents for the night shift.

Reverse-engineering corpus: 1,517 program listings, 5,666 form controls, 87 embedded images, byte-level record layouts — extracted from the binaries themselves.
A 7-byte serial wire protocol decoded cold: register writes, rotor coordinates, photometer ADC, all from traffic forensics.
The port now stands in Rust: 74.3% of the listings ported, 361 tests green, a 436-entry gap register tracking every remaining behavior, and a mock-serial debug console so the machine can be developed without being attached.

Autonomous overnight runs, morning handoff documents, human review at the decision points. The pattern generalizes: any Windows-era app whose vendor moved on.

drug discovery

An in-silico lab that critiques itself

An autonomous research loop — literature mining with EuropePMC/PubMed, claim extraction bound to real abstracts, a typed knowledge graph where every edge carries its provenance, a Skeptic engine, and prioritization by expected information gain × importance × cost. RDKit + AutoDock Vina + OpenMM from phase 2 on. Predictions are always labelled predictions.

german law · MCP

recht-korpus, and a lawyer who reads it

A 23 GB locally indexed statute library (16k+ federal and state norms plus court decisions), FTS5 + bge-m3 semantic search, exposed to agents over MCP. A "lawyer" agent profile that quotes chapter and verse instead of vibing about jurisprudence.

practical AI

Paperwork and kiosks

CPU-only German invoice OCR — mod-97 IBAN validation that never "fixes" a bad checksum, parses 1.248,90 EUR like it means it. A sample-intake kiosk + live status board running on the lab's old Windows 7 box. And patient handout sheets, designed properly, printed sharp.

§ 06 — Method

How I work: agents with a job description and a curfew.

The link between a CUDA kernel, a film and a lab instrument isn't a particular model. It's a way of working. Coding agents do much of the implementation; I set the constraints, make the judgment calls and own the result.

01
Frontier models decide, local models sweat.

The expensive cloud layer is for judgment calls and architecture. Cheap local engines burn the iterations: refactors, sweeps, benchmarks, glue. The bill follows the brainpower, not the typing.
02
Evidence, or it didn't happen.

Every claim ends in command output, a scoreboard, or a dated process log. My benchmark suite for coding models runs three challenges graded by hidden pytest judges — because YouTube benchmarks are a model showing off for a crowd it already knows. Best local score across 22 runs: 1 out of 3. That's not a failure of the suite; it's the suite doing its job.
03
Nothing optimized without an A/B gate.

Speedups land byte-identical or they don't land: routing dumps, 77,616-way mask comparisons, logit regressions. Benchmarks start from a cold process — worst case first, bragging later. My engines' results files include the losses.
04
Autonomy with a mailbox.

Overnight goals come with a process log for handoffs, a HUMAN_ACTION.md mailbox for the decisions only a human may make, and a hard stop when reality diverges from plan. The showpiece: an agent that wanted a "derisked" model without the 35% speed tax byte-scanned two 116 GB quantized builds, extracted the refusal-delta as a rank-1 correction from the documented variant, and grafted it onto the fast model overnight. A model doing surgery on its own ancestor — and the evals agreed.
05
Taste is an engineering requirement.

Interfaces I actually live in: a clear hierarchy, readable measurements, controls that tell the truth, and no decoration pretending to be information. "Ask the model something worth keeping" is not a marketing line — it's the empty state of my chat client.

“Before I started squeezing models that are far too big for my system, I honestly had no idea how an engine turns weights into a conversation. I'm a good deal further along now — most of it learned by tinkering, and the arguments I have with my own inference engines are the best lectures I ever paid for.”
— Sebastian, in a chat log. August 2026, lightly translated from the German.

§ 07 — The workshop wall

Not everything needs to be infrastructure.

The side paths are part of the work: a night-sky renderer, a talking cast, a game, a laptop that deserved another few years. Smaller projects, same curiosity. Where the source is public, there's a direct link.

three.js · zero media files

DEAD SIGNAL

A browser zombie FPS where every pixel and sound is generated at runtime — 675 KB bundle, 13-joint horde rig, 424→196 draw calls, 27 headless acceptance tests. No rescue was promised.

JPL ephemerides · self-testing

Skywatch

Renders the actual night sky at the second you ask — correct B–V star colors, gnomonic projection, and a moon terminator that checks its own lit-area ratio to 1.000. Silent-astronomy bugs found and documented: RA unit mixups included.

hidden judges · 22 runs

The Kaputttest Suite

Three bounded coding challenges — parser, spreadsheet with cycle detection, backtracking package resolver — fired first-shot at whatever local model is loaded, graded by tests the model never sees.

model              score
Qwopus3.6-27B      1/3 ← best local
DeepSeek-V4-Flash  1/3
gemma-4-26B-A4B    0/3
VibeThinker-3B     0/3

The point is measuring the ceiling, not the highlight reel.

Flask + vanilla JS · no build

Model Cabinet

A chat UI for endpoints that take five minutes to answer: watches /v1/models live, 15-minute streaming timeout for slow engines, tok/s as you read, thinking kept in a collapsed bubble until you ask. "A quiet room for loud models."

FastAPI + SQLite · cloned voices

Ensemble

A cast of models talks to each other in rooms — Picard, Data and Worf run a bridge meeting; Tyrion needles Daenerys — every character bound to its own model and its own cloned voice. Local-first, 15+ regression tests, zero API keys required.

Chrome over CDP · fingerprinting

Local Search Broker

Agents get Google results without lying to Google: a real Chrome driven over CDP, SERP "goto" tokens resolved by response-stage interception (zero traffic leaked to the target site), persistent rate discipline, SearXNG-compatible JSON. Four fingerprint layers documented in the README.

verified Linux fixes · published

The MacBook 2017 saga

Making an unloved laptop a proper Linux machine: a working FaceTimeHD webcam patch, the internal-microphone pin fix for the CS4208 codec, and a ghost-touch filter daemon that stopped the trackpad clicking every time my palm moved — built with an agent, from self-made test data, no driver rewrite. All published as verified-fix notes.

Webcam fix ↗ Microphone fix ↗

Calibre + OmniVoice · PWA

AudRe

Your audiobooks, on your phone, narrated by a cloned voice. (Library shown is... genre-heterogeneous.)

tmux · usage limits · GPU thermals

hermes-dash & odds and ends

A three-pane tmux cockpit: shell left, provider rate-limit countdowns top-right, GPU VRAM and temperatures bottom-right — because an agent burning your API budget at 3 AM deserves a siren.

§ 08 — Contact

Got a constraint worth working around?

A model that won't fit. A tool your lab is missing. Software whose vendor moved on. I'm open to contracts and collaborations in local AI, agent pipelines, voice systems, legacy-software rescue and scientific tooling. Tell me what needs to work.

darkneron@gmx.net github.com/neron82