Reproducible evidence · v0.16.0

Heliox OS benchmarks and evidence.

The current software snapshot was captured on : 25.234 ms median and 26.930 ms p95 across 100 guarded local CPU-status requests, with zero model calls. These are local software measurements, not full-workflow or human-performance results.

Heliox OS is the desktop automation agent at helioxos.dev. Its v0.16.0 installer release was published on 26 October 2026, and its release notes cite this benchmark snapshot. Release publication and benchmark capture remain separate records. See the product identity and current answers.

162 declared actions21 specialists162/162 provider coverage22 independent post-condition verifiers
Generated from runtime authorities

Capability coverage

Current Heliox source declares 162 action types and registers concrete providers for all 162 through 21 executable specialists. Availability still depends on operating system, permissions, credentials, hardware, and third-party services.

Important limitation

Verification depth

Provider coverage is not the same as independent outcome verification. Twenty-two actions currently have a separate observed post-condition verifier; 140 rely on the executor result. Heliox must not describe all 162 actions as independently verified.

Test certificate validated

Windows signing

The SignPath test-policy workflow has signed and verified the Windows EXE, MSI, and embedded application. The production certificate remains pending, so current public installers must not yet be described as production-signed.

Measured guarded path · 100 iterations

Latency distributions, not one headline number

The ready-daemon CPU-usage path measured 25.234 ms median, 26.930 ms p95, and 27.992 ms p99 on Windows/Python 3.13.13 with zero model calls. The path includes local planning, routing, risk assessment, real execution, post-condition verification, and response shaping.

59/59 regression cases

Intent routing controls

The deterministic dispatch corpus covers URLs, browser clicks, screen analysis, status queries, app launches, file reads, forensics, a browser workflow, and ambiguous phrases that must reach model planning. Median routing latency was 0.026 ms. This fixed corpus is not population-level language accuracy.

Shared-loop protection

Concurrent responsiveness

A real one-second CPU monitor allowed 65 concurrent scheduler heartbeats; the maximum gap was 16.618 ms on Windows. This protects voice, camera, approval, and WebSocket work from a one-second event-loop freeze. It is not UI or hardware-input latency.

Process-isolated local speech

Heavy voice memory is released

A real Kokoro file-synthesis run measured 21.401 seconds cold and 0.138 seconds warm. The parent retained zero Torch/CUDA modules, and the speech worker exited after its 10-second idle window. This does not measure audible quality, speaker compatibility, or universal latency.

Subscription-backed planning · planning only

3/3 fixed planning cases

One developer-machine Codex CLI run passed three fixed health, browser, and evidence-first planning prompts at 14.708 seconds median. No proposed action was executed and every case contained zero destructive actions. This does not establish Claude behavior, universal provider availability, or runtime outcomes.

Calibrated learned-risk evidence

World model: useful, bounded, inspectable

The shipped coarse risk predictor records 25,200 training and 5,400 temporal-validation samples across 12 action types. Held-out MAE improved 54.0154% for disk delta and 99.4124% for process delta versus a zero predictor; 5/5 direction invariants passed. Deterministic policy remains authoritative, and this is not a general physical-world or user-intent model.

Measured control path · v0.16.0 · 1,820 replies

Built-in commands answered in code: 91/91

Each of the 91 command-guide examples, the list behind Ctrl+K, was sent through the daemon's control path 20 times after two warm-up runs. All 91 were handled in code without planning, at 1.384 ms median and 14.111 ms p95 per reply across 1,820 replies, with zero model calls and zero actions executed. Stand-ins replace the clipboard, speech, the settings file and machine readings, so this measures the control path only, not speech recognition, text-to-speech or end-to-end voice latency.

The suite covers the v0.16.0 to-do list, watchers, repeating and app reminders, good morning and good night, habits, quick answers, file and health questions, clipboard history and snippets. Watchers, the health check, the file finder and focus heads-ups only read, and a generated password goes to the clipboard only. The daemon now registers 301 WebSocket RPC methods.

Software path tested · since v0.15.0

Undo, receipts and built-in commands

Every example in the command guide is run through the real handlers by tests/test_command_guide.py. An exported task receipt can be checked anywhere with python -m pilot.workflows.task_receipts verify; receipts describe what happened and never grant, approve or run anything. Undo restores only what Heliox recorded, re-checks each item right before restoring it and leaves later changes alone. Effects that leave the computer and system changes such as shell commands or package installs are never undoable. Undo and receipts start with tasks run after the upgrade, and the activity timeline, daily brief, usage nudges and phone requests stay off until turned on.

Software path tested

Voice, gesture and gaze

Routing, cancellation, calibration, geometry, temporal filtering, and fusion have automated coverage. Accuracy across microphones, cameras, lighting, backgrounds, languages, accents, and users still requires human hardware testing.

Research boundary

Neural intent

Synthetic BrainFlow and recorded EEG playback paths are implemented. Heliox has not established live headset control accuracy, medical validity, or clinical use. It must be described as recorded/synthetic EEG research until live evidence exists.

Known limits remain visible

What this evidence does not prove

  1. Universal compatibility with websites, applications, operating systems, peripherals, or third-party APIs.
  2. Physical camera, microphone, speaker, accessibility-permission, EEG, or human-factors accuracy.
  3. That snapshots or per-task undo can reverse messages, purchases, remote operations, pushed commits, or every external effect.
  4. That learned-risk or world-model output can override deterministic safety policy.
  5. That the currently published Windows installers carry a production-trusted certificate.
Simulated users only · tuning families, in-sample

Assistance co-adaptation evidence

The quick suite evaluates two simulated families, five simulated people per family and two sessions, against the plain interface, uniform random assistance, and the adaptive engine. All three policy agency audits passed. The adaptive engine does not beat the plain interface for every family. These families share the model's functional forms, so these results do not establish held-out generalization or an improvement for real people.

The consented study is off by default. Assistance changes presentation only and cannot approve or execute actions. No model was promoted by this benchmark.

Inspect the complete evidence record

The Markdown version includes the platform matrix, closed regression history, and reproduction commands.

Read full evidence