AI Butler

Your AI assistant that works wherever you chat. Exceptional memory. Any model. Self-hosted. Apache 2.0. Built in the open.

Get Started View on GitHub Read the FAQ

What is AI Butler?

AI Butler is a self-hosted personal AI agent that works across every channel you chat on, remembers everything you tell it, connects to any AI model (Claude, GPT, Gemini, or fully-local Ollama), and is built with a security-first architecture. One Go binary. Zero dependencies. Runs from a Raspberry Pi to the cloud.

The core is production-ready today — memory, web chat, scheduler, agent loop, MCP integration, cost tracking. Several advanced features are in beta and clearly labeled below. We’re building this in the open, and we’d love your help finishing it.

💬 12 Channels, One Agent

Web chat + terminal [ready] · Telegram, WhatsApp, Slack, Discord, Teams, Google Chat, LINE, IRC, custom webhook, Nostr [beta]. Same memory, same assistant, everywhere.

🧠 Memory That Actually Works

Knowledge graph with typed edges + FTS5 full-text search + vector embeddings + true reciprocal-rank fusion with recency decay. Local, yours, never LLM-summarized — and now correctable: fix or permanently delete any remembered fact from the Memories panel.

🤖 Any AI Model

Claude, Ollama, Ollama Cloud [ready] · GPT, Gemini, Grok, LM Studio, vLLM, Groq, DeepSeek [beta]. Bring your own key, swap per task, or run fully offline.

🔗 Agent Ecosystem Hub

Google A2A v2 [beta — conformance suite pending]. Built-in MCP client [ready] + MCP server [beta]. Subprocess bridges for any CLI tool. Dynamic capability-based agent discovery.

🛡️ Security First

59 internal security audit passes. Capability-gated tools. RBAC + OIDC + WebAuthn + TOTP. WASM plugin sandbox. SSRF protection. Shell allowlisting.

🖥️ Clean Built-in Web UI

7-panel sidebar dashboard. Chat · Home · Memories · Connected apps · Missions · Spending · Settings. No framework, no build step, embedded in the binary.

📅 Schedule by talking

“Every weekday at 8am, brief me on the day.” Natural language → cron, persisted in SQLite, delivered to any channel. One command sets up a whole routine.

💾 Self-hosted & Private

Single Go binary. Your data stays on your machine. Zero telemetry unless you opt in. Runs offline. Apache 2.0 licensed with explicit patent grant.

What’s in this release

We’re shipping honest labels instead of marketing. Here’s exactly what’s production-ready, what’s beta (works but needs community testing), and what’s on the roadmap.

Production-ready

Agent loop with streaming, tool calls, and capability gating
Memory — FTS5 + knowledge graph + vector embeddings + hybrid search
Anthropic Claude and Ollama (local models + embeddings, auto-detected)
Web chat — 7-panel sidebar, streaming, file/voice upload, responsive, dark mode
Terminal REPL — interactive, streaming, slash commands, session resume
Scheduler — natural-language cron, persistent, reliable
Cost tracking — per-model, per-session, live dashboard
MCP client — connects to external MCP servers on boot
Capability engine — per-tool permissions + full audit trail
Vault — credential storage with OS keyring integration
File, shell, and git tools — capability-gated, sandboxed
SQLite database — 74 tables, 22 migrations, WAL mode
Three-tier config system — Settings / Configurations / Options
Per-project BUTLER.md instruction files
Single-binary distribution — zero CGO, cross-compiles to any Go platform

Beta — code complete, needs real-world validation

Help us push these from beta → ready by testing with your own credentials and filing a report.

10 messaging channels (Telegram, Slack, Discord, WhatsApp, Teams, Google Chat, LINE, IRC, webhook, Nostr) — help us test
OpenAI GPT + Azure OpenAI, Google Gemini, xAI Grok — model adapters
LM Studio, vLLM, Groq, DeepSeek — OpenAI-compatible providers
Multi-agent swarm orchestration — engine runs (with per-profile memory isolation), cookbook on the roadmap
Full A2A v2 protocol compliance — handler works, formal conformance testing pending
MCP server mode — expose AI Butler’s tools to other MCP-compatible clients
WASM plugin sandbox (Extism) — runtime ready, sample plugins on the roadmap
OIDC SSO (Auth0, Okta, Keycloak, Authelia, ZITADEL) — implementation complete, needs provider testing
FIDO2 / WebAuthn — hardware security key support
TOTP 2FA + RBAC (admin/user/viewer/agent roles)
LAN mode with mDNS device discovery and PIN pairing
Subprocess bridges (ffmpeg, imagemagick, any CLI tool)
Smart home via Home Assistant — tool surface + PIN safety gating ready, HA adapter on the roadmap
Whisper STT and Piper TTS — local and cloud options
Fact quality & correction — remembered facts carry provenance and confidence; contradictions supersede with a review queue; fix or permanently delete any fact (deletion cascades to everything derived from it) from the Memories panel
Scored core memory — the always-in-context fact set picked by pinning, importance, usage, and recency within a token budget
File checkpoints & undo — pre-images captured before agent file edits; inspect and restore with checkpoint.list / checkpoint.restore
Repeat-call circuit breaker + verify hooks — identical-call loops get stopped with an advisory; your configured checker runs after file edits and its real output feeds back to the agent
Internal eval harness (aibutler eval) — a deterministic benchmark suite that runs through the real agent loop; unit-mode baseline green in CI, first live baseline published in the repo
Self-authored skill proposals (opt-in) — successful multi-step work gets distilled into staged skills; activation is always an explicit human approval
Idle-time reflection cycle (opt-in) — a nightly deterministic maintenance pass that flags contradictions and stale facts; scheduled jobs can declare a strict capability subset
Per-profile memory banks — background workers read and write isolated memory by default

Computer control — the v0.2 → v0.5 arc

AI Butler acts on your computer through a tiered automation model, preferring the most reliable and auditable surface available before falling back to coarser ones:

Mission engine — long-running goals with supervisor → manager → worker orchestration, parallel (DAG) step dispatch, replanning on step failure, pause/resume/cancel, and a webchat dashboard panel (aibutler mode mission)
Native OS scripting — capability-gated AppleScript, D-Bus, Shortcuts, and PowerShell execution with per-action recording
Real browser automation — JavaScript-capable click/type/submit, not just HTTP fetches
Accessibility-tree reader (accessibility.read_ui) — read an app’s live UI hierarchy on macOS, Linux, and Windows, allowlist-gated
Vision + input tier (screen.capture, input.click/type/key) — cross-OS screen capture and synthetic input behind a double gate: a capability grant AND an explicit enable flag, both off by default
Vision input — images flow to vision-capable models (base64 and URL)
Clipboard tools, wait.until, cost.forecast, vault.request credential broker, permissions.check

On the roadmap

Hosted playground at demo.aibutler.dev
Smart home full integration — working Home Assistant adapter out of the box
Plugin marketplace with sample plugins
Multi-agent swarm cookbook with real-world delegation examples
Voice TUI — terminal mic capture + playback
Image generation via official APIs (Flux, Stable Diffusion, DALL-E)
Advanced TTS (ElevenLabs adapter)
Internet mode with autocert TLS + password + TOTP
PWA (installable web app)
Homebrew formula, SLSA Level 3 provenance, cosign binary signing
14-language i18n (including Arabic RTL, CJK)
External security audit + bug bounty program

By the numbers


Tests passing (race-free)	2,148
Go packages	120+
SQLite tables	74
Internal security audit passes	59
CVEs (`govulncheck` verified)	0
External Go dependencies	12 direct
CGO required	No
Channels wired	12 (2 ready: web chat + terminal; 10 beta)
AI providers wired	6+ (Claude, Ollama, Ollama Cloud ready; others beta)
License	Apache 2.0

Get started in 3 commands

git clone https://github.com/LumabyteCo/aibutler.git
cd aibutler && CGO_ENABLED=0 go build -o aibutler .
./aibutler vault set anthropic_api_key sk-ant-... && ./aibutler start

Then open http://localhost:3377. That’s it. You’re chatting with your own personal AI in under a minute.

Prefer a guided walkthrough with screenshots?

→ Read the 5-minute Quick Start guide — a screenshot-driven walkthrough that takes you from git clone to your first chat, with a tour of every panel in the dashboard (memories, connected apps, missions, spending, settings), plus dark mode and mobile.

Or jump straight to: Full install guide · Choose a model · All 12 channels · FAQ

Why we built this

We think a personal assistant earns trust when everything stays under your control: your chat history lives on your machine, your memory never resets, nothing you say trains anyone’s model, you pick the model — cloud or fully local — and nothing disappears if you stop paying anyone.

That’s AI Butler: self-hosted, yours. Your data, your models, your rules. Apache 2.0 licensed with a patent grant. Built by people who want their AI assistant to work offline, remember everything, and answer only to them.

The wow factor comes from what you can do with it, not from who’s hosting it.

Help build AI Butler

We’re a small team and we want this to be a community-driven project. Here’s where we need help most:

🧪 Test a beta channel with real credentials — file a report, push it from beta → ready
📝 Write a sample WASM plugin so new users have a reference implementation
🤝 Verify A2A v2 interop with a real third-party agent
🏠 Wire the Home Assistant adapter — the tool interface is ready, just needs the HTTP bindings
🌐 Translate the web UI to your language (14 languages planned)
🧑‍💻 Build editor extensions — VS Code, JetBrains, Zed — using the dashboard API
📹 Record a demo video showing memory + scheduling + channels

Start a discussion · Open an issue · Read the contributing guide