AI automation
Agent-run operations
Telegram-first agents that run real business operations, with approval gates and capability limits enforced in the architecture rather than in a prompt.
- Python
- Telegram Bot API
- OpenClaw
- Gemini
- Firefly III
- MinIO
- ffmpeg
- Instagram Graph API
- Docker
- Tailscale
- Operatorone Telegram thread
Self-hosted gateway
- Per-agent workspacespairing-based access
- Tool denialper agent, not per request
- Exec allowlistone binary per agent
Single-purpose agents
- Finance agentrecords transactions
- Content agentqueues posts
- Media agentfetches and classifies
Approval gate
- Scheduledqueued with a slot
- Awaiting approvalasked 24h before
- Approvedpublishes on time
- Skippedunanswered, never late
Systems of record
- Firefly IIIprivate network only
- Media archivebucket-scoped key
- Social platformhost scheduler publishes
- Staging and publishing run only from a host-side scheduler, so no agent tool path reaches the platform out of band.
- Classification produces a shortlist for a person to review; it is a guideline layer, never an automation that acts on its own output.
Overview
A set of small AI agents that actually operate things, reached from a phone over Telegram. One records spending into a self-hosted ledger from a sentence, a photo, or a voice note. One runs a scheduled social publishing queue that will not post without a human approval. One indexes a media archive so the useful footage can be found. They share a gateway, a security posture, and a rule: an agent proposes, a person disposes.
Problem
Most personal and small-business automation fails in one of two directions. Either it stays manual, so it depends on someone sitting down at a desk, or it goes fully autonomous, and the first time it acts on a wrong inference it does something public and irreversible. Both failure modes are about where the human sits, not about model quality. The design question is which capabilities an agent gets at all, and what it has to ask for.
Solution
Single-purpose agents on one self-hosted gateway, each in its own workspace with its own tool set. Reach is granted per agent through an explicit allowlist of executable paths, so an agent can run exactly one wrapper binary and nothing else. Anything that leaves the building goes through an approval gate delivered to the operator on Telegram: a scheduled post asks for approval in the twenty-four hours before its slot, and if nobody answers it is skipped rather than published late. Late is treated as a worse outcome than absent.
Architecture
Agents run in a self-hosted gateway container with per-agent workspaces, pairing-based access, and per-agent tool denial so a bot that must not see business or personal data cannot reach those tools at all. Each integration is a small standard-library wrapper reading its credential from a mounted secret file, never from the environment. The finance agent talks to Firefly III over a private container network so the ledger keeps no public exposure and needs no TLS to maintain. The publishing pipeline stages media, normalises aspect ratio with ffmpeg before the platform ever sees it, serves it from a short-lived public URL because the platform fetches rather than accepts uploads, then deletes it. The media archive is an S3-compatible bucket on the private network with a bucket-scoped key, filled from a picker-based fetcher and indexed by a vision model that writes a content category and a plain description per file.
Key decisions
Split by purpose, one agent per job, so a compromised or confused agent has a small blast radius. Enforce capability limits structurally: the publishing tools exposed to agents can queue, approve, and hold, but staging and publishing run only from a host-side scheduler, so no agent tool path reaches the platform out of band. Give every wrapper an explicit allowlist entry rather than a broad shell. Keep the ledger and the archive bound to the private network only. Treat classification as a guideline layer that produces a shortlist, never an automation that acts on its own output. Encode the domain rules the model would otherwise get wrong, for example that a credit-card payment is a transfer and not an expense, since recording it as spending double counts a purchase that already happened.
Challenges
The instructive failures were all quiet ones. An agent asked to pay a card that had no matching account picked a different account and reported success, so the confirmation read perfectly while the data was wrong; it now lists the real accounts and asks. A notification service logged nothing, swallowed its exceptions, and buffered its output, so a month of failures produced an empty journal behind a healthy service status. A media platform silently downgraded out-of-spec video to an ordinary post instead of rejecting it, and validated aspect ratio at publish time, long after anyone was watching. The pattern worth keeping is that the dangerous bugs here do not raise errors, so the work is in making the systems observable and making them ask.
Outcome
Working automation for things that genuinely used to be manual: a ledger that fills itself from a text message, a publishing schedule that survives a busy week, and a searchable archive. More usefully as engineering evidence, it is a worked example of drawing capability boundaries for AI agents, where the guarantee comes from the tool never having been built rather than from a prompt asking the model to behave.