As I am increasingly getting used to daily working with LLM agents, I also started to discover ways how an LLM agent could help me with daily SysOps tasks.
When I lately discovered the project Umpteenth, something clicked. Because an LLM agent usually doesn't do things in a deterministic way, I wasn't very inclined to use it for recurring tasks, which I want to be done every time the same way, when I know that they work.
The idea of Umpteenth: "Describe a recurring job in plain language. An AI agent runs it in a fresh sandbox and learns until a script does the work for next to nothing." This led me to ask my current LLM agent of choice, Pi, to create a Git project that contains rules and tools to work similarly: record what I ask the agent to do, encode this in scripts, and when it works three times in a row, finalize the script and use it from now on when tasked to do the job at hand.
Everything is tracked in Git, so I can see the history, and the agent has something to store the knowledge gathered to. Currently, I don't have a need for scheduled tasks, so I don't really need a long-running process; Pi CLI does the job for me. It's as simple as just spinning up Pi in my "tbrnt-ops" repo and asking it for sysadmin things to do.
The core loop
The whole process is defined in AGENTS.md:
- Before the task, look for a playbook. The agent searches
playbooks/for a file that matches the job. If one exists, the agent follows its steps exactly. It does not invent the steps again. - After the task, write the playbook. The agent records the exact commands that worked, the commands that failed, and why. New findings go under
## Learningswith a date. Therunscounter in the frontmatter goes up by one. - After three stable runs, write a script. If
runs >= 3and the steps did not change since the last run, the playbook graduates. The agent writesscripts/<task>.sh, tests it once, and setsstable: true. - If a script fails, read the playbook first. The agent repairs the script and records the failure cause in the playbook.
This worked quite well so far, and I love to see how the scripts evolve. All tracked in Git with proper history. I even started other repos working similarly, not tied to sysadmin tasks. Having different repos help me to keep different topics apart.
I will certainly iterate on this idea and probably give Umpteenth a try at some time.
If you want to try this yourself, either point your agent to this post or to Umpteenth and ask it to bootstrap a similar process. That's how I did get started and refined the process over time.
