From Laptop Chaos to Fedora Cloud: Decomposing Falcon into Podman Quadlets

From Laptop Chaos to Fedora Cloud: Decomposing Falcon into Podman Quadlets

When I wrote about the PAI framework, I said Falcon validates the whole thing against real cloud-native infrastructure — “Podman, Quadlets, bootable containers.” I left that as a promise. This is the post that pays it off. It is also the written version of a talk I prepared for DevConf.cz.

Raise your hand if you have ever wired an AI app together with a folder of shell scripts. Yeah. This is the talk about getting out of that.

The chaos (you know this stack)

Falcon is a real, running agentic framework. The domain happens to be markets, but the shape is any multi-service agentic app:

  • AI screening agents that rank the universe and explain their decisions with an LLM (falcon-screener)
  • A multi-agent trader whose agents propose, vote, and act (falcon-trader)
  • Schedulers that fire on a clock — pre-market, midday, evening
  • A messenger that publishes signals out to Bluesky and Discord (falcon-messenger)
  • Data and infra sidecars — PostgreSQL, Redis, a metrics stack, a router

And for a long time, the thing that started all of it looked like this:

# start-everything.sh  ← the thing we all have
podman run -d --name pg postgres ...
sleep 5                       # "wait for the db, probably"
podman run -d --name redis ...
nohup python screener_agent.py &      # which terminal was this?
nohup python trader_agent.py  &
( while true; do python sync.py; sleep 60; done ) &   # my "scheduler"

Start order? Hope. A crash? Gone. Reboot? Re-run it all by hand. Logs in five terminals. It “works on my laptop” and ships nowhere.

Underneath, the stack is really four kinds of thing wired together with one pile of ad-hoc glue:

Tier Components Today wired by…
Agents screener (LLM), trader (multi-agent), messenger nohup … &
Schedulers pre-market run, data sync while true; sleep
Data PostgreSQL, Redis “did I start it?”
Sidecars router, metrics, service registry more scripts

The glue is the enemy. The rest of this post replaces it with declarative units.

The insight

The thing supervising your laptop already exists: systemd. It has dependency ordering, restart-on-failure, boot integration, a scheduler (timers), logging (journald), and rootless services.

You don’t need a control plane to run a multi-agent app. You need to declare it. Podman Quadlets are how you hand your containers to systemd — as files.

I am not here to trash Kubernetes. Right tool, wrong scale. For one node — a laptop, a cloud VM — systemd is the orchestrator, and Quadlets are the on-ramp.

Why not Compose? Why not Kubernetes?

compose Kubernetes Quadlets
Boot integration bolt-on cluster native systemd
Restart / health basic great systemd + Podman
Scheduling none CronJob systemd timers
Ops surface a daemon enormous journald + systemctl
Rootless awkward n/a first-class

Already have a compose file? Don’t rewrite it — migrate it. podlet compose compose.yaml emits quadlet units. Note the tool: podman compose / podman-compose only run a compose file, they do not generate quadlets. The converter is podlet (dnf install podlet). And podman generate systemd is now deprecated in favor of Quadlet.

What is a quadlet?

A quadlet is a declarative unit file. Podman ships a systemd generator that turns it into a real .service:

# screener.container   →  generated as  screener.service
[Container]
Image=localhost/falcon-screener:latest
Network=falcon.network

You write *.container, *.network, *.timer. You run systemctl daemon-reload, the generator emits the .service, and systemctl --user start screener does the rest. Rootless by default. You never hand-write ExecStart=podman run … — the generator does it for you. One file per service replaces your start-everything.sh.

Decompose: one agent → one quadlet

Here is the screener agent as a single unit. Read it top to bottom — every line is something I used to fake with a shell script:

[Unit]
Description=Screener agent
After=postgresql.service redis.service     # ordering, not sleep 5
Requires=postgresql.service

[Container]
Image=localhost/falcon-screener:latest
Network=falcon.network
EnvironmentFile=/etc/falcon/falcon.env
EnvironmentFile=/etc/falcon/secrets.env    # model API keys, layered
HealthCmd=python3 -c "import urllib.request; urllib.request.urlopen('http://localhost:5000/health')"

[Service]
Restart=always
RestartSec=5

[Install]
WantedBy=default.target

Ordering replaces sleep 5. A layered EnvironmentFile is where the model API keys live. There is a healthcheck and a restart policy. One file replaces three lines of the chaos script and makes the agent survive a reboot.

The whole stack, as files

Falcon is about 19 quadlets across three tiers on one shared network:

  • Agents — screener · trader · messenger · dashboard
  • Schedulers (timer + oneshot) — pre-market run · midday · evening · data-sync
  • Data + sidecars — postgresql · redis · traefik · consul · prometheus · grafana

All on falcon.network (a 10.89.0.0/24 bridge). The pile of scripts is now an auditable, version-controlled directory of declarations.

Service discovery for free

# falcon.network
[Network]
Subnet=10.89.0.0/24
Driver=bridge

Every service gets Network=falcon.network, and the agents resolve each other by name — postgresql://postgresql:5432, http://screener:5000. No IP wrangling, no links, no --add-host. One file, referenced everywhere. This is the thing people reach for pods or compose to get; you get it from one .network file.

Pods vs. a shared network — my call

A pod is one network namespace: containers share localhost and one IP. I chose a shared bridge network instead, because for agents:

  • each agent needs an independent lifecycle — start/stop/restart/schedule alone
  • DNS-by-name already gives discovery; I don’t need a shared localhost
  • ordering comes from After= / Requires= without coupling lifecycles
  • scheduled agents run and exit — awkward inside a long-lived pod

A pod is one unit (app + sidecar). A network is peers with independent lives. Agents are peers.

Schedulers = timer + oneshot quadlet

# screener-morning.timer
[Timer]
OnCalendar=*-*-* 04:00:00 America/New_York   # timezone-aware
Persistent=true                              # caught up if the box slept
# screener-morning.container
[Container]
Image=localhost/falcon-screener:latest
Exec=--run-type morning
[Service]
Type=oneshot                                 # run the agent, write to DB, exit

cron can’t do timezones, Persistent=true catch-up, dependency ordering, journald, or systemctl list-timers. My while true; sleep 60 becomes a real scheduler. The common shape for AI here is a batch/ETL/eval agent that fires on a clock and exits — and Persistent=true matters for a laptop that sleeps.

Reliability you stop writing yourself

  • HealthCmd / HealthInterval — Podman healthchecks, surfaced to systemd
  • Restart=always / RestartSec — an agent crashes, it is back in 5s
  • After= / Requires= — DB and Redis before the agents
  • WantedBy=default.target — the whole stack returns on reboot, in order

No supervisor, no restart: unless-stopped cargo-culting. The OS does it. I deleted my babysitting scripts and got k8s-lite self-healing from files.

Where your model keys live

EnvironmentFile=/etc/falcon/falcon.env      # non-secret config
EnvironmentFile=/etc/falcon/secrets.env     # ANTHROPIC_API_KEY, … (chmod 600)
# or, better:
Secret=anthropic-key,type=env,target=ANTHROPIC_API_KEY

Last file wins, so you layer defaults then overrides. Podman secrets keep keys out of the image and out of ps. Same unit, different env per environment: laptop to cloud, no code change. Every agent needs model API keys, and this is the bridge to running the exact same units in the cloud.

From laptop to Fedora Cloud Edition

The same quadlets, unchanged, leave the laptop:

# Fedora Cloud Edition VM (cloud-init drops your quadlets in place)
/etc/containers/systemd/*.container
systemctl daemon-reload && systemctl start falcon.target

Boot the image, lay down the unit files, done. Rootless or system units — the same files you ran locally. That is the whole promise of the title: laptop chaos becomes reproducible cloud, because it is declarative.

Containers all the way down (bootc)

I build the agent images on Fedora — quay.io/fedora/*, dnf, a clean base. Then bootc / image mode ships the node itself as a versioned container image. My EKS worker is a Fedora 42 bootc image; an update is a rebuild and a reboot, with no config drift. Quadlets plus bootc means the OS and the workload are both declarative and rebuildable.

The rough edges (a WSL war story)

I run my real stack in my own Fedora WSL distro, and I wanted to see it in Podman Desktop. Podman Desktop is machine-centric: it manages its own machine as a first-class citizen (the Quadlet extension lists and edits units there), but it sees your own distro only over an SSH system connection on the CLI.

systemctl --user enable --now podman.socket   # rootless API
loginctl enable-linger "$USER"                 # survives logout
podman system connection add falcon --identity %USERPROFILE%\.ssh\key `
  ssh://you@localhost:22/run/user/1000/podman/podman.sock

Things that bit me: --default silently didn’t stick (podman system connection default falcon fixed it); the GUI shows machines, not SSH connections; and surviving wsl --shutdown needs systemd=true plus linger plus a logon task to boot the distro. Honest CLI-vs-GUI reality beats pretending it is seamless.

Lessons / gotchas

  • daemon-reload after every quadlet edit — the generator only re-runs then
  • EnvironmentFile order matters — last wins; layer config, then secrets
  • :Z on volumes, or fight SELinux AVCs
  • oneshot timers beat a long-running while-sleep agent every time
  • rootless plus privileged ports — publish, or give one service host networking

Takeaways

  1. Your start-everything.sh is an orchestrator spec — write it as quadlets.
  2. For one node, systemd is the orchestrator; quadlets expose it.
  3. Agents fit cleanly: long-running = .container, scheduled = timer + oneshot.
  4. The same files go laptop → Fedora Cloud Edition → bootc node.

From laptop chaos to Fedora Cloud — it’s just files.