Runnerly · Private preview

Self-hosted runners, without guesswork.

Runnerly is a control room for the runner layer GitHub doesn't manage for you: fleet readiness, repository policy, tokenized enrollment, and CI audit trails, in one place a senior SRE would actually leave open. Runnerly reflects CaeliCode's platform engineering work around self-hosted CI infrastructure.

Request preview access See how it works
Status Private preview, accepting design partners Stack Go · Postgres · GitHub App Source github.com/caelicode/runnerly
What it is

A control plane for the runner layer GitHub doesn't manage for you.

Runnerly sits beside GitHub Actions and turns self-hosted runners into an observable, governed fleet, not a set of machines people hope are still alive. Four capabilities, each with the artifact that backs it up.

01 -

Fleet readiness

Heartbeats, labels, architecture, status, runner scope, and recent checks from managed agents and GitHub events, surfaced in one table you can leave open on a second monitor.

# GET /api/v1/runners/caelicode-runner-01
{
  "name":        "caelicode-runner-01",
  "labels":      ["self-hosted", "linux", "arm64"],
  "status":      "online",
  "heartbeat":   "2026-05-18T14:22:08Z",
  "scope":       "org:caelicode",
  "checks": { "dns": "pass", "actions_api": "pass" }
}
02 -

Repository policy

Which repositories are allowed, which labels they require, and whether an online runner can satisfy them, checked when the workflow event arrives, not after it fails.

repositorygithub-user-management
requires[self-hosted, arm64]
online matchcaelicode-runner-01
fallbackcaelicode-runner-02
decisionallow · 2 candidates
03 -

Tokenized enrollment

Short-lived registration tokens minted through a GitHub App, runner setup never depends on a long-lived personal access token sitting on a workstation.

$ runnerly enroll --repo github-user-management

✓ minted registration token  ttl=900s
✓ signed by GitHub App        app=runnerly-prod
✓ issued to operator          d.marin@caelicode
✓ bound to runner             caelicode-runner-03

  # expires 2026-05-18T14:37:08Z · revocable
04 -

Job evidence

Every workflow job links back to the repository, the runner labels it required, the webhook delivery, and the operator events around it, a queryable audit trail, not a binder.

jobbuild-image #2491
workflowonboarding.yml @ 9c2f7d8
runnercaelicode-runner-01
deliverywh_01HXR3J2W4
operatord.marin · 14:18:42
Architecture

GitHub schedules the work. Runnerly owns the runner boundary.

The control plane receives workflow events, keeps a local system of record, coordinates runner agents through authenticated API calls, and gives operators a private dashboard for readiness and audit trails.

Inbound
workflow_job events arrive at a signed webhook; HMAC + replay-protected.
System of record
Postgres-backed policy store with append-only audit log; every decision is queryable.
Outbound
Authenticated API calls to runner agents; short-lived tokens via the GitHub App.
Guardrails

Useful now. Honest about what comes next.

Runnerly is a private-preview operations product, not a SaaS. We tell you what it covers today, what we're building, and what we're explicitly not claiming yet, before you ask.

Now · v0.4

What it does today.

  • Private labs and small-team fleets
  • Repository onboarding with label matching
  • ARM64 and x86_64 runner visibility
  • workflow_job evidence with audit trail
  • Operator runbooks and on-call dashboards
Not yet

What we aren't claiming.

  • Multi-tenant SaaS or public sign-up
  • Billing or metered usage
  • Public dashboard exposure
  • Compliance certification claims
  • A replacement for GitHub Actions scheduling
First hour

One repo, one runner, one terminal window.

The operating model is small enough to evaluate in an afternoon. Pick one repository, register one runner, watch a workflow_job land, then decide whether to scale it.

01

Install the GitHub App

Five clicks. Scoped to a single org or a single repo. No personal access tokens, no shared service accounts.

~2 min
02

Add the first repository

Declare the labels it needs. Runnerly checks them against the online fleet on every workflow_job.

~3 min
03

Enroll a runner

One CLI command. A 15-minute, revocable token is minted, bound to the runner, and rotated automatically.

~5 min
04

Trigger a workflow

Watch the job hit the queue, get matched to your runner, and land in the audit log with every linkage intact.

~10 min for the first run
runnerly, first run, zsh
$ runnerly app install --org caelicode ✓ GitHub App authorized app=runnerly-prod ✓ webhook endpoint verified /v1/hooks/github $ runnerly repo add github-user-management \ --requires self-hosted,arm64 ✓ policy stored requires=[self-hosted, arm64] ! no online runner satisfies this label set yet $ runnerly enroll --repo github-user-management ✓ minted registration token ttl=900s · revocable ✓ issued to operator d.marin@caelicode ✓ bound to runner caelicode-runner-01 # on the runner host: $ ./config.sh --url $URL --token $RUNNERLY_TOKEN ✓ runner registered labels=[self-hosted, linux, arm64] ✓ heartbeat received latency=18s $ gh workflow run onboarding.yml ✓ workflow_job queued ✓ matched runner caelicode-runner-01 ✓ picked up +1.4s ✓ completed +47s · audit=wh_01HXR3J2W4
Questions we hear most

The boring answers, up front.

Is Runnerly a replacement for GitHub Actions?

No. GitHub still schedules the work. Runnerly sits beside Actions and owns the runner boundary, fleet readiness, policy, enrollment, and evidence. If you turn Runnerly off, your workflows still run; you just lose the operations console and the audit trail.

What does "private preview" mean in practice?

It means we're working with a small set of design partners on real fleets, not running a public sign-up. We deploy Runnerly into your environment, operate it with you for the first month, and we're explicit about the things on the "not yet" list above.

Does Runnerly host my runners?

No. Your runners stay on your infrastructure, EC2, on-prem, GovCloud, wherever they already live. Runnerly talks to them through authenticated API calls. The runner boundary is yours; we just make it observable.

How does enrollment avoid long-lived tokens?

Registration tokens are minted by a GitHub App with a 15-minute TTL, bound to a specific runner identity, and revocable from the dashboard. Operators never copy a personal access token onto a runner host.

Where does the audit log live?

In your Postgres. Append-only. We give you a queryable schema and a JSON export endpoint so you can pipe it into Splunk, Loki, or whatever your SOC already runs. We do not retain a copy on our side.

Is Runnerly open source?

The runner agent and the CLI are MIT-licensed at github.com/caelicode/runnerly. The control plane is source-available to preview customers under a non-compete license. We expect to relicense parts of the control plane once the API surface stabilizes.

Private preview

Bring discipline to the runners already carrying your CI.

Start with one repository and one self-hosted runner. Prove the operating model in an afternoon, then decide how far Runnerly should go in your environment.

Start with Runnerly DevOps services