repoman internals β€” a developer's guide

repoman is a self-hosted Go daemon that consolidates Git repositories from GitHub, GitLab, Codeberg and Gitea into a single Forgejo instance. This guide explains each package in plain language first, then links you into the full generated API reference for the details.

How to read this: every card below has an ELI5 paragraph (the mental model), the key types/seams to grep for, and who the package talks to. The πŸ“„ API reference link opens the complete go doc for that package. Regenerate the API pages any time with ./docs/dev/gen.sh (or make docs-dev) β€” they read straight from the source doc comments, so they never go stale silently.

The one-paragraph mental model

Think of repoman as a warehouse with a loading dock. People (users) drop off shipping orders (migration configs) that say "copy these repos from GitHub/GitLab/… into my Forgejo". A dispatcher (scheduler) decides when an order runs, a single forklift driver (the worker) does one order at a time, and the engine (GoRunner) actually drives to each source forge, picks up the repo, and delivers it to Forgejo β€” including the tricky cargo (releases, assets, issues). Everything is logged, encrypted, and survives a crash mid-delivery.

Request flow (a click in the browser)

browser ──HTTP──▢ server/router
                    β”‚  (outermost first)
                    β–Ό
        access-log → CSRF (same-origin) → Auth (session→user) → RBAC role gate
                    β”‚
                    β–Ό
              server/handlers   ── the "controllers": parse form, check ownership,
                    β”‚               call a domain service, render a .gohtml template
                    β”œβ”€β”€β–Ά auth / globalconfig / migrationconfig / theme / …  (domain)
                    └──▢ db (hand-written SQL)  ◀── crypto encrypts secrets at rest

Run flow (a migration actually happening)

trigger (button β”‚ cron β”‚ recovery)
        β”‚
        β–Ό
   run_queue (FIFO, SQLite)
        β”‚
        β–Ό
   migrate.Worker  ── pops one entry, opens a per-run log, streams events over SSE
        β”‚
        β–Ό
   migrate.GoRunner (the Runner seam)
        β”œβ”€ source.Client   ── list + clone repos from GitHub/GitLab/Gitea/Codeberg
        β”œβ”€ forgejo.Client  ── /repos/migrate, push-mirror, upload release assets
        β”œβ”€ syncReleases    ── repoman imports releases/assets itself (streamed, idempotent)
        └─ issuesync       ── optional bidirectional issue/comment sync
        β”‚
        β–Ό
   run + events persisted (run.Repository) β†’ monitor page tails them live
Cross-cutting rules worth knowing before you read code (full rationale lives in CLAUDE.md): SQLite is the pure-Go modernc.org/sqlite so the binary is CGO_ENABLED=0 static; all secrets are AES-256-GCM encrypted at rest via crypto; SQL is hand-written (no ORM) so the encrypt/decrypt seam stays visible; CSRF is stdlib origin-checking, not tokens; the frontend is vanilla JS with no build step.

Entry point & web layer

How a process starts and how an HTTP request travels inward.

cmd/repoman

main

The CLI and the daemon β€” every way you start repoman.

This is the front door. Run repoman serve and it wires every package together and starts the web server plus the background loops (worker, scheduler, backups, issue reconcile, token-expiry). Run any other subcommand (user create-admin, backup, emergency-reset…) and it does that one job and exits. Every flag also reads a REPOMAN_* env var, so containers configure by environment.

Key files
serve.go (wires deps + starts loops), bootstrap.go (paths, DB, pepper), main.go (flags + env binding)
Talks to
everything β€” it's the composition root

internal/server

internal/server

Builds the HTTP router and stitches handlers + middleware together.

The receptionist's floor plan. Build(Deps) takes one big bag of dependencies and returns a ready http.Handler: it mounts every route, wraps them in the middleware chain (in the right order), and exempts the machine-to-machine /webhooks/* routes from the same-origin CSRF check.

Key types
Deps, Build(), bypassForPrefix
Talks to
handlers, middleware

internal/server/handlers

internal/server/handlers

The "controllers": one handler per page/endpoint.

Each handler is a clerk for one task: read the form, check the signed-in user is allowed (admin or owner β€” denials return 404 so a config's existence never leaks), call a domain package to do the work, then render a .gohtml template. Templates are parsed once at boot (a bad template panics at startup, not per request). This is the biggest package β€” start from render.go (how pages render) and ownership.go (who may see/manage what).

Key types
RenderPage, BaseData, canManageConfig/canViewConfig, the *Handler structs
Talks to
every domain package; renders frontend templates

internal/server/middleware

internal/server/middleware

The gauntlet every request passes through before a handler.

A stack of security guards. Auth turns a session cookie into a *User in the request context (handlers read it with UserFromContext, never the raw cookie). CSRF is the Go 1.25 stdlib CrossOriginProtection β€” it allows same-origin form posts and rejects cross-site ones with 403, with no tokens to manage. RequireRole is the coarse RBAC gate.

Key types
Auth, RequireAuth, RequireRole, CSRF, UserFromContext
Talks to
auth (sessions/roles)

The migration domain

The heart of repoman: what to migrate, the engine that does it, and when it runs.

internal/migrationconfig

internal/migrationconfig

The data model for "a migration order" β€” plus validation and token rules.

A shipping order. A MigrationConfig says which sources (github/gitlab/gitea/codeberg, each with an optional custom URL) go to which Forgejo owner, and how (import-only vs. push-mirror, sync issues, warn on token expiry…). It owns the single source-of-truth rule for which token a source uses (ResolveSourceToken: per-source β†’ config override β†’ the owner's account token, keyed by platform + instance URL). Service creates/updates configs transactionally.

Key types
MigrationConfig, Source, Settings, ResolveSourceToken, OwnerTokenKey, Service, ConflictAnalyser
Talks to
db (via TxRunner), read by migrate

internal/migrate

internal/migrate

The engine that actually performs a migration, plus the worker that drives it.

The forklift and its driver. The Worker pops one run off the FIFO queue at a time, opens a per-run log, and broadcasts each line over SSE to the live monitor. GoRunner (the default, behind the Runner seam) does the real work: list repos, clone-and-migrate into Forgejo, then import releases + assets itself (streamed, idempotent, resumable) and optionally sync issues. It is built to survive a crash: it skips repos that already exist and resumes asset/issue sync, so a restarted run continues where it stopped.

Key types
Runner (seam), GoRunner, ScriptRunner, Worker, RunConfig, RecoverOrphans
Talks to
source, forgejo, issuesync, run

internal/run

internal/run

The vocabulary of runs: types + repository interfaces (no logic).

The shared dictionary. It defines what a Run, an Event (one log line), and a QueueEntry are, plus the interfaces (Repository, EventRepository, QueueRepository) that db implements and tests fake. Keeping these here lets the worker, scheduler, and handlers agree on shapes without importing the database.

Key types
Run, Event, QueueEntry, ExitStatus, the three *Repository interfaces
Talks to
implemented by db; used by migrate/scheduler

internal/scheduler

internal/scheduler

Cron β†’ enqueue a run at the right time.

The dispatcher with a wall clock. It wraps robfig/cron, polls the config list every ~60s, rebuilds the cron schedule, and when a config's time comes it drops an entry on the run queue β€” it never touches the worker directly, so the two stay decoupled. NextRun(expr) also powers the "next run at…" UI.

Key types
Scheduler, NextRun
Talks to
migrationconfig (read), run (enqueue)

Talking to forges

The HTTP clients for the source platforms and the Forgejo destination.

internal/source

internal/source

One client interface, four forge implementations (GitHub/GitLab/Gitea/Codeberg).

The universal translator for "the places repos come from". NewClient maps a platform + optional URL to the right API base (including GitHub Enterprise's /api/v3), and each implementation knows how to list repos, list releases, stream asset bytes, manage issues, and provision webhooks. Optional capabilities are type-asserted seams (ReleaseClient, ReleasePublisher, WebhookProvisioner, TokenInspector, AccessVerifier) so a forge lights up a feature simply by implementing the interface.

Key types
Client, NewClient, ReleaseClient/ReleasePublisher, IssueClient, WebhookProvisioner, TokenInspector
Talks to
called by migrate, issuesync, tokenexpiry

internal/forgejo

internal/forgejo

The client for the destination: the one Forgejo instance everything lands in.

The delivery driver's relationship with the warehouse. It knows the Forgejo API: check/create repos, kick off /repos/migrate (deliberately without releases, issues, or PRs β€” those are imported separately to keep the call fast and robust), add push mirrors, stream-upload multi-GB release attachments via an io.Pipe so they never buffer in memory, and provision issue webhooks.

Key types
Client, MigrateRepo, RepoExists, AddPushMirror, UploadReleaseAttachment, EnsureIssueWebhook
Talks to
called by migrate and issuesync

internal/repoarchive

internal/repoarchive

Download a synced repo as a restorable .zip (all branches/tags).

A photocopier for repos. An admin can grab a backup of a Forgejo repo: it does an all-refs mirror clone with the pure-Go go-git library (no git binary needed β€” keeps repoman a single static binary) and zips the bare repo. Unzip and git clone it to restore everything.

Key types
Archiver
Talks to
used by handlers' repo-download endpoints

Issue hub

Optional, opt-in per config: keep issues and comments in sync both ways.

internal/issuesync

internal/issuesync

Bidirectional issue + comment sync between Forgejo (the hub) and each source.

A shared whiteboard with conflict rules. Forgejo is treated as canonical. For each linked issue it remembers the last content it sent to each side (a "baseline"); a side changed iff its current hash differs from its baseline β€” that single idea powers both loop-avoidance and conflict detection (two-sided edits resolve hub-wins). Webhooks are the fast path (signature-verified, CSRF-exempt) and a 15-minute reconcile poll is the correctness backstop. Webhook secrets are derived from the pepper, never stored.

Key types
Importer, Engine (3-way merge), Reconciler, webhook verify/parse
Talks to
forgejo, source, crypto (secret derivation)

Data & secrets

Where state lives, how it's encrypted, and how it's backed up.

internal/db

internal/db

All hand-written SQL: repositories, migrations, and the encrypt/decrypt seam.

The filing cabinet, and the only package that speaks SQL. It uses the pure-Go modernc.org/sqlite driver (so the binary needs no C toolchain) and database/sql directly β€” no ORM, on purpose: the repos take an *crypto.AESEncryptor and encrypt-on-write / decrypt-on-read around explicit queries, which an ORM would fight. Schema changes are versioned .up/.down.sql files applied at Open(). Config + ownership writes go through one transaction (ConfigTxRunner) so there's never a half-saved config.

Key types
Open, the *Repo types (User/Session/Run/MigrationConfig/Theme/…), ConfigTxRunner, InspectBackup, migrations/
Talks to
wraps crypto; implements interfaces from run, auth, migrationconfig, …

internal/crypto

internal/crypto

AES-256-GCM encryption for secrets at rest + per-config webhook-secret derivation.

The safe and the key-cutter. Every token, TOTP secret, and credential is AES-256-GCM encrypted before it touches disk; the key is derived from the on-disk pepper file via HKDF-SHA256 (generated once on first run, never rotate without a migration). The same pepper also derives a stable per-config webhook secret (HKDF + per-id HMAC) so issue-hub hooks need nothing stored or rotated.

Key types
AESEncryptor, WebhookSecretDeriver
Talks to
used by db and issuesync

internal/backup

internal/backup

Scheduled DB snapshots, grandfather-father-son retention, and safe restore.

The off-site backups. It writes a consistent snapshot with SQLite's VACUUM INTO on a daily schedule (polled, so a downtime gap is caught up), prunes old ones with a GFS policy (newest of last N days / M weeks / K months; all-zero = keep everything), and restores by staging an upload and applying it at the next startup before any DB connection opens β€” the live DB is never swapped under open connections, and the old DB+pepper are copied aside first for rollback.

Key types
Create, Scheduler, GFSPolicy, ApplyPendingRestore
Talks to
operates on the db file + pepper

internal/globalconfig

internal/globalconfig

Instance-wide settings: Forgejo target, SMTP, auth policy, backups, token-expiry.

The control panel for the whole instance. Settings holds the knobs an admin sets once (Forgejo URL/token, SMTP, session timeouts, backup schedule, token-expiry on/off), and Service loads/saves them. Note there are no global platform tokens here anymore β€” source credentials are per-user (see auth/account tokens).

Key types
Settings, Service, AuthCfg, BackupCfg, TokenExpiryCfg
Talks to
persisted via db; read almost everywhere

Identity & safety

Who you are, what you may do, and the safety nets.

internal/auth

internal/auth

Passwords, sessions, two-factor auth, and role-based access control.

The bouncer and the wristband desk. It hashes/verifies passwords, runs TOTP 2FA enrolment and login, issues and validates sessions (rolling window + absolute timeout + a server-enforced idle logout via last_activity_at), and defines the roles/permissions the middleware checks. Deliberately self-contained β€” it reaches the DB only through small interfaces declared at the top of session.go, so it unit-tests in isolation.

Key types
Authenticator, SessionManager, Role/permissions, TOTP helpers, HashPassword/VerifyPassword
Talks to
used by middleware & handlers; backed by db

internal/security

internal/security

A fail2ban-compatible security event log.

The CCTV feed for break-in attempts. It writes structured lines (failed logins, etc.) to a security-*.log in a format the shipped fail2ban jail can parse β€” so repeated attackers get banned at the firewall. It writes to an injected io.Writer rather than owning a file, so rotation is handled by logging.

Key types
Logger (security events)
Talks to
writes via logging's rotating writer; fed by auth handlers

internal/audit

internal/audit

An append-only audit trail of privileged actions.

The logbook you can't erase. Every sensitive action (role change, config delete, password reset…) appends an entry recording who did what, from where. It's fire-and-forget and nil-safe so logging can never break the action it records.

Key types
Logger, Action constants, LogIgnoringError
Talks to
called from handlers (auditEntry); stored in db

internal/emergency

internal/emergency

The break-glass admin reset, run from the CLI.

The spare key under the mat. If an admin is locked out, repoman emergency-reset calls Reset() to restore access (new password / clear 2FA) directly against the DB β€” a last resort that doesn't need a working web login.

Key types
Reset
Talks to
auth + db; invoked by cmd/repoman

internal/tokenexpiry

internal/tokenexpiry

Warn β€” by email and dashboard banner β€” before a source token lapses.

A "your passport expires soon" reminder. A daily probe reads token expiry through the source.TokenInspector seam (GitHub/GitLab report it; Gitea/Forgejo/Codeberg PATs don't expire) and warns ahead of time. The crucial rule: a warning only reaches people who can edit that token (global β†’ admins, config β†’ config editors). State is stored by token location, never the token value (only a hash to detect rotation).

Key types
Checker, Warner, EnumerateLocations, WithinThreshold
Talks to
source (probe), notify/email (warn), db (state)

Notifications

Telling users when something needs their attention.

internal/notify

internal/notify

One entry point to raise an in-app notification (with an email copy).

The town crier. Call Notify/NotifyUsers and it writes to the per-user in-app feed (the nav bell) and, when SMTP is configured, sends an email copy β€” so callers don't repeat the "feed + maybe email" logic everywhere.

Key types
Service, Notify, NotifyUsers
Talks to
db (feed) + email

internal/email

internal/email

SMTP sending + the message templates.

The post office. It sends mail over SMTP and renders the bodies (password reset, token-expiry warning…). dynamicSender re-reads SMTP settings live, so changing them in the UI takes effect without a restart.

Key types
Sender, dynamicSender, SendPasswordReset, SendTokenExpiryWarning
Talks to
config from globalconfig; used by notify, auth, tokenexpiry

Look & feel

Everything the user sees: pages, themes, and the in-app manual.

frontend

git.griefed.de/griefed/repoman/frontend

The single //go:embed of all web assets (templates, CSS, JS, themes, help).

The box that bakes the website into the binary. This tiny package exists only to //go:embed web/templates + web/static + web/themes + web/help into frontend.WebFS, which handlers parse at boot. The JS is hand-written vanilla ES modules (no bundler/TypeScript β€” evaluated and rejected to avoid a second toolchain), unit-tested with Vitest (make test-js, a test-time-only Node dep).

Key types
WebFS
Talks to
consumed by handlers, theme, help

internal/theme

internal/theme

UI themes as a fixed set of CSS variables, validated and rendered to one stylesheet.

The paint palette. A theme is just values for ~14 known CSS custom properties (colours + radius + shadow); the app's CSS derives every widget colour from those via color-mix(). Built-ins are .css files; custom/personal themes live in the DB. Because theme CSS is served to everyone, ValidateValue is a strict whitelist (no url(), no stray punctuation) to prevent CSS injection.

Key types
KnownVars, ValidateValue, Parse, Stylesheet, CustomBlocks
Talks to
themes stored in db; served by handlers' /themes.css

internal/help

internal/help

The in-app user manual: Markdown topics rendered once at boot.

The glovebox manual. Each topic is a Markdown file under web/help/ plus an entry in the ordered Topics registry; Load renders them all with goldmark (pure Go) into an immutable Manual at startup β€” no per-request rendering, and a missing/duplicate file fails the boot. The same files read fine straight from the repo.

Key types
Topics, Load, Manual, Topic
Talks to
reads frontend's embedded help files; served by the help handler

Plumbing

Cross-cutting helpers that don't fit a feature.

internal/logging

internal/logging

In-process date-rotated log files (no external logrotate).

The self-rotating notebook. RotatingWriter keeps one file per UTC day (<base>-YYYY-MM-DD.log), rolls on the first write of a new day, and prunes past a retention window β€” pure Go, no dependency. It backs the app log, the security log, and the per-run logs, all teeing where appropriate.

Key types
RotatingWriter
Talks to
used by cmd/repoman, security, migrate

internal/integrationtest

internal/integrationtest

Helpers for the opt-in, env-gated live integration tests.

The test harness's "are we allowed to hit the network?" switch. Live tests compile normally but skip unless REPOMAN_INTEGRATION=1 and the specific endpoint env vars are set, so the default go test never touches real services. These helpers read/require those vars.

Key types
Require, Env, Enabled, Flag
Talks to
imported by the integration_test.go files