repoman internals β a developer's guide
repoman is a self-hosted Go daemon that consolidates Git repositories from
GitHub, GitLab, Codeberg and Gitea into a single Forgejo instance. This guide explains
each package in plain language first, then links you into the full generated API
reference for the details.
How to read this: every card below has an ELI5 paragraph (the
mental model), the key types/seams to grep for, and who the package
talks to. The π API reference link
opens the complete go doc for that package. Regenerate the API pages any time with
./docs/dev/gen.sh (or make docs-dev) β they read straight from the
source doc comments, so they never go stale silently.
The one-paragraph mental model
Think of repoman as a warehouse with a loading dock. People (users) drop off
shipping orders (migration configs) that say "copy these repos from GitHub/GitLab/β¦ into my
Forgejo". A dispatcher (scheduler) decides when an order runs, a single
forklift driver (the worker) does one order at a time, and the
engine (GoRunner) actually drives to each source forge, picks up the repo, and
delivers it to Forgejo β including the tricky cargo (releases, assets, issues). Everything is logged,
encrypted, and survives a crash mid-delivery.
Request flow (a click in the browser)
browser ββHTTPβββΆ server/router
β (outermost first)
βΌ
access-log β CSRF (same-origin) β Auth (sessionβuser) β RBAC role gate
β
βΌ
server/handlers ββ the "controllers": parse form, check ownership,
β call a domain service, render a .gohtml template
ββββΆ auth / globalconfig / migrationconfig / theme / β¦ (domain)
ββββΆ db (hand-written SQL) βββ crypto encrypts secrets at rest
Run flow (a migration actually happening)
trigger (button β cron β recovery)
β
βΌ
run_queue (FIFO, SQLite)
β
βΌ
migrate.Worker ββ pops one entry, opens a per-run log, streams events over SSE
β
βΌ
migrate.GoRunner (the Runner seam)
ββ source.Client ββ list + clone repos from GitHub/GitLab/Gitea/Codeberg
ββ forgejo.Client ββ /repos/migrate, push-mirror, upload release assets
ββ syncReleases ββ repoman imports releases/assets itself (streamed, idempotent)
ββ issuesync ββ optional bidirectional issue/comment sync
β
βΌ
run + events persisted (run.Repository) β monitor page tails them live
Cross-cutting rules worth knowing before you read code (full rationale lives in
CLAUDE.md): SQLite is the pure-Go
modernc.org/sqlite so the
binary is
CGO_ENABLED=0 static;
all secrets are AES-256-GCM encrypted at rest
via
crypto; SQL is hand-written (no ORM) so the encrypt/decrypt seam stays visible;
CSRF is stdlib origin-checking, not tokens; the frontend is vanilla JS with no build step.
Entry point & web layer
How a process starts and how an HTTP request travels inward.
cmd/repoman
main
The CLI and the daemon β every way you start repoman.
This is the front door. Run repoman serve and it wires every package
together and starts the web server plus the background loops (worker, scheduler, backups, issue
reconcile, token-expiry). Run any other subcommand (user create-admin,
backup, emergency-resetβ¦) and it does that one job and exits. Every flag
also reads a REPOMAN_* env var, so containers configure by environment.
- Key files
serve.go (wires deps + starts loops), bootstrap.go (paths, DB, pepper), main.go (flags + env binding)
- Talks to
- everything β it's the composition root
API referenceCLAUDE.md β build & run
internal/server
internal/server
Builds the HTTP router and stitches handlers + middleware together.
The receptionist's floor plan. Build(Deps) takes one big bag of
dependencies and returns a ready http.Handler: it mounts every route, wraps them in the
middleware chain (in the right order), and exempts the machine-to-machine /webhooks/*
routes from the same-origin CSRF check.
- Key types
Deps, Build(), bypassForPrefix
- Talks to
- handlers, middleware
API reference
internal/server/handlers
internal/server/handlers
The "controllers": one handler per page/endpoint.
Each handler is a clerk for one task: read the form, check the signed-in user is
allowed (admin or owner β denials return 404 so a config's existence never
leaks), call a domain package to do the work, then render a .gohtml template. Templates
are parsed once at boot (a bad template panics at startup, not per request). This is the
biggest package β start from render.go (how pages render) and ownership.go
(who may see/manage what).
- Key types
RenderPage, BaseData, canManageConfig/canViewConfig, the *Handler structs
- Talks to
- every domain package; renders frontend templates
API referencemulti-tenant.md
internal/server/middleware
internal/server/middleware
The gauntlet every request passes through before a handler.
A stack of security guards. Auth turns a session cookie into a
*User in the request context (handlers read it with UserFromContext, never
the raw cookie). CSRF is the Go 1.25 stdlib CrossOriginProtection β it
allows same-origin form posts and rejects cross-site ones with 403, with no tokens to manage.
RequireRole is the coarse RBAC gate.
- Key types
Auth, RequireAuth, RequireRole, CSRF, UserFromContext
- Talks to
- auth (sessions/roles)
API reference
The migration domain
The heart of repoman: what to migrate, the engine that does it, and when it runs.
internal/migrationconfig
internal/migrationconfig
The data model for "a migration order" β plus validation and token rules.
A shipping order. A MigrationConfig says which sources
(github/gitlab/gitea/codeberg, each with an
optional custom URL) go to which Forgejo owner, and how (import-only vs. push-mirror, sync issues,
warn on token expiryβ¦). It owns the single source-of-truth rule for which token a source
uses (ResolveSourceToken: per-source β config override β the owner's account token,
keyed by platform + instance URL). Service creates/updates configs transactionally.
- Key types
MigrationConfig, Source, Settings, ResolveSourceToken, OwnerTokenKey, Service, ConflictAnalyser
- Talks to
- db (via
TxRunner), read by migrate
API referencemulti-tenant.md
internal/migrate
internal/migrate
The engine that actually performs a migration, plus the worker that drives it.
The forklift and its driver. The Worker pops one run off the FIFO queue
at a time, opens a per-run log, and broadcasts each line over SSE to the live monitor.
GoRunner (the default, behind the Runner seam) does the real work: list
repos, clone-and-migrate into Forgejo, then import releases + assets itself (streamed,
idempotent, resumable) and optionally sync issues. It is built to survive a crash: it skips repos
that already exist and resumes asset/issue sync, so a restarted run continues where it stopped.
- Key types
Runner (seam), GoRunner, ScriptRunner, Worker, RunConfig, RecoverOrphans
- Talks to
- source, forgejo, issuesync, run
API referenceCLAUDE.md β run recovery / releases
internal/run
internal/run
The vocabulary of runs: types + repository interfaces (no logic).
The shared dictionary. It defines what a Run, an Event (one
log line), and a QueueEntry are, plus the interfaces (Repository,
EventRepository, QueueRepository) that db implements and
tests fake. Keeping these here lets the worker, scheduler, and handlers agree on shapes without
importing the database.
- Key types
Run, Event, QueueEntry, ExitStatus, the three *Repository interfaces
- Talks to
- implemented by db; used by migrate/scheduler
API reference
internal/scheduler
internal/scheduler
Cron β enqueue a run at the right time.
The dispatcher with a wall clock. It wraps robfig/cron, polls the config
list every ~60s, rebuilds the cron schedule, and when a config's time comes it drops an entry on the
run queue β it never touches the worker directly, so the two stay decoupled.
NextRun(expr) also powers the "next run atβ¦" UI.
- Key types
Scheduler, NextRun
- Talks to
- migrationconfig (read), run (enqueue)
API reference
Talking to forges
The HTTP clients for the source platforms and the Forgejo destination.
internal/source
internal/source
One client interface, four forge implementations (GitHub/GitLab/Gitea/Codeberg).
The universal translator for "the places repos come from". NewClient maps
a platform + optional URL to the right API base (including GitHub Enterprise's /api/v3),
and each implementation knows how to list repos, list releases, stream asset bytes, manage issues,
and provision webhooks. Optional capabilities are type-asserted seams
(ReleaseClient, ReleasePublisher, WebhookProvisioner,
TokenInspector, AccessVerifier) so a forge lights up a feature simply by
implementing the interface.
- Key types
Client, NewClient, ReleaseClient/ReleasePublisher, IssueClient, WebhookProvisioner, TokenInspector
- Talks to
- called by migrate, issuesync, tokenexpiry
API reference
internal/forgejo
internal/forgejo
The client for the destination: the one Forgejo instance everything lands in.
The delivery driver's relationship with the warehouse. It knows the Forgejo API:
check/create repos, kick off /repos/migrate (deliberately without releases,
issues, or PRs β those are imported separately to keep the call fast and robust), add push mirrors,
stream-upload multi-GB release attachments via an io.Pipe so they never buffer in
memory, and provision issue webhooks.
- Key types
Client, MigrateRepo, RepoExists, AddPushMirror, UploadReleaseAttachment, EnsureIssueWebhook
- Talks to
- called by migrate and issuesync
API referenceCLAUDE.md β releases
internal/repoarchive
internal/repoarchive
Download a synced repo as a restorable .zip (all branches/tags).
A photocopier for repos. An admin can grab a backup of a Forgejo repo: it does an
all-refs mirror clone with the pure-Go go-git library (no git binary
needed β keeps repoman a single static binary) and zips the bare repo. Unzip and
git clone it to restore everything.
- Key types
Archiver
- Talks to
- used by handlers' repo-download endpoints
API reference
Issue hub
Optional, opt-in per config: keep issues and comments in sync both ways.
internal/issuesync
internal/issuesync
Bidirectional issue + comment sync between Forgejo (the hub) and each source.
A shared whiteboard with conflict rules. Forgejo is treated as canonical. For each
linked issue it remembers the last content it sent to each side (a "baseline"); a side
changed iff its current hash differs from its baseline β that single idea powers both
loop-avoidance and conflict detection (two-sided edits resolve hub-wins). Webhooks
are the fast path (signature-verified, CSRF-exempt) and a 15-minute reconcile poll is the
correctness backstop. Webhook secrets are derived from the pepper, never stored.
- Key types
Importer, Engine (3-way merge), Reconciler, webhook verify/parse
- Talks to
- forgejo, source, crypto (secret derivation)
API referenceissue-hub.md (design of record)
Data & secrets
Where state lives, how it's encrypted, and how it's backed up.
internal/db
internal/db
All hand-written SQL: repositories, migrations, and the encrypt/decrypt seam.
The filing cabinet, and the only package that speaks SQL. It uses the pure-Go
modernc.org/sqlite driver (so the binary needs no C toolchain) and
database/sql directly β no ORM, on purpose: the repos take an
*crypto.AESEncryptor and encrypt-on-write / decrypt-on-read around explicit
queries, which an ORM would fight. Schema changes are versioned .up/.down.sql files
applied at Open(). Config + ownership writes go through one transaction
(ConfigTxRunner) so there's never a half-saved config.
- Key types
Open, the *Repo types (User/Session/Run/MigrationConfig/Theme/β¦), ConfigTxRunner, InspectBackup, migrations/
- Talks to
- wraps crypto; implements interfaces from run, auth, migrationconfig, β¦
API referenceCLAUDE.md β no ORM / CGO-free
internal/crypto
internal/crypto
AES-256-GCM encryption for secrets at rest + per-config webhook-secret derivation.
The safe and the key-cutter. Every token, TOTP secret, and credential is AES-256-GCM
encrypted before it touches disk; the key is derived from the on-disk pepper file via
HKDF-SHA256 (generated once on first run, never rotate without a migration). The same pepper also
derives a stable per-config webhook secret (HKDF + per-id HMAC) so issue-hub hooks need
nothing stored or rotated.
- Key types
AESEncryptor, WebhookSecretDeriver
- Talks to
- used by db and issuesync
API reference
internal/backup
internal/backup
Scheduled DB snapshots, grandfather-father-son retention, and safe restore.
The off-site backups. It writes a consistent snapshot with SQLite's
VACUUM INTO on a daily schedule (polled, so a downtime gap is caught up), prunes old
ones with a GFS policy (newest of last N days / M weeks / K months; all-zero = keep everything),
and restores by staging an upload and applying it at the next startup before any DB
connection opens β the live DB is never swapped under open connections, and the old DB+pepper
are copied aside first for rollback.
- Key types
Create, Scheduler, GFSPolicy, ApplyPendingRestore
- Talks to
- operates on the db file + pepper
API reference
internal/globalconfig
internal/globalconfig
Instance-wide settings: Forgejo target, SMTP, auth policy, backups, token-expiry.
The control panel for the whole instance. Settings holds the knobs an
admin sets once (Forgejo URL/token, SMTP, session timeouts, backup schedule, token-expiry on/off),
and Service loads/saves them. Note there are no global platform tokens here
anymore β source credentials are per-user (see auth/account tokens).
- Key types
Settings, Service, AuthCfg, BackupCfg, TokenExpiryCfg
- Talks to
- persisted via db; read almost everywhere
API reference
Identity & safety
Who you are, what you may do, and the safety nets.
internal/auth
internal/auth
Passwords, sessions, two-factor auth, and role-based access control.
The bouncer and the wristband desk. It hashes/verifies passwords, runs TOTP 2FA
enrolment and login, issues and validates sessions (rolling window + absolute timeout + a
server-enforced idle logout via last_activity_at), and defines the roles/permissions
the middleware checks. Deliberately self-contained β it reaches the DB only through small interfaces
declared at the top of session.go, so it unit-tests in isolation.
- Key types
Authenticator, SessionManager, Role/permissions, TOTP helpers, HashPassword/VerifyPassword
- Talks to
- used by middleware & handlers; backed by db
API reference
internal/security
internal/security
A fail2ban-compatible security event log.
The CCTV feed for break-in attempts. It writes structured lines (failed logins, etc.)
to a security-*.log in a format the shipped fail2ban jail can parse β so repeated
attackers get banned at the firewall. It writes to an injected io.Writer rather than
owning a file, so rotation is handled by logging.
- Key types
Logger (security events)
- Talks to
- writes via logging's rotating writer; fed by auth handlers
API reference
internal/audit
internal/audit
An append-only audit trail of privileged actions.
The logbook you can't erase. Every sensitive action (role change, config delete,
password resetβ¦) appends an entry recording who did what, from where. It's fire-and-forget and
nil-safe so logging can never break the action it records.
- Key types
Logger, Action constants, LogIgnoringError
- Talks to
- called from handlers (
auditEntry); stored in db
API reference
internal/emergency
internal/emergency
The break-glass admin reset, run from the CLI.
The spare key under the mat. If an admin is locked out, repoman
emergency-reset calls Reset() to restore access (new password / clear 2FA)
directly against the DB β a last resort that doesn't need a working web login.
- Key types
Reset
- Talks to
- auth + db; invoked by
cmd/repoman
API reference
internal/tokenexpiry
internal/tokenexpiry
Warn β by email and dashboard banner β before a source token lapses.
A "your passport expires soon" reminder. A daily probe reads token expiry through the
source.TokenInspector seam (GitHub/GitLab report it; Gitea/Forgejo/Codeberg PATs don't
expire) and warns ahead of time. The crucial rule: a warning only reaches people who can
edit that token (global β admins, config β config editors). State is stored by token
location, never the token value (only a hash to detect rotation).
- Key types
Checker, Warner, EnumerateLocations, WithinThreshold
- Talks to
- source (probe), notify/email (warn), db (state)
API reference
Notifications
Telling users when something needs their attention.
internal/notify
internal/notify
One entry point to raise an in-app notification (with an email copy).
The town crier. Call Notify/NotifyUsers and it writes to the
per-user in-app feed (the nav bell) and, when SMTP is configured, sends an email copy β so callers
don't repeat the "feed + maybe email" logic everywhere.
- Key types
Service, Notify, NotifyUsers
- Talks to
- db (feed) + email
API reference
internal/email
internal/email
SMTP sending + the message templates.
The post office. It sends mail over SMTP and renders the bodies (password reset,
token-expiry warningβ¦). dynamicSender re-reads SMTP settings live, so changing them in
the UI takes effect without a restart.
- Key types
Sender, dynamicSender, SendPasswordReset, SendTokenExpiryWarning
- Talks to
- config from globalconfig; used by notify, auth, tokenexpiry
API reference
Look & feel
Everything the user sees: pages, themes, and the in-app manual.
frontend
git.griefed.de/griefed/repoman/frontend
The single //go:embed of all web assets (templates, CSS, JS, themes, help).
The box that bakes the website into the binary. This tiny package exists only to
//go:embed web/templates + web/static + web/themes + web/help into
frontend.WebFS, which handlers parse at boot. The JS is hand-written vanilla ES modules
(no bundler/TypeScript β evaluated and rejected to avoid a second toolchain), unit-tested with
Vitest (make test-js, a test-time-only Node dep).
- Key types
WebFS
- Talks to
- consumed by handlers, theme, help
API reference
internal/theme
internal/theme
UI themes as a fixed set of CSS variables, validated and rendered to one stylesheet.
The paint palette. A theme is just values for ~14 known CSS custom properties
(colours + radius + shadow); the app's CSS derives every widget colour from those via
color-mix(). Built-ins are .css files; custom/personal themes live in the
DB. Because theme CSS is served to everyone, ValidateValue is a strict whitelist (no
url(), no stray punctuation) to prevent CSS injection.
- Key types
KnownVars, ValidateValue, Parse, Stylesheet, CustomBlocks
- Talks to
- themes stored in db; served by handlers'
/themes.css
API referencetheme-sharing.md
internal/help
internal/help
The in-app user manual: Markdown topics rendered once at boot.
The glovebox manual. Each topic is a Markdown file under web/help/ plus
an entry in the ordered Topics registry; Load renders them all with
goldmark (pure Go) into an immutable Manual at startup β no per-request rendering, and a
missing/duplicate file fails the boot. The same files read fine straight from the repo.
- Key types
Topics, Load, Manual, Topic
- Talks to
- reads frontend's embedded help files; served by the help handler
API reference
Plumbing
Cross-cutting helpers that don't fit a feature.
internal/logging
internal/logging
In-process date-rotated log files (no external logrotate).
The self-rotating notebook. RotatingWriter keeps one file per UTC day
(<base>-YYYY-MM-DD.log), rolls on the first write of a new day, and prunes past a
retention window β pure Go, no dependency. It backs the app log, the security log, and the per-run
logs, all teeing where appropriate.
- Key types
RotatingWriter
- Talks to
- used by
cmd/repoman, security, migrate
API reference
internal/integrationtest
internal/integrationtest
Helpers for the opt-in, env-gated live integration tests.
The test harness's "are we allowed to hit the network?" switch. Live tests compile
normally but skip unless REPOMAN_INTEGRATION=1 and the specific endpoint env vars are
set, so the default go test never touches real services. These helpers read/require
those vars.
- Key types
Require, Env, Enabled, Flag
- Talks to
- imported by the
integration_test.go files
API reference