package source // import "git.griefed.de/griefed/repoman/internal/source" Package source provides clients for fetching repository lists from source Git platforms. Each platform implements the Client interface. Supported platforms (each has a default URL that a source may override with a custom instance URL — so self-hosted GitLab/Gitea/GitHub Enterprise is "<platform> + URL override", not a distinct platform): - github (default https://github.com; Enterprise via override) - gitlab (default https://gitlab.com; self-hosted via override) - gitea (default https://gitea.com; self-hosted via override) - codeberg (default https://codeberg.org — runs Forgejo/Gitea API) VARIABLES var ErrAssetDeleteUnsupported = errors.New("source: release-asset delete not supported for this platform") ErrAssetDeleteUnsupported is returned by a ReleasePublisher.DeleteReleaseAsset for a platform that cannot delete a release asset by the identity repoman holds (GitLab, whose assets are package-registry-backed links with no size to verify against in the first place). Callers treat it as "leave the existing asset". TYPES type AccessVerifier interface { VerifyAccess(ctx context.Context) error } AccessVerifier confirms a source client's token authenticates — a cheap "whoami" call used as a run-start preflight so an invalid/expired/revoked token surfaces clearly and early, before any migration work. It is a type-asserted optional interface (like TokenInspector / ReleasePublisher), not part of Client, so callers verify it when present and skip it otherwise. Scope: this checks the token is VALID, not that it has write access to a specific namespace — repoman never writes to a source on import, and outbound write-permission still surfaces at push time. It catches the most common credential failure uniformly across platforms. type Client interface { // ListRepos returns all repos for the configured namespace. // For user namespaces this is the user's repos; for org namespaces // it is the org's repos. ListRepos(ctx context.Context) ([]*Repo, error) ReleaseClient IssueClient IssueWriter } Client fetches repositories, releases and issues from a source platform. It embeds ReleaseClient and IssueClient so a single platform client serves the repo inventory, the release/asset import, and the issue hub; callers needing only one surface can depend on the narrower interface instead. func NewClient( platform, namespace, token string, ownerType migrationconfig.OwnerType, instURL string, ) (Client, error) NewClient returns a Client for the given source platform. instURL is the optional web-base URL override; when empty the platform's default (migrationconfig.PlatformDefaultURL) is used. type Issue struct { // Number is the platform's issue number (GitHub/Gitea number, GitLab iid). Number int64 Title string Body string // State is normalised to "open" or "closed" (GitLab's "opened" is mapped). State string // Author is the original author's login/username, preserved in the synced // copy's attribution header (no platform lets us post as another user). Author string // URL links to the original issue (also used in the attribution header). URL string CreatedAt time.Time UpdatedAt time.Time } Issue is an issue on a source platform. Pull/merge requests are excluded by the clients (the issue hub syncs issues only), so callers never see them. type IssueClient interface { // ListIssues returns every issue (not pull/merge requests) of repo, open and // closed. ListIssues(ctx context.Context, repo string) ([]*Issue, error) // ListIssueComments returns the comments on one issue, oldest first, skipping // system/auto-generated notes. ListIssueComments(ctx context.Context, repo string, issueNumber int64) ([]*IssueComment, error) } IssueClient reads issues and their comments from a source platform — the inbound half of the issue hub. It is embedded in Client; every source platform implements it. type IssueComment struct { ID string Body string Author string CreatedAt time.Time UpdatedAt time.Time } IssueComment is one comment on a source issue. ID is the platform's comment id rendered as a string (uniform across platforms; stored in comment_links.source_comment_id). Auto-generated/system notes are excluded. type IssueSpec struct { Title string Body string State string } IssueSpec is the desired content of an issue when creating or editing it on a source platform (the outbound half of the hub). State is "open" or "closed". type IssueWriter interface { // CreateIssue creates an issue (honouring spec.State, closing it after create // where the platform cannot create-closed) and returns it with its number. CreateIssue(ctx context.Context, repo string, spec IssueSpec) (*Issue, error) // EditIssue updates an existing issue's title/body/state. EditIssue(ctx context.Context, repo string, number int64, spec IssueSpec) error // CreateIssueComment posts a comment and returns its platform id as a string. CreateIssueComment(ctx context.Context, repo string, issueNumber int64, body string) (string, error) // EditIssueComment edits a comment. issueNumber is required by some platforms // (GitLab notes are addressed by issue + note id); others ignore it. EditIssueComment(ctx context.Context, repo string, issueNumber int64, commentID, body string) error } IssueWriter creates and edits issues + comments on a source platform — the outbound half of the issue hub. Embedded in Client; every source platform implements it. The per-platform quirks (GitHub/GitLab can't set state at create time; GitLab edits a note by issue+note id; GitLab uses state_event) are hidden behind these methods. type Release struct { // ID is the platform's numeric release id. It is populated when listing a // platform's releases (github/gitea) and is what OUTBOUND release HA uses to // attach assets to an existing remote release; it is 0/unused on the inbound // path where releases are matched by tag. ID int64 // TagName is the git tag the release points at (e.g. "v1.2.3"). It is the // stable identity used to match a source release to a Forgejo one, so a // re-run skips releases that already exist instead of duplicating them. TagName string // Name is the human-facing release title; may be empty (defaults to the tag). Name string // Body is the release notes / changelog markdown. Body string // Draft reports an unpublished release; Prerelease a pre-GA one. Both are // carried across so the Forgejo release keeps the same status. Draft bool Prerelease bool // CreatedAt is the source publish/creation time, kept for reference only. CreatedAt time.Time // Assets are the files attached to the release. Source-archive entries that // platforms auto-generate from the tag are deliberately excluded — Forgejo // regenerates those from the imported git tag, so re-uploading them is waste. Assets []ReleaseAsset } Release is a single release on a source platform together with its downloadable binary assets. repoman imports releases itself — rather than letting Forgejo's migrate pull them — because Forgejo's bundled importer is serial, slow, and exposes no completion signal, and because Forgejo MIRRORS drop releases entirely. See internal/migrate for the sync that consumes this. type ReleaseAsset struct { // ID is the platform's numeric asset id, populated when listing a target's // releases so a mismatched copy can be deleted before re-upload (github/gitea). // 0 when the platform exposes none (gitlab links) or on the inbound path. ID int64 // Name is the file name as it should appear on the Forgejo release. Name string // Size is the asset size in bytes as the platform reports it (0 when unknown); // informational only — the actual bytes come from DownloadAsset. Size int64 // ContentType is the MIME type the platform reported (may be empty). ContentType string // DownloadURL is the platform URL DownloadAsset fetches the bytes from. It is // opaque to callers: only the owning Client knows how to authenticate it. DownloadURL string // Digest is the platform-reported content digest as "algo:hex" (e.g. // "sha256:abc…"), or empty when the platform exposes none. GitHub populates it // (asset.digest, since 2025); GitLab/Gitea/Codeberg do not. When present it // lets a (re)upload verify the streamed bytes in-flight (see uploadAsset). Digest string } ReleaseAsset is one downloadable file attached to a Release. DownloadURL is platform-specific and is fetched with that platform's auth scheme via the owning Client's DownloadAsset, so callers never hand-roll the credentials. type ReleaseClient interface { ReleaseLister // DownloadAsset opens the bytes of asset for streaming, authenticating with // the platform's scheme. The caller MUST Close the returned reader. The body // is streamed (not buffered) so multi-gigabyte assets don't sit in memory. DownloadAsset(ctx context.Context, asset *ReleaseAsset) (io.ReadCloser, error) } ReleaseClient is the inbound (read) surface: list a platform's releases and stream their assets down, for importing INTO Forgejo. Client embeds it. type ReleaseLister interface { // ListReleases returns every release of repo (the short name; the namespace // is fixed by the Client) with its non-source assets and platform ID, newest // first where the platform orders them. An empty slice means no releases. ListReleases(ctx context.Context, repo string) ([]*Release, error) } ReleaseLister lists a platform's releases — shared by the inbound importer (read source releases) and the outbound publisher (read what a mirror already has, for idempotency). type ReleasePublisher interface { ReleaseLister // TagExists reports whether tag is already present on the target repo. The // publisher must NOT create a release before its tag has been pushed there // (by the git mirror), or the platform would fabricate the tag at the wrong // commit — so the caller gates CreateRelease on this. TagExists(ctx context.Context, repo, tag string) (bool, error) // CreateRelease creates a release from spec and returns it. The returned // *Release carries whatever the platform needs to attach assets to it next — // a numeric ID (github/gitea) or just the tag (gitlab). CreateRelease(ctx context.Context, repo string, spec ReleaseSpec) (*Release, error) // UploadReleaseAsset streams asset onto rel. It takes the whole *Release // because platforms address a release differently (by ID or by tag), and an // UploadAsset rather than a bare reader because a streamed upload needs to // declare its length and to be re-openable (see UploadAsset). Streamed (not // buffered) so multi-GB assets stay out of memory. UploadReleaseAsset(ctx context.Context, repo string, rel *Release, asset UploadAsset) error // DeleteReleaseAsset removes asset from rel so a partial/size-mismatched copy // can be re-uploaded (release-HA integrity repair). asset carries the platform // identity it needs — ID for github/gitea. A platform that can't delete by that // identity returns ErrAssetDeleteUnsupported, which the caller treats as // "keep the existing asset" rather than a hard failure. DeleteReleaseAsset(ctx context.Context, repo string, rel *Release, asset *ReleaseAsset) error } ReleasePublisher is the outbound (write) surface: publish Forgejo releases onto a mirror target so downstream users can fetch them from any platform ("release HA"). Implemented by github and gitea/codeberg; NOT by every platform (GitLab's link/package-registry model is deferred), so callers type-assert to it and skip targets that don't support it. It reuses ListReleases to stay idempotent. type ReleaseSpec struct { TagName string Name string Body string Draft bool Prerelease bool } ReleaseSpec is the metadata needed to create a release on a target platform — the write counterpart of Release, free of platform IDs. type Repo struct { // Name is the short name, e.g. "my-project". Name string // Namespace is the owner, e.g. "alice" or "my-org". Namespace string // CloneURL is the HTTPS clone URL including authentication, // suitable for passing to Forgejo's migration API. // Format: https://<token>@host/namespace/name.git // or: https://oauth2:<token>@host/namespace/name.git CloneURL string // CloneURLNoAuth is the clean public URL without credentials, // used for display and logging. CloneURLNoAuth string // Description is the repo description (may be empty). Description string // Website is the repo's homepage/website URL (may be empty). GitLab projects // have no such field, so it is always empty there. Website string // Topics are the repo's topic labels (GitHub topics, GitLab topics, // Gitea/Codeberg topics). Nil when the platform reported none. Topics []string // Private reports whether the source repo is private. Private bool // Archived reports whether the source repo is archived. Archived bool // UpdatedAt is the source repo's last push/activity time. It is the // tiebreaker the GoRunner uses to pick the most up-to-date copy when the // same repo name appears on more than one configured source. Zero when // the platform did not report it (the dedupe then falls back to source // order). Mapped per platform: GitHub pushed_at, GitLab last_activity_at, // Gitea/Codeberg updated_at. UpdatedAt time.Time } Repo is a repository found on a source platform. type RepoMetadata struct { Description string // Website is skipped on platforms without the concept (GitLab). Website string // Topics replaces the target's topic set wholesale. Topics []string // Private, when non-nil, ENFORCES the target's visibility: true makes it // private, false makes it public. Nil — the default — leaves the target's // visibility alone entirely, which is what an ordinary metadata sync does. // // It is a pointer rather than a bool precisely so "do not touch visibility" // and "make it public" are different requests; a plain bool would make every // metadata sync silently assert public. Only a config with // Settings.ForcePrivacyOverrideOnTargets set passes it. // // GitLab note: its three-way visibility (private/internal/public) is read as // Private = (visibility != "public"), so an INTERNAL project compares equal to // a private source and is therefore left as internal rather than narrowed to // private. Enforcement only ever moves a target that genuinely disagrees. Private *bool } RepoMetadata is the descriptive metadata mirrored onto a target. Avatars are NOT here, and that is a platform limitation rather than an oversight: GitHub exposes **no public API** for a repository's avatar or its social-preview image (only an undocumented internal upload endpoint the web UI calls), so an avatar field would work on some targets and silently do nothing on GitHub. Rather than sync metadata that is quietly partial, avatars are left to the operator. Revisit only if GitHub ships a real endpoint. type RepoProvisioner interface { // RepoExists reports whether repo exists in the client's namespace. A missing // repo is (false, nil); only a genuine transport/API failure returns an error, // so a caller can tell "absent" from "could not tell". RepoExists(ctx context.Context, repo string) (bool, error) // CreateRepo creates an empty repository in the client's namespace. It is the // caller's job to only call this when RepoExists reported false; a platform // asked to create an existing repo returns its own already-exists error. CreateRepo(ctx context.Context, spec RepoSpec) error // EnsureReleasesEnabled makes repo able to ACCEPT releases and their assets, // because a target with the feature switched off rejects every publish — and // does so with an opaque status rather than a "feature disabled" message: // - Gitea/Forgejo: the `releases` repo unit can be off, and uploads then fail // (observed as HTTP 502 on an asset upload). Fixed by PATCHing // has_releases, the same field forgejo.Client.EnableReleases uses. // - GitLab: `releases_access_level` off makes the API answer 403 on EVERY // release endpoint regardless of token scope, and repoman streams GitLab // assets through the Generic Package Registry, so // `package_registry_access_level` must be on as well. // - GitHub: releases cannot be disabled, so this is a documented no-op. // Idempotent — enabling an already-enabled feature is a no-op on every // platform, so it is safe to call before each sync. EnsureReleasesEnabled(ctx context.Context, repo string) error // SyncMetadata copies descriptive metadata onto the target, OVERWRITING what // is there. A mirror is meant to look like what it mirrors, so a value edited // on the target is replaced rather than preserved — including being cleared // when the source has none. // // It writes only when the target actually DIFFERS. Implementations read the // target's current metadata first and return (false, nil) when it already // matches, because an unconditional write is both pointless churn and actively // harmful: a repo archived on the target rejects every write, so re-PATCHing // metadata that was already correct turned each run into a warning // ("Repository was archived so is read-only") for a repo that had nothing to // sync. A failed read falls through to the write, so an unreadable target // behaves as before rather than silently skipping the sync. // // changed reports whether the target was actually written to. The caller needs // it to log a visibility change HONESTLY: without it the run log announced // "enforcing visibility" once per repo per target whether or not anything // moved, which is noise on a large config and a claim about an action that may // not have happened. // // Not every field exists everywhere and the missing ones are skipped, not // faked: GitLab projects have no website/homepage field, and Gitea/Forgejo // keep topics behind a separate endpoint from the rest of the repo settings. // Avatars are deliberately absent from this seam — see RepoMetadata. SyncMetadata(ctx context.Context, repo string, meta RepoMetadata) (changed bool, err error) // ArchiveRepo marks repo archived — read-only — on the target platform, so a // project whose Forgejo hub has been archived is visibly finished everywhere it // is mirrored instead of looking live on every platform but the hub. // // One direction only, deliberately. UNarchiving is not offered: it would have // repoman reopen a repository a person deliberately froze on someone else's // platform, and there is no signal that distinguishes "the hub was unarchived // on purpose" from "the hub was never archived and this target was frozen by // hand". Reviving a mirror stays a human action. // // Idempotent on every platform: archiving an already-archived repo is a no-op // (GitLab documents this explicitly), so a caller may call it without checking // — though the engine skips targets it already knows are archived. // // Platform shapes differ and that is why this is its own method rather than a // RepoMetadata field: GitHub and Gitea/Forgejo take `archived` on the ordinary // repo PATCH, while GitLab has no such field and archives through a dedicated // POST /projects/:id/archive endpoint. ArchiveRepo(ctx context.Context, repo string) error } RepoProvisioner is the optional capability an outbound target needs for repoman to create the mirror destination itself. Why it exists: a Forgejo push mirror pushes to a remote that must ALREADY exist — no platform auto-creates it. Before this seam, outbound targets were wired up only while importing a repo for the first time, so enabling "sync back to" for a platform AFTER the import left every existing repo with no destination: the release publisher then hit 404 on the target's releases and tags on every run, forever. Implemented by the github, gitlab and gitea (also Codeberg/Forgejo) clients via a type assertion, in the same style as ReleasePublisher, AccessVerifier and TokenInspector — a platform that cannot provision simply does not satisfy it, and the caller reports that instead of failing the run. type RepoSpec struct { Name string Description string Private bool } RepoSpec describes a repository to create on an outbound target. Visibility is carried over from what is being mirrored rather than defaulted, so a private repo never becomes public by being mirrored. type TokenIdentifier interface { TokenIdentity(ctx context.Context) (TokenIdentity, error) } TokenIdentifier reports who a token belongs to. Optional and type-asserted, like TokenInspector and ReleasePublisher, so a caller that has only a Client asks for it when present and degrades to a plain reachability check when not. type TokenIdentity struct { // Account is the login the token authenticates as ("Griefed"). Always set on // a successful probe — it is the whole point of asking. Account string // Name is the account's display name, when the platform exposes one. Name string // Scopes are the permissions the platform reports for THIS token, sorted. // Empty where the platform does not tell us (Gitea/Forgejo expose no scope // listing for the token in use) — empty means "unknown", never "none". Scopes []string } TokenIdentity is what a token turns out to BE when you ask the platform: the account it authenticates as and, where the platform will say, what it is allowed to do. It answers the question a "test token" button is really asking — not "does this string work" but "whose token is this, and what can it reach". type TokenInspector interface { TokenExpiry(ctx context.Context) (expiresAt time.Time, known bool, err error) } TokenInspector reports the configured access token's expiry, when the platform exposes it. It is satisfied by all three concrete clients but kept separate from Client (like ReleasePublisher / WebhookProvisioner) so callers type-assert it. The `known` return distinguishes "the platform told us an expiry" from "there is nothing to warn about" — the latter covers a non-expiring token AND platforms with no token-expiry concept at all (Gitea/Forgejo). err is only for a failed probe, never for a missing/absent expiry. type UploadAsset struct { // Name is the file name the asset should have on the target. Name string // Size is the exact number of bytes Open will yield. It becomes the request's // Content-Length, so a wrong value fails the upload loudly (net/http refuses a // body that does not match) rather than truncating it silently. Zero means the // asset is EMPTY and is checked as such (see openUploadBody) — it is not a // stand-in for "unknown", and a negative size is rejected. Size int64 // Open starts a fresh read of the bytes. The caller closes what it returns. Open func() (io.ReadCloser, error) } UploadAsset is one file on its way ONTO a release: what to call it, how many bytes it is, and how to start reading them. Size and Open exist because "just hand me a reader" is not enough for a streamed HTTP upload. Without a known length Go cannot set Content-Length and falls back to chunked transfer encoding, which GitHub's upload endpoint rejects outright with "400 Bad Content-Length" — every asset, every run. And a one-shot reader cannot be replayed, so net/http refuses to retry a request whose body was already written ("cannot retry err ... after Request.Body was written; define Request.GetBody"), turning an ordinary REFUSED_STREAM into a lost upload. Open is called once per attempt, so the bytes come fresh each time. type WebhookProvisioner interface { // EnsureIssueWebhook idempotently ensures a webhook on repo delivers issue + // comment events to targetURL, signed/authenticated with secret. It returns // true when it created a new hook, false when an equivalent one already // existed (matched by delivery URL). EnsureIssueWebhook(ctx context.Context, repo, targetURL, secret string) (bool, error) } WebhookProvisioner provisions repoman's issue-hub webhook on a source repo so source-side issue/comment edits are delivered back to repoman. It is satisfied by all three concrete clients (github/gitlab/gitea, the last also serving codeberg) but kept separate from Client — like ReleasePublisher — so callers type-assert it and a future platform without an implementation degrades gracefully rather than failing to compile.