package gitsync // import "git.griefed.de/griefed/repoman/internal/gitsync" Package gitsync brings a Forgejo repository's git refs up to date with its source, for repos that are NOT Forgejo pull mirrors. Why this exists: a one-time Forgejo migration copies git history once and never refetches it. repoman creates releases itself (Forgejo's bundled importer is serial, opaque, and skipped entirely by mirrors), and binds each release to an EXISTING tag. So on every run after the first, a release created upstream names a tag the Forgejo copy has never seen, and Forgejo rejects it with HTTP 409 "Release has no Tag". Fetching the source's refs and pushing them into Forgejo closes that gap, so the commits a tag points at exist before the release referencing it is created. It also carries refs/notes/*, which is not an optional extra: semantic-release stores each prerelease tag's release CHANNEL in refs/notes/semantic-release, and a mirror without those notes cannot tell which channel its prerelease tags belong to — so a release run on a prerelease branch discards them all and walks back to the last stable tag. Tags without their notes are an incomplete mirror. Mirrored repos do not come through here — Forgejo pulls those itself, and forgejo.Client.MirrorSync asks it to do so now rather than on its interval. (A pull mirror fetches every ref including notes, so that path needs nothing.) Like internal/repoarchive this uses the pure-Go go-git library rather than shelling out to git, keeping repoman a single static CGO-free binary with no host tooling dependency. go-git resolves transports through a PROCESS-GLOBAL registry, so the certificate-skipping HTTPS client that repoarchive installs for the dev-only SkipTLSVerify flag also applies to the pushes made here — no separate opt-in is needed or wanted. VARIABLES var ErrIncompleteOutboundSpec = ErrIncompleteRefSyncSpec ErrIncompleteOutboundSpec is the Forgejo→target spelling of ErrIncompleteRefSyncSpec, kept so outbound callers can keep naming the error they already handle. var ErrIncompleteRefSyncSpec = errors.New( "gitsync: both a source and a destination clone URL are required") ErrIncompleteRefSyncSpec is returned when a RefSyncSpec lacks the source or the destination clone URL. There is no default for either side of a push. var ErrIncompleteSpec = errors.New("gitsync: both a source and a Forgejo clone URL are required") ErrIncompleteSpec is returned when a Spec lacks a source or target clone URL. Both are required; there is no meaningful default for either. FUNCTIONS func RemoteBranches(ctx context.Context, cloneURL string) ([]string, error) RemoteBranches returns the short branch names a remote advertises, without cloning it. It is the target half of the push-mirror prune check. An unfiltered Forgejo push mirror runs `git push --mirror --force`, which deletes every ref the target has and Forgejo does not — so to warn about that, repoman needs the target's branch list. A ref advertisement is the cheapest way to get it and the only one that works the same on all four platforms: one round trip, no objects transferred, no per-platform branches API to implement. The caller already builds an authenticated git URL for the target (the same one the mirror pushes to), so there is nothing extra to configure. Only `refs/heads/*` comes back. A mirror prunes tags and notes too, but the warning is about branches — reporting a tag among them would misdescribe what is at risk — and the peeled `^{}` entries a server advertises are not refs of their own. An empty repository is not an error: a target that exists and has never been pushed to genuinely has no branches. Anything else IS returned, because the caller must not read "could not ask" as "nothing at risk". func RemoteRefs(ctx context.Context, cloneURL, token string) (map[string]string, error) RemoteRefs returns every mirrorable ref a remote advertises — branches, tags and notes — as a map of full ref name to object hash, without cloning it. It exists to make the union import affordable. SyncRefs has to fetch a peer's whole history into a scratch repo before it can decide anything, and on a run that walks every repository on every platform that is the dominant cost. Two ref advertisements cost a round trip each and no objects, and they answer the only question that matters first: does this peer hold anything the hub does not? In the steady state of Forgejo-owned mirroring the answer is no — the push mirror has already made the two identical — so the fetch never happens. token authenticates the request when cloneURL carries no userinfo, using Forgejo's convention of accepting an access token as the basic-auth username. That is what lets the hub side be read from the clone_url Forgejo reports, which is credential-free. Peeled "^{}" entries are dropped: they are a server's way of advertising what an annotated tag points at, not refs of their own, and keeping them would make every annotated tag compare unequal against an identical copy. An empty repository is not an error — a repo that exists and has never been pushed to genuinely advertises nothing. Every other failure IS returned, because a caller must never read "could not ask" as "nothing to import". TYPES type DivergedRef struct { // Name is the short ref name, e.g. "9.0.0-alpha.1". Name string // Source is the object the source has for that ref. Source string // Forgejo is the object Forgejo has for it. Forgejo string } DivergedRef is one ref that both sides have, pointing at different objects. Both hashes are carried because "they differ" is not actionable on its own: an operator needs to see what the forge is holding to decide whether to delete it, force it, or leave it. type OutboundDivergence = RefDivergence OutboundDivergence is the outbound spelling of RefDivergence, aliased for the same reason as OutboundResult. type OutboundResult = RefSyncResult OutboundResult is the outbound spelling of RefSyncResult. An ALIAS rather than a second struct, so a result produced by either entry point satisfies both names and no conversion can lose a field. func PushOutbound(ctx context.Context, spec OutboundSpec) (OutboundResult, error) PushOutbound mirrors a Forgejo repository's refs onto an outbound target under an explicit destruction policy, and is the alternative to delegating that job to a Forgejo push mirror. It is a thin naming of SyncRefs: the hard part — deciding per ref what to do — is the same whichever way the refs travel, and keeping one implementation is what stops the two directions drifting apart on the classification that makes either of them safe. type OutboundSpec struct { // ForgejoCloneURL and ForgejoToken read the hub. ForgejoCloneURL string ForgejoToken string // TargetCloneURL is the remote repo to push to, possibly authed. TargetCloneURL string // Target names the platform for error messages; Label names the repo. Target string Label string // MatchBranch, AllowForce and AllowPrune carry the operator's policy; see // RefSyncSpec. MatchBranch func(branch string) bool AllowForce bool AllowPrune bool } OutboundSpec is the Forgejo→target spelling of RefSyncSpec, kept because the outbound call site reads better naming the two ends it actually has. type PruneResult struct { // Deleted holds the short names of branches removed from the hub. Populated // only when the deleting push succeeded; it reports what WAS removed, never // what was attempted. Deleted []string // Unmerged holds the short names of branches that no platform has any more but // that still carry work the default branch does not, and were therefore kept. Unmerged []string // Unresolved holds branches whose object could not be read as a commit, or // whose ancestry could not be computed. They are kept — an answer that cannot // be computed is never permission to delete — and named, so a branch that was // neither deleted nor explained does not simply vanish from the report. Unresolved []string } PruneResult reports what the rule did and, as importantly, what it declined to do — a branch kept for being unmerged leaves no other trace. func PruneMergedBranches(ctx context.Context, spec PruneSpec) (PruneResult, error) PruneMergedBranches deletes from the hub the branches that are finished everywhere, and returns what it removed and what it deliberately kept. It fetches the hub into a scratch bare repo because the decision needs the commit graph: "merged" is an ancestry question and no ref advertisement can answer it. Callers should therefore establish from the advertisements they already hold that a CANDIDATE exists before calling — a repo whose every branch is still live somewhere must not pay for a clone to learn that. An empty hub is a no-op rather than an opportunity, for the same reason an empty source is in SyncRefs: it is far more likely to be a failed import than an instruction to delete anything. type PruneSpec struct { // CloneURL is the hub repository to prune. It may embed credentials as // userinfo; they are lifted out before the remote is configured. CloneURL string // Token authenticates when CloneURL carries no userinfo, using Forgejo's // convention of accepting an access token as the basic-auth username. Token string // Label identifies the repo in error messages, e.g. "Griefed/repo". Label string // DefaultBranch is the hub's default branch, by short name. Required: without // it there is nothing to measure "merged" against, and a missing or // unresolvable value is an error rather than a reason to guess. DefaultBranch string // Keep holds the FULL ref names every platform still advertises. A ref in here // is never deleted whatever its ancestry. Keep map[string]bool } PruneSpec describes the one deletion repoman is allowed to perform on the Forgejo hub: removing a branch that is finished everywhere. It exists because the union import only ever ADDS. Once a bot's branch has been merged into the hub it outlives the pull request it belonged to, and the next sync pushes it back to the platform that deleted it — so the hub's branch list grows with every update branch ever opened. Before outbound configs stopped importing as pull mirrors this could not happen: a pull mirror prunes, so an upstream deletion propagated in by itself. Three conditions must ALL hold before a branch goes, and each one rules out a specific way of destroying something that mattered: - No platform still advertises it (Keep). A branch that is live somewhere is not finished, and deleting it here would have the mirror delete it there. - It is not the default branch. - Its tip is an ANCESTOR of the default branch — i.e. everything in it is already in the default branch. That is the same test a forge applies when it offers to delete a head branch on merge, and it is what distinguishes a finished bot branch from somebody's work in progress on the hub. Tags and notes are never considered. Forgejo is canonical for releases, so a tag the hub holds and a platform does not is something to push OUT, and a note carries a release channel nothing else records. type RefDivergence struct { // Name is the short ref name, e.g. "develop" or "1.2.3". Name string // From and To are the objects the source and the destination each have. From string To string } RefDivergence is one ref both sides hold at different objects, where the destination's object is NOT an ancestor of the source's — so sending it would rewrite history the destination owns. Both objects are carried because "they differ" is not actionable: an operator deciding whether to allow forcing needs to see what each side is holding. type RefSyncResult struct { // Pushed holds the short names of refs created or advanced on the destination. Pushed []string // Pruned holds the short names of refs deleted from the destination. Always // empty unless AllowPrune was set, and emptied again if the delete push failed // — it reports what WAS removed, never what was attempted. Pruned []string // Diverged names every ref the destination holds at an object that is not an // ancestor of the source's, and that was therefore skipped because AllowForce // was off. This is the only trace those refs leave, so it is never silent. Diverged []RefDivergence // TagsError, HeadsError, NotesError and PruneError hold the per-group failures. // They are separate because the groups are pushed separately: one rejected // branch must not hide the tags that went through, and a refused deletion (a // protected branch on the far side) must not hide anything at all. TagsError error HeadsError error NotesError error PruneError error } RefSyncResult reports what one sync achieved, in enough detail for a run log to name what moved and what was deliberately left alone. func SyncRefs(ctx context.Context, spec RefSyncSpec) (RefSyncResult, error) SyncRefs copies one repository's refs from one forge to another under the spec's destruction policy. It fetches the source into a scratch bare repo, reads the destination's own ref advertisement ONCE, and decides per ref what to do — rather than pushing a wildcard refspec and letting the far side reject it. That per-ref decision is not a refinement, it is required: go-git validates every ref in a refspec BEFORE sending any of them, so a single ref needing a force fails the whole group. On this project's own mirror that is exactly how six bad tags stopped 375 good ones, and the same trap applies in both directions — in the union direction it is worse, because the ref that fails the group is usually `main` diverging between two platforms, and the refs it would take down with it are the target-only branches the union exists to rescue. Tags go first because the outbound release publisher gates on the tag already being present on the target; then branches, then notes, each in its own push so one group's failure cannot hide another's. Deletions, if permitted at all, go LAST and alone: a destination that refuses one deletion (GitLab answers "You can only delete protected branches using the web interface") would otherwise take every update down with it. The scratch clone is always removed, so disk use is bounded by one repo at a time. type RefSyncSpec struct { // FromCloneURL is the repo to read from. It may embed credentials as // userinfo; they are lifted out before the remote is configured. FromCloneURL string // FromToken authenticates the fetch when FromCloneURL carries no userinfo, // using Forgejo's convention of accepting an access token as the basic-auth // username — the same arrangement Spec uses, so neither direction has to // embed a credential in a URL to say who it is. FromToken string // ToCloneURL is the repo to push to, likewise possibly authed. ToCloneURL string // ToToken authenticates the push the same way FromToken authenticates the // fetch. The union direction needs it: Forgejo reports a repo's clone_url // without credentials, so the destination URL carries none. ToToken string // FromName and ToName name the two ends for error messages — "github", // "Forgejo". Without them a failure reads the same whichever way it was // going, which is the one thing an operator needs to know first. FromName string ToName string // Label identifies the repo in error messages, e.g. "Griefed/repo". Label string // MatchBranch decides which branches are in scope, both for pushing and for // pruning. Nil means every branch. It is a func rather than a filter string // so this package stays free of the config model's glob syntax — the caller // owns what a filter MEANS, this owns what a push DOES. MatchBranch func(branch string) bool // AllowForce permits overwriting a destination ref whose object is not an // ancestor of the source's. Off, such a ref is reported in Diverged and // skipped, and every other ref still goes. AllowForce bool // AllowPrune permits DELETING refs the destination has and the source does // not. AllowPrune bool } RefSyncSpec describes one repository's refs being copied from one forge to another under an explicit destruction policy. It is deliberately direction-neutral, because repoman performs this operation BOTH ways and the hard part is identical in each: - OUTBOUND (Forgejo→target), as the in-process alternative to a Forgejo push mirror, which is `git push -f --mirror` and admits no variation: it force-overwrites whatever the target moved on its own and deletes every ref the push did not produce. - INBOUND UNION (platform→Forgejo), which collects the refs every other platform holds into the hub so that handing the outbound push to a Forgejo push mirror stops being destructive — a mirror can only prune what the hub is missing. What must NOT be shared between the two is the policy: the union never forces and never prunes, because repoman must not rewrite or delete Forgejo history. Those are fields on this struct precisely so each caller states its own, rather than the code inferring intent from which way it happens to be pointing. type Result struct { // TagsUpdated is true when the tag push moved at least one ref. TagsUpdated bool // HeadsUpdated is true when the branch push moved at least one ref. HeadsUpdated bool // NotesUpdated is true when the notes push moved at least one ref. NotesUpdated bool // HeadsError holds a non-fatal branch-push failure (typically a diverged // branch). Tags may still have been pushed successfully. HeadsError error // NotesError holds a non-fatal notes-push failure. Notes carry release-channel // metadata rather than history, so a rejected notes push must not hide the tags // and branches that went through — but it must be reported, because a mirror // without them breaks downstream release tooling in a way nothing else reveals. NotesError error // DivergedTags names every tag Forgejo holds at a different object than the // source does. They are skipped rather than pushed — updating one needs a force, // and repoman never rewrites a forge's history — so this is the only trace they // leave. Empty on a healthy sync. DivergedTags []DivergedRef } Result reports what the sync achieved, so the caller can log something meaningful and decide whether re-listing tags is worthwhile. func PushRefs(ctx context.Context, spec Spec) (Result, error) PushRefs fetches every ref from the source into a temporary bare repository and pushes the tags, then the branches, into Forgejo. It is idempotent: an already-current repo pushes nothing and reports no error. The temporary clone is always removed, so disk use is bounded by one repository at a time. type Spec struct { SourceCloneURL string ForgejoCloneURL string ForgejoToken string // Label identifies the repo in error messages, e.g. "Griefed/repo". Label string } Spec describes one source→Forgejo ref sync. SourceCloneURL may embed credentials as userinfo (that is how the source clients hand it out); ForgejoToken authenticates the push, using Forgejo's convention of accepting an access token as the basic-auth username.