How REX Turned ROSE Upstream Sync Into A Verified Workflow
REX still has a relationship with upstream ROSE, but it is no longer a simple fork that can rebase every few months.
The projects have diverged in ways that matter. Upstream ROSE still carries frontends, tools, and platform support that REX intentionally removed. REX uses the Clang and LLVM 22 frontend path for C and C++, uses Flang for Fortran, and has a different OpenMP implementation. It removed EDG, the old OFP parser, binary analysis, CodeThorn, Sawyer, Rosebud source trees, and several language frontends.
That does not make upstream irrelevant. ROSE still receives useful fixes in shared infrastructure: AST nodes, Sage interfaces, analysis utilities, unparser details, warning cleanups, version markers, and engineering repairs. The hard part is separating those useful changes from history that belongs to subsystems REX no longer has.
The 2026 sync turned that work from an ad hoc exercise into a workflow.
The goal was not merely:
| |
The goal was:
| |
That is a different standard.
Figure 1. The sync workflow is a controlled decision loop. Each upstream commit is evidence, not an automatic patch.
Why A Skill Was Needed
The old sync process was mostly human memory plus a document.
That worked while syncs were small, but it was fragile for an agent. A long sync has many ways to drift:
- skip an upstream commit by accident,
- port a change from a dropped subsystem,
- restore legacy files that REX intentionally removed,
- create a generic fixup commit instead of amending the sync commit that caused the problem,
- update the CSV ledger before commit SHAs are stable,
- run only local focused tests and miss a full-suite regression,
- treat review comments as new work instead of mapping them back to the owning upstream change.
Those are process failures, not just coding failures.
The REX repo now carries a rose-upstream-sync Codex skill and a matching guide. The skill is deliberately procedural. It tells an agent how to start, what is forbidden, how to classify commits, how to record decisions, how to validate, and how to handle review rounds after the PR exists.
The most important rule is that upstream ROSE is read-only evidence.
REX may fetch upstream ROSE, inspect logs, show commits, compare diffs, and use that information to decide what to port. It must never push to upstream ROSE, open upstream issues or PRs, request upstream reviews, or mention upstream maintainers. All work happens inside the REX fork.
That sounds administrative, but it affects the technical design. A sync is not collaboration with upstream in the moment. It is a downstream audit of upstream history.
Freeze The Range
The first operational rule is to freeze the upstream target.
A sync starts by fetching upstream ROSE, recording the exact upstream tip, and treating that SHA as the end of the session. If ROSE receives newer commits while the sync is in progress, those commits wait for the next sync.
This matters because otherwise “pending upstream work” keeps moving. A long agent session can easily lose track of what it already processed if the target changes underneath it.
The frozen range gives the task a stable question:
| |
There are only a few valid answers:
- apply the change,
- partially apply the useful retained-scope part,
- drop it because it belongs to a removed subsystem,
- mark it already present,
- defer or supersede it as an intermediate version marker,
- record it as not applicable.
The exact wording is less important than the invariant:
| |
For the 2026 sync, the final ledger recorded 201 upstream commits. That number is important because it is not just a count of applied commits. It is a count of decisions. A dropped binary-analysis commit and a picked read-write analysis fix are both part of the audit trail.
One Upstream Change, One Adapted REX Commit
The workflow intentionally avoids raw rebases.
Some upstream commits can be cherry-picked nearly as-is. Many cannot. REX has different frontends, different build systems, deleted paths, and locally modernized code. A useful ROSE change often needs REX-specific adaptation to fit the retained architecture.
The rule is:
| |
That one REX commit contains both the upstream idea and the adaptation needed for REX.
This matters later. If a test fails because a synced change was incomplete, the fix belongs in the same REX sync commit. It should not become a later “fix the sync” commit unless the issue is genuinely independent of a particular upstream change.
That policy keeps history readable:
| |
Y is the whole downstream answer to X. It is not the first guess.
The REX commit also carries trailers:
| |
Those trailers make the relationship searchable from Git history. The CSV ledger makes it searchable from the sync record. Both are needed because they answer different questions. Git tells a reviewer why a code commit exists. The CSV tells a future sync agent which upstream commits have already been processed.
The Dropped-Subsystem Guard
The biggest sync trap is accidentally restoring abandoned code.
ROSE still has many components that REX removed on purpose. A naive cherry-pick can bring those paths back as untracked or staged files. A subtler version is worse: a commit touches both retained infrastructure and a dropped subsystem, and a conflict resolution accidentally preserves the dropped part.
The skill therefore treats dropped subsystems as hard boundaries.
Examples include EDG, OFP, old Fortran parser paths, binary analysis, Sawyer, CodeThorn, Rosebud source trees, Ada, Jovial, Java, UPC, X10, Csharp, Matlab, JavaScript, Python, YAML, legacy autotools, and dropped OpenMP designs.
The guard script checks staged paths and branch ranges for those boundaries. It does not replace judgment, but it catches a common mechanical failure. If a useful idea appears inside dropped upstream code, the implementation must be ported manually into retained REX code. The path itself does not come back.
That was one of the strongest lessons from the sync: “useful upstream” and “safe to cherry-pick” are not the same thing.
Version Markers Are Special
Upstream ROSE version commits are another trap.
ROSE may increment version markers many times inside a sync range. REX should not replay every intermediate marker. It should record the intermediate version commits as superseded, then apply only the latest final marker after the rest of the range is processed.
For this sync, the final upstream marker was ROSE 2.1.0. REX applied that to the retained version files only:
| |
It did not restore upstream configure.ac or src/frontend/CxxFrontend/EDG_VERSION.
The encoded value also has to follow upstream’s current numbering rule. For the current three-component ROSE version, 2.1.0 maps to:
| |
The patch bound matches the current upstream convention; without it, adjacent minor and patch values could collide. The rule is checked by the sync validation script. The point is to avoid manual post-adjustment where one version file says one thing and another validation site says another.
Validation Is Layered
The sync workflow does not wait until the end to find out whether the branch builds.
After a code-affecting pick or partial pick, the branch should build before unrelated upstream work continues. Risky commits get focused tests. The final branch gets the full suite.
The validation ladder has several layers:
- path guard for dropped components,
- version marker verification,
- CSV coverage verification,
- normal build,
- focused tests for affected subsystems,
- full CTest before PR,
- pre-push and CI gates.
Figure 2. The sync is accepted only when evidence exists at several levels, not just when the last edited file compiles.
Coverage verification is especially important. It proves that the CSV ledger covers every upstream commit in the frozen range. Without that check, a large sync can accidentally skip a commit and still build cleanly.
The final 2026 sync passed:
| |
Those numbers are not decorative. They are the evidence that the sync did not simply produce plausible commits. It completed the frozen range and preserved the test suite.
PR Review Is Part Of The Sync
The workflow does not stop when the PR opens.
Review comments on a sync PR are handled differently from comments on a normal feature branch. A normal branch might add a follow-up review-fix commit. A sync branch usually should not.
If a reviewer points to a problem in synced behavior, the agent must identify which REX sync commit introduced the affected code or invariant. Then it amends that commit, rebases later sync commits, refreshes the CSV mapping to the rewritten SHAs, reruns validation, and force-pushes with lease.
That sounds expensive, but it protects the one-commit policy.
The difficult case is when a review comment does not name an upstream commit. It might just say that file A, B, and C have a problem. The workflow still requires ownership analysis:
- use
git blame, - inspect
git showfor candidate commits, - compare against upstream trailers,
- reproduce the failure when needed,
- decide whether one sync commit owns it or several do.
If several sync commits own separate issues, each one is amended separately. If the issue is truly independent integration work, a separate REX integration commit is allowed, but that is the exception.
Figure 3. Review comments are routed back to the owning sync commit whenever possible, so the final history remains a clean upstream-to-REX mapping.
CI failures follow the same model. Inspect the logs, reproduce locally when possible, find the culprit sync commit, amend it, refresh the ledger, and rerun the gates. Do not accumulate generic “fix CI” commits for sync-caused regressions.
The Catch
The catch is that this workflow is slower than blindly cherry-picking.
It asks the agent to stop at every upstream commit and make a real decision. It asks for builds during the sync, focused tests around risky changes, and full CTest before the PR is considered ready. It asks the PR author to rewrite history during review instead of stacking easy fixups.
That cost is the point.
REX is a compiler infrastructure. A sync that makes history look busy while quietly restoring abandoned paths or masking test failures is worse than no sync at all. The workflow is designed to make bad shortcuts hard:
- dropped components are guarded,
- skipped upstream commits are detected,
- version markers are verified,
- review fixes must map back to ownership,
- final acceptance depends on full CTest.
The result is a sync process that an agent can repeat without needing the whole policy conversation every time.
What The 2026 Sync Proved
The first 2026 run was not huge because every commit was applied. It was huge because every commit had to be understood.
Many upstream commits were dropped because they belonged to removed binary-analysis, CodeThorn, Sawyer, EDG, CI, or autotools paths. Some were partially ported because they contained retained analysis or AST utility changes mixed with upstream-only infrastructure. The useful retained work included read-write analysis improvements, dependence tracking attributes, warning cleanups, preprocessing-related cleanups, enum declaration signatures, memory-access subreference handling, and the final ROSE 2.1.0 version marker.
The important outcome was not only that the PR merged.
The important outcome was that future syncs now have a contract:
| |
That is what turns upstream sync from archaeology into maintenance.