Documentation/Tutorials/ Multi-agent review CI

Production pattern · GitHub Actions

Put a resilient review panel in CI.

Four read-only agents review one pull-request diff in parallel. A fifth agent consolidates duplicate findings, then deterministic code turns the result into a job summary, downloadable reports, and GitHub code-scanning annotations.

01 / ARCHITECTURE

Parallel opinions, one deterministic delivery contract

The production-derived graph contains 19 nodes and 55 connections. It separates probabilistic work from deterministic CI work: models discover and consolidate findings, while Knodr controls concurrency and failures and Python produces stable report formats.

INPUTPR diff + contextfile path · title · repository
PARALLEL4 reviewersCodex · Claude · Gemini · Grok
REDUCEConsensus + renderclusters · SARIF · reports
Reviewers are read-only

Each agent receives the same task and repository context. It returns only a JSON array of findings and cannot edit the checkout.

Partial failure is expected

One unavailable provider contributes an error section but does not abort the graph. A deterministic total-outage guard still fails the run.

Consensus is separate

A fifth agent clusters duplicate findings, preserves source attribution, and chooses the strongest severity and confidence.

Rendering is code

Python—not another prompt—writes Markdown, HTML, and SARIF and fails loud when consensus output is missing or unparseable.

02 / INPUTS & OUTPUTS

Pass paths through variables; keep the diff out of the command line

The CI job writes the full UTF-8 diff to disk. The prompt contains only its path, so command-line limits and prompt-template escaping do not truncate or corrupt large changes.

VariableExamplePurpose
repoC:\actions\_work\sample-app\sample-appRead-only working directory and root for report artifacts.
diffPath<repo>\.mar-tmp\diff.txtBOM-free UTF-8 unified diff generated by CI.
commentsPull-request titleExtra context shared by all reviewers.
rendererPath<base-tooling>\render.pyTrusted renderer checked out from the PR base revision.

Generated artifacts

PathConsumerContent
.mar-out/mar-report.mdJob summary · humans · agentsFindings grouped as Critical, Should fix, and Advisory.
.mar-out/mar-report.htmlArtifact browserStandalone human-readable report.
.mar-out/mar-results.sarifGitHub code scanningInline file and line annotations with stable fingerprints.
.mar-out/agents-raw.logFailure diagnosisExit code, stdout, stderr, and error summary for all five agents.
.mar-out/agents-aggregate.txtConsensus agentFour result sections and four error sections.

03 / FLOW PROCESS

Follow the graph from diff to CI annotation

  1. 01
    Format one strict review task

    repo, diffPath, and comments feed a prompt that limits review to bugs and risks introduced by this diff. Every reviewer must return a JSON array.

  2. 02
    Fan out to four vendors

    Codex and Claude run directly; Gemini and Grok run through OpenCode with isolated data directories. All four branches start eagerly and run concurrently.

  3. 03
    Capture success and failure separately

    Each agent exposes stdout and a typed error output. failOnError=false keeps surviving branches useful when one provider is down.

  4. 04
    Reject a false green outage

    An availability formatter collects reviewer stdout. logic.assert fails the run when every result is empty.

  5. 05
    Write before consensus

    The aggregate is written to disk, and the returned file.write.path activates the consensus task. This edge guarantees that the file exists before the fifth agent reads it.

  6. 06
    Cluster, attribute, and rank

    Consensus merges only the same underlying problem, records all source reviewers, takes the strongest severity, and marks high-agreement clusters critical.

  7. 07
    Render and preserve evidence

    The trusted Python renderer writes reports and SARIF. In parallel, the raw capture is written for diagnosis. Malformed consensus or total reviewer outage exits non-zero after artifacts are saved.

Failure semantics: an individual reviewer outage is yellow information inside a successful report. Missing consensus, an invalid render, or all reviewers being unavailable is a real CI failure.

04 / WINDOWS RUNNER PREREQUISITES

Provision the runner before connecting secrets

  1. Add a dedicated Windows x64 self-hosted runner at repository or organization level and give it the custom label ai-review. The workflow matches all four labels: self-hosted, windows, x64, and ai-review.
  2. Run the GitHub Actions runner as a service under a low-privilege account. Prefer an ephemeral VM or a machine dedicated to review jobs.
  3. Install current versions of Knodr, Git, Python 3, Claude CLI, Codex CLI, and OpenCode. Put every executable on the service account’s system PATH, not only your interactive user path.
  4. Update the Actions runner to at least 2.327.1. The current checkout@v6 and upload-artifact@v7 actions use Node 24 and require a current self-hosted runner.
  5. Allow outbound HTTPS to GitHub and the selected model providers. Do not expose inbound ports.
PowerShell · run as the runner service account
$ knodr --version
$ python --version
$ git --version
$ claude --version
$ codex --version
$ opencode --version
$ knodr validate .\ci\mar\ci-multi-agent-review.knodrflow

GitHub documents the supported Windows editions, routing behavior, labels, and service setup in its self-hosted runner reference and workflow routing guide.

05 / ACCESS KEYS IN CI

Store provider keys in a protected GitHub environment

Create an environment named ai-review under Settings → Environments. Add required reviewers or deployment-branch restrictions, then create these environment secrets:

GitHub secretKnodr credentialInjected into agent
MAR_ANTHROPIC_API_KEYclaudeANTHROPIC_API_KEY
MAR_OPENAI_API_KEYopenaiCODEX_API_KEY
MAR_GEMINI_API_KEYgeminiGEMINI_API_KEY
MAR_XAI_API_KEYgrokXAI_API_KEY

The workflow maps GitHub secrets to KNODR_CREDENTIAL_*. Knodr resolves those values at runtime and injects each one only into the process environment named by that node’s secretEnv. No key is serialized into the flow or passed through --var.

GitHub CLI · values are entered interactively
$ gh secret set MAR_ANTHROPIC_API_KEY --env ai-review
$ gh secret set MAR_OPENAI_API_KEY --env ai-review
$ gh secret set MAR_GEMINI_API_KEY --env ai-review
$ gh secret set MAR_XAI_API_KEY --env ai-review
Never put keys in YAML, --var, a flow file, or runner-wide plaintext configuration. GitHub only exposes a secret when the workflow references it; environment protection adds an approval boundary before the job receives it. See GitHub’s official secrets guide.

06 / INSTALL THE EXAMPLE

Commit three files with neutral, repository-relative paths

Repository layout
.github/
  workflows/
    multi-agent-review.yml
ci/
  mar/
    ci-multi-agent-review.knodrflow
    render.py
  1. Save the downloaded flow and renderer under ci/mar/.
  2. Save the workflow under .github/workflows/.
  3. Validate the flow locally. It should report valid: 19 nodes, 55 connections.
  4. Add /.mar-tmp/ and /.mar-out/ to .gitignore.
  5. Protect .github/workflows/** and ci/mar/** with CODEOWNERS.

The GitHub variant takes rendererPath explicitly. The workflow checks the PR code into the normal workspace but checks flow tooling from the trusted base SHA into _mar-tooling. A PR therefore cannot silently replace the executable renderer used for its own review.

07 / GITHUB ACTIONS PROCESS

Generate one clean diff, then let Knodr own the panel

Checkout headCheckout base toolingBuild diffRun KnodrSummary + SARIF + artifacts

The complete downloadable workflow includes cancellation of obsolete runs, a 40-minute job limit, BOM-free UTF-8 output, pathspec exclusions for binaries and generated files, and seven-day artifacts. The core invocation is:

.github/workflows/multi-agent-review.yml · core steps
jobs:
  review:
    runs-on: [self-hosted, windows, x64, ai-review]
    environment: ai-review
    timeout-minutes: 40
    continue-on-error: true
    env:
      KNODR_CREDENTIAL_CLAUDE: ${{ secrets.MAR_ANTHROPIC_API_KEY }}
      KNODR_CREDENTIAL_OPENAI: ${{ secrets.MAR_OPENAI_API_KEY }}
      KNODR_CREDENTIAL_GEMINI: ${{ secrets.MAR_GEMINI_API_KEY }}
      KNODR_CREDENTIAL_GROK: ${{ secrets.MAR_XAI_API_KEY }}

    steps:
      - uses: actions/checkout@v6
        with:
          ref: ${{ github.event.pull_request.head.sha }}
          fetch-depth: 0
          persist-credentials: false

      - uses: actions/checkout@v6
        with:
          ref: ${{ github.event.pull_request.base.sha }}
          path: _mar-tooling
          sparse-checkout: ci/mar
          persist-credentials: false

      - name: Run Knodr multi-agent review
        shell: pwsh
        run: |
          $toolRoot = "$env:GITHUB_WORKSPACE\_mar-tooling\ci\mar"
          & knodr run "$toolRoot\ci-multi-agent-review.knodrflow" `
            --var "repo=$env:GITHUB_WORKSPACE" `
            --var "diffPath=$env:GITHUB_WORKSPACE\.mar-tmp\diff.txt" `
            --var "comments=$env:MAR_COMMENTS" `
            --var "rendererPath=$toolRoot\render.py" `
            --timeout 30m --log-level Debug --output-format json

      - uses: github/codeql-action/upload-sarif@v4
        with:
          sarif_file: .mar-out/mar-results.sarif

      - uses: actions/upload-artifact@v7
        with:
          path: .mar-out/**

continue-on-error: true makes the review advisory, matching a CI job that must never block the build. Remove it when Critical findings or panel failures should block merging. SARIF upload needs security-events: write; for private repositories, GitHub Code Security must be enabled. GitHub’s SARIF guide documents the permission and upload-sarif@v4 integration.

08 / SECURITY & OPERATIONS

Treat reviewed code as untrusted input

No fork PRs

The template skips pull requests whose head repository differs from the target repository. GitHub does not pass Actions secrets to ordinary fork PR workflows, and a self-hosted runner should not process arbitrary fork code.

Protected tooling

The flow and renderer come from the base revision. Add CODEOWNERS and environment approval so a workflow edit cannot receive provider keys unnoticed.

Dedicated runner

Do not share this machine with production credentials or broad internal-network access. Self-hosted runners persist between jobs unless you make them ephemeral.

Raw logs are sensitive

agents-raw.log can contain code excerpts and model diagnostics. Keep artifact retention short and repository access restricted.

Bound time and cost

The workflow caps the job at 40 minutes and the Knodr run at 30 minutes. Provider timeouts are 15 minutes per reviewer and 10 minutes for consensus.

Pin or maintain actions

The example follows current major tags. Security-sensitive organizations can pin full commit SHAs and update them with Dependabot.

Do not switch this workflow to pull_request_target and then check out PR code. That combination can expose secrets with elevated base-branch privileges. Keep pull_request, the same-repository guard, protected environment approval, and a restricted runner.