Operations Runbook

Reference guide for running the LightChallenge off-chain pipeline in development and production.

Pipeline Architecture

Provider APIs (Strava, Fitbit, FACEIT, OpenDota, Riot) + Manual uploads (Apple Health, Garmin, Google Fit)
        │
        ▼ evidenceCollector (polls linked_accounts)
public.evidence
        │
        ▼ evidenceEvaluator
public.verdicts
        │
        ▼ challengeDispatcher (gates on active + verdict)
public.aivm_jobs (queued)
        │
        ▼ challengeWorker
AIVMInferenceV2.requestInferenceV2() [on-chain]
        │
        │ [Lightchain native workers: commit → reveal → attest until quorum]
        │
        ▼ InferenceFinalized event
aivmIndexer → attemptFinalizationBridge()
        → ChallengePay.submitProofFor() → ChallengePayAivmPoiVerifier.verify() + ChallengePay.finalize() [on-chain]
        │
public.aivm_jobs (done)
public.challenges (status = Finalized)

Claims path (parallel):
ChallengePay *Claimed events → claimsIndexer → public.claims

Status sync (parallel):
ChallengePay status events → statusIndexer → public.challenges.status

Prerequisites

• Node.js 22 + npm install at repo root
• webapp/.env.local with all required variables set (see .env.example)
• LightChain testnet RPC accessible (https://light-testnet-rpc.lightchain.ai)
• PostgreSQL database (Neon or local) with DATABASE_URL set

1. Database Migration

Run once before starting any workers, and after adding new migration files:

npx tsx db/migrate.ts

Applied migrations are tracked in public.schema_migrations. Re-running is safe — already-applied files are skipped.

Current migrations: 001_evidence_verdicts through 019_seed_demo_challenges. See db/DATABASE.md for full schema documentation.

2. Identity Seed (one-time)

Migrate legacy offchain/.state/identity_bindings.json → public.identity_bindings:

npx tsx db/seed_identity.ts

Safe to re-run (upsert on conflict). Only needed if migrating from a pre-DB version of the system.

3. Evidence Collector

Polls public.linked_accounts for registered provider connections, fetches recent activity or match data from provider APIs, and stores normalized records in public.evidence.

npx tsx offchain/workers/evidenceCollector.ts

Auto-collection providers (polled by evidence collector):

• strava — OAuth, token auto-refresh, fetches activities (running, cycling, etc.)
• fitbit — OAuth, token auto-refresh, fetches daily steps + activity logs
• opendota — free API, fetches Dota 2 matches by Steam32 ID
• riot — API key required, fetches LoL matches by PUUID
• faceit — API key required, fetches CS2 matches by Steam64→FACEIT player ID

Upload-only providers (skipped by evidence collector — evidence via file upload):

• apple — no API; users upload Apple Health ZIP export
• garmin — no public API (enterprise-only); users upload TCX/GPX/JSON export
• googlefit — API deprecated by Google in 2025; users upload Google Takeout JSON

Writes to: public.evidence

Note: The collector skips insertion when the incoming evidence_hash matches the previous row for the same (challenge_id, subject, provider) — no duplicate rows.

4. Evidence Evaluator

Polls public.evidence for rows that have no corresponding verdict, runs the appropriate evaluator per provider, and upserts the result to public.verdicts.

npx tsx offchain/workers/evidenceEvaluator.ts

Evaluators:

• fitnessEvaluator — for apple, garmin, strava, fitbit, googlefit
• gamingEvaluator — for opendota, riot, steam, faceit

Writes to: public.verdicts

Safety: Unknown providers produce a pass: false verdict to drain the queue rather than block it.

Must run before: challengeDispatcher (dispatcher gates on verdict existence).

5. Challenge Dispatcher

Scans public.challenges for active challenges that have a matching passing verdict and queues them into public.aivm_jobs.

The dispatcher supports two evaluation modes:

Threshold mode (default): dispatches as soon as the challenge subject has a passing verdict. This is the standard pass/fail flow.

Competitive mode: for challenges with rule.mode === "competitive". The dispatcher waits until the proof deadline passes (all evidence is in), then:

1. Fetches all verdicts for the challenge with score IS NOT NULL
2. Ranks participants by score descending
3. Breaks ties by earliest evidence submission (created_at ascending)
4. Marks the top-N participants as winners (pass=true), the rest as losers (pass=false)
5. Enqueues a single AIVM job (normal flow from here)
6. The AIVM indexer calls submitProofFor for each passing participant during finalization

No new environment variables are required for competitive mode. Detection is automatic based on the challenge’s proof.params.rule.mode field.

npx tsx offchain/dispatchers/challengeDispatcher.ts

Reads from: public.challenges (status = active), public.verdicts (pass = true for threshold; score IS NOT NULL for competitive)

Writes to: public.aivm_jobs (status = queued), public.verdicts (pass and metadata updated during competitive ranking)

Idempotent: already-queued challenges are not re-queued (UNIQUE constraint on challenge_id).

6. Challenge Worker

Dequeues jobs from public.aivm_jobs and submits them to the Lightchain AIVM network via requestInferenceV2. Sets job status to submitted. Does NOT attempt commit/reveal/attest — those are performed autonomously by Lightchain workers and validators.

npx tsx offchain/workers/challengeWorker.ts

Required env vars: LCAI_WORKER_PK, AIVM_INFERENCE_V2_ADDRESS, AIVM_TASK_REGISTRY_ADDRESS

Reads from: public.aivm_jobs (status = queued)

Writes to: public.aivm_jobs (status → submitted or failed), public.challenges (binding recorded)

7. AIVM Indexer

Watches events from the Lightchain AIVMInferenceV2 contract and updates the DB as the Lightchain network processes submitted tasks. When InferenceFinalized is observed, triggers the finalization bridge to call submitProofFor and ChallengePay.finalize().

npx tsx offchain/indexers/aivmIndexer.ts

Required env vars: AIVM_INFERENCE_V2_ADDRESS, NEXT_PUBLIC_RPC_URL

Event → DB action:

Checkpoint: last processed block stored in public.indexer_state under key last_aivm_block.

Finalization bridge: idempotent — will not retry if proof.finalizationAttempted is already set. If finalize() reverts (e.g. BeforeDeadline), the error is logged and processing continues.

8. Claims Indexer

Watches ChallengePay claim events and Treasury ClaimedETH events, persisting each claim into public.claims. This is the secondary/hardening source of truth for claimed state. The UI also writes to public.claims immediately after a successful transaction (primary path).

npx tsx offchain/indexers/claimsIndexer.ts

Events indexed: WinnerClaimed, LoserClaimed, RefundClaimed, ClaimedETH

Checkpoint: last processed block stored in public.indexer_state under key last_claims_block.

Writes to: public.claims (upsert on conflict — idempotent)

9. Status Indexer

Watches ChallengePay status-changing events and keeps public.challenges.status aligned with on-chain state. This closes the gap where only aivmIndexer wrote Finalized — canceled challenges now sync automatically.

npx tsx offchain/indexers/statusIndexer.ts

Events indexed: Finalized, Canceled

Checkpoint: last processed block stored in public.indexer_state under key last_status_block.

Writes to: public.challenges.status (idempotent — only updates when status differs)

10. Recommended Startup Order

Run each in a separate terminal or process manager (e.g. PM2):

# Step 1 — DB migration (run once before starting workers)
npx tsx db/migrate.ts
 
# Step 2 — Workers and indexers (run persistently)
npx tsx offchain/workers/evidenceCollector.ts       # provider APIs → public.evidence
npx tsx offchain/workers/evidenceEvaluator.ts       # public.evidence → public.verdicts
npx tsx offchain/dispatchers/challengeDispatcher.ts # verdicts → public.aivm_jobs queue
npx tsx offchain/workers/challengeWorker.ts         # aivm_jobs → requestInferenceV2 on-chain
npx tsx offchain/indexers/aivmIndexer.ts            # Lightchain AIVM events → finalize
npx tsx offchain/indexers/claimsIndexer.ts          # ChallengePay claim events → public.claims
npx tsx offchain/indexers/statusIndexer.ts         # ChallengePay status events → challenges.status
 
# Webapp (can run at any time independently)
cd webapp && npm run dev

Dependency ordering: evidenceEvaluator must be running before challengeDispatcher, which must be running before challengeWorker. The indexers are independent.

11. Deployed Contract Addresses (Testnet)

All addresses also stored in webapp/public/deployments/lightchain.json.

Archived contracts (deployed on-chain historically, not part of active product)

Previous contract addresses (superseded)

Roles and admin (current)

Post-deploy checklist (after any ChallengePay redeploy)

1. Accept admin: Call ChallengePay.acceptAdmin() from the ADMIN_PRIVATE_KEY wallet
2. Grant OPERATOR_ROLE: Call Treasury.grantRole(OPERATOR_ROLE, <new ChallengePay address>) from the Treasury admin wallet
3. Register dispatcher: Register the worker wallet on ChallengeTaskRegistry (automated by scripts/deployPoiVerifierV2.ts if LCAI_WORKER_PK is set)
4. Update env: Set CHALLENGEPAY_ADDRESS / NEXT_PUBLIC_CHALLENGEPAY_ADDR in .env.local to the new address
5. Rebuild webapp: cd webapp && npm run build to pick up new ABI and addresses

Dispatcher setup (one-time after deploy)

When ChallengeTaskRegistry is redeployed, register the worker wallet as a dispatcher so it can call recordBinding(). This is automated in scripts/deployPoiVerifierV2.ts if LCAI_WORKER_PK is set during deploy.

Testnet AIVM workers

The Lightchain testnet has active native workers that process inference requests for any model. Native workers commit + reveal automatically; native validators attest until quorum, emitting InferenceFinalized. Our aivmIndexer then drives ChallengePay.finalize().

Verified: requests for apple_health.steps@1 and other LightChallenge model IDs are picked up and finalized by native workers — no local simulation needed.

To drive the pipeline locally (e.g. no active workers for a given model):

PRIVATE_KEY=0x... CHALLENGE_TASK_REGISTRY_ADDRESS=0x... \
  CHALLENGEPAY_AIVM_POI_VERIFIER_ADDRESS=0x... DATABASE_URL=... \
  LIGHTCHAIN_RPC=https://light-testnet-rpc.lightchain.ai \
  npx tsx scripts/_e2e_simulate_aivm.ts

This script is testnet-only. Do not use in production.

12. Ops Scripts Reference

The scripts below are located in scripts/ops/. Each has a clear lifecycle classification.

seedStatusIndexer.ts — one-time, first deploy

Run once before starting statusIndexer on a fresh or newly deployed environment. Sets last_status_block checkpoint to current_block - LOOKBACK so the indexer does not scan from genesis. Safe to re-run (uses ON CONFLICT DO UPDATE).

npx tsx scripts/ops/seedStatusIndexer.ts
# Optional: override lookback (default 50000 blocks ≈ ~12hrs on this chain)
STATUS_INDEXER_SEED_LOOKBACK=100000 npx tsx scripts/ops/seedStatusIndexer.ts

After seeding, start the indexer and it will backfill only recent blocks.

backfillChainOutcome.ts — on-demand, idempotent

Queries challenges WHERE status='Finalized' AND chain_outcome IS NULL and reads the outcome field directly from ChallengePay.getChallenge() on-chain for each one. Run after a new deployment on an existing DB, or if the statusIndexer missed events.

npx tsx scripts/ops/backfillChainOutcome.ts

Safe to run any number of times — only updates rows where chain_outcome IS NULL.

cancelTerminalJobs.ts — on-demand, idempotent

Cancels any queued/failed/processing aivm_jobs rows whose challenges have reached a terminal state (Finalized/Canceled). The challengeDispatcher handles this automatically each poll cycle going forward — this script is only needed for pre-existing stale rows on a fresh deployment.

npx tsx scripts/ops/cancelTerminalJobs.ts

Safe to run any number of times — only updates rows that match the stale condition.

Run AIVM job for a specific challenge manually

DATABASE_URL=... LCAI_WORKER_PK=0x... AIVM_INFERENCE_V2_ADDRESS=0x... \
  npx tsx offchain/runners/runChallengePayAivmJob.ts <challengeId>

Sign an AIVM proof manually (fitness / file-based)

LIGHTCHAIN_RPC=https://light-testnet-rpc.lightchain.ai \
PRIVATE_KEY=0x... \
AIVM_VERIFIER=0x... \
CHALLENGE_ID=42 \
SUBJECT=0x... \
EXPECTED_CALLER=0x... \
RULE_JSON=data/examples/rule_10k_3x_week.json \
ACTIVITIES_JSON=data/examples/activities_run.json \
  npx hardhat run scripts/ops/signAivmProof.ts --network lightchain

backfillRegistry.ts — on-demand, idempotent

Finds challenges in public.challenges where registry_status != 'success' and attempts MetadataRegistry.ownerSet() for each. The write-once contract policy means AlreadySet reverts are treated as success (the URI is already on-chain).

DATABASE_URL=... BASE_URL=https://app.lightchallenge.ai \
  npx hardhat run scripts/ops/backfillRegistry.ts --network lightchain

Safe to run any number of times. The signer must be the MetadataRegistry owner.

13. MetadataRegistry Architecture

Source model

Write policy

• ownerSet() — write-once. Reverts with AlreadySet if URI already exists.
• ownerForceSet() — explicit overwrite for corrections. Emits distinct MetadataForceSet event with old+new URI.
• ownerClear() — removes URI. Emits MetadataCleared.
• All writes are owner-only. The owner is the system/admin wallet (METADATA_REGISTRY_KEY).
• Creators do not write to MetadataRegistry directly.

Active flow

1. Frontend creates challenge on-chain (ChallengePay.createChallenge)
2. Frontend calls POST /api/challenges → DB upsert (public.challenges)
3. API route attempts MetadataRegistry.ownerSet(challengePay, id, uri) using METADATA_REGISTRY_KEY
4. Result stored in public.challenges.registry_status / registry_tx_hash / registry_error
5. If write fails → registry_status = 'failed' → backfillRegistry.ts retries later

Failure policy

Soft failure with retry. Challenge creation never fails because of a registry write failure. The DB is the primary metadata store for the product. The on-chain registry is for external discovery. Failed writes are logged in registry_status and retried via backfillRegistry.ts.

DB tracking columns

Monitoring

• Query: SELECT id, registry_status, registry_error FROM public.challenges WHERE registry_status IN ('pending', 'failed');
• Any MetadataForceSet event on-chain indicates an admin correction — investigate.
• MetadataCleared events indicate an admin removal.

14. DB Quick-Checks

-- Pending evidence (no verdict yet)
SELECT e.id, e.challenge_id, e.subject, e.provider, e.created_at
FROM   public.evidence e
LEFT   JOIN public.verdicts v
         ON v.challenge_id = e.challenge_id
        AND lower(v.subject) = lower(e.subject)
WHERE  v.id IS NULL
ORDER  BY e.created_at;
 
-- Latest verdicts
SELECT challenge_id, subject, pass, evaluator, updated_at
FROM   public.verdicts
ORDER  BY updated_at DESC
LIMIT  20;
 
-- All AIVM job statuses
SELECT challenge_id, status, attempts, task_id, last_error, updated_at
FROM   public.aivm_jobs
ORDER  BY updated_at DESC
LIMIT  30;
 
-- Jobs stuck waiting on Lightchain network
SELECT challenge_id, status, task_id, updated_at
FROM   public.aivm_jobs
WHERE  status IN ('submitted', 'committed', 'revealed')
ORDER  BY updated_at ASC;
 
-- Recent claims
SELECT challenge_id, subject, claim_type, amount_wei, source, claimed_at
FROM   public.claims
ORDER  BY claimed_at DESC
LIMIT  20;
 
-- Indexer checkpoints
SELECT key, value FROM public.indexer_state ORDER BY key;
 
-- Achievement mints
SELECT token_id, challenge_id, recipient, achievement_type, minted_at
FROM   public.achievement_mints
ORDER  BY minted_at DESC
LIMIT  20;
 
-- Reputation leaderboard
SELECT subject, points, level, completions, victories
FROM   public.reputation
ORDER  BY points DESC
LIMIT  20;

15. Troubleshooting

Evidence evaluator not creating verdicts

1. Check public.evidence has rows: SELECT count(*) FROM public.evidence;
2. Check evaluator logs for unknown provider errors — these produce pass: false verdicts immediately
3. Verify DATABASE_URL is set and the evaluator process can reach the DB

Jobs stuck in queued (never submitted)

1. Check challengeWorker is running
2. Verify LCAI_WORKER_PK wallet has sufficient LCAI for gas
3. Check AIVM_INFERENCE_V2_ADDRESS and AIVM_TASK_REGISTRY_ADDRESS are set correctly
4. Check AIVM_REQUEST_FEE_WEI — if set too low, requestInferenceV2 may revert

Jobs stuck in submitted/committed/revealed (Lightchain not finalizing)

1. Confirm the aivmIndexer is running and checkpointing (query indexer_state)
2. Verify AIVM_INFERENCE_V2_ADDRESS in .env.local matches the live address
3. Check if the AIVM request deadline has expired (~1hr on testnet) — expired requests cannot be finalized. A new request must be submitted.
4. The testnet has active native workers — check whether the task_id appears in AIVM contract events

InferenceFinalized observed but ChallengePay.finalize() reverted with BeforeDeadline

This is normal when the challenge finalize window has not opened yet. The indexer logs the revert and continues. Finalization will succeed when the challenge period ends.

Claims indexer not persisting claims

1. Verify CHALLENGEPAY_ADDRESS and NEXT_PUBLIC_TREASURY_ADDR are set
2. Query indexer_state to check the last_claims_block checkpoint value
3. If checkpoint is far behind current block, the indexer may need time to catch up

AIVM request deadlines on testnet

AIVM requests expire approximately 1 hour after creation on the Lightchain testnet. If an old request has an expired deadline, the finalization bridge will revert. Create a fresh request by re-running the challenge worker job.

16. API Authentication Headers

Webapp API routes that perform writes or return user-specific data require the following authentication headers. The frontend sends these automatically via middleware; external callers (scripts, monitoring) must set them manually.

The server verifies that x-lc-signature is a valid EIP-191 signature of the x-lc-timestamp value by x-lc-address. Stale timestamps (older than 5 minutes) are rejected.

Admin endpoints (e.g. /api/admin/*) additionally require the ADMIN_KEY env var to be set on the server, and the calling address must match the configured admin wallet.

17. Dispatcher Role for submitProofFor

The aivmIndexer finalization bridge calls ChallengePay.submitProofFor() to submit AIVM proofs on behalf of challenge participants. This function is restricted to addresses with the dispatcher role on ChallengePay.

The dispatcher is set via ChallengePay.setDispatcher(address) by the contract admin. The LCAI_FINALIZE_PK wallet must be registered as the dispatcher before the indexer can finalize challenges.

If submitProofFor reverts with an authorization error, check:

1. The LCAI_FINALIZE_PK address is the current dispatcher: query ChallengePay.dispatcher()
2. The admin has called setDispatcher() after the most recent ChallengePay deployment
3. The finalization wallet has sufficient LCAI for gas

See DEPLOY.md Step 4 for setup instructions.

18. Deprecated Columns

public.challenges.aivm_request_started and aivm_request_started_at have no writers or readers in the codebase. They are superseded by public.aivm_jobs.status. Safe to ignore; will be removed in a future migration.

19. Legacy Compatibility

Active product architecture: Lightchain AIVM + PoI (Proof of Inference) via ChallengePayAivmPoiVerifier. No alternate signer-based or manual verifier is part of the active product.

Active verification path: evidence → verdict → AIVM request → Lightchain commit/reveal/attest → InferenceFinalized → ChallengePay.submitProofFor() → ChallengePayAivmPoiVerifier.verify() → ChallengePay.finalize().

Contract classification:

• Core product: ChallengePay, Treasury, ChallengePayAivmPoiVerifier, ChallengeTaskRegistry
• Core product: ChallengeAchievement (soulbound ERC-721 + ERC-5192; read-only dependency on ChallengePay)
• Core product: MetadataRegistry (active on-chain metadata pointer layer; write-once by default, system-managed)
• Admin-only optional: EventChallengeRouter (multi-outcome event routing; admin scripts only, not on user-facing product path)
• Dormant infrastructure: TrustedForwarder (EIP-2771 gasless relay; deployed but not activated, relay disabled by default via RELAY_ENABLED)
• Archived: AivmProofVerifier (Path A EIP-712 trusted-signer; moved to .attic/contracts_archive/, admin scripts archived, not part of active product)

The following legacy artifacts exist in the codebase for backward compatibility but are not part of the active product:

Policy: Do not use legacy ZK/Plonk concepts for new models, admin UX, product flows, or documentation unless explicitly reactivated by a product decision.

20. Health Checks & Monitoring

Process health

All 7 worker/indexer processes log their service name on every poll cycle. A healthy system produces periodic log output. If a process goes silent, it has crashed or hung.

Recommended: Run all workers under a process manager (PM2, systemd, Docker) that auto-restarts on crash. All workers handle SIGINT/SIGTERM gracefully and drain DB connections before exit.

Indexer lag detection

-- Check indexer checkpoint vs chain head
-- If (chain_head - checkpoint) > 100 blocks, the indexer is lagging
SELECT key, value::bigint AS last_block FROM public.indexer_state ORDER BY key;

Compare against chain head: cast block-number --rpc-url $LCAI_RPC

Expected lag: ≤ 12 blocks (CONFIRMATION_BLOCKS) + MAX_BLOCK_RANGE (2000) in worst case.

Job pipeline health

-- Jobs stuck for >1 hour (needs investigation)
SELECT challenge_id, status, attempts, updated_at, last_error
FROM   public.aivm_jobs
WHERE  status NOT IN ('done', 'canceled', 'dead')
  AND  updated_at < now() - interval '1 hour'
ORDER  BY updated_at ASC;
 
-- Dead jobs (exhausted all retries)
SELECT challenge_id, last_error, attempts, updated_at
FROM   public.aivm_jobs WHERE status = 'dead'
ORDER  BY updated_at DESC LIMIT 20;

Wallet balance monitoring

Worker and finalizer wallets must maintain LCAI balance for gas + AIVM request fees.

# Check worker wallet balance
cast balance $LCAI_WORKER_ADDRESS --rpc-url $LCAI_RPC
 
# Check finalizer wallet balance (if separate)
cast balance $LCAI_FINALIZE_ADDRESS --rpc-url $LCAI_RPC

Alert threshold: < 0.1 LCAI (100 finalize transactions at ~0.001 LCAI each).

Evidence pipeline throughput

-- Evidence waiting for evaluation (should stay near 0)
SELECT count(*) AS pending_evidence
FROM   public.evidence e
LEFT   JOIN public.verdicts v
         ON v.challenge_id = e.challenge_id
        AND lower(v.subject) = lower(e.subject)
WHERE  v.id IS NULL;

21. Failure Recovery Procedures

Worker crash and restart

All workers are safe to restart at any time. On restart:

• evidenceCollector: resumes polling; no checkpoint needed (stateless re-fetch)
• evidenceEvaluator: picks up unevaluated evidence from DB (idempotent)
• challengeDispatcher: re-scans eligible challenges; ON CONFLICT prevents duplicates
• challengeWorker: claims jobs via FOR UPDATE SKIP LOCKED; in-flight jobs remain in processing and will be retried after timeout
• aivmIndexer: resumes from last_aivm_block checkpoint in DB
• statusIndexer: resumes from last_status_block checkpoint
• claimsIndexer: resumes from last_claims_block checkpoint

RPC failure

If the Lightchain RPC is unreachable:

• Indexers log errors and continue polling (next cycle retries)
• Worker’s AIVM request submission fails → job marked failed, retried on next cycle
• Finalization bridge failures are logged without setting finalizationAttempted, allowing automatic retry

Database failure

• All workers crash on fatal DB errors (correct behavior — process manager restarts them)
• pg.Pool handles transient connection drops automatically — no manual reconnection needed
• On DB recovery, all workers resume normal operation immediately

Stuck jobs recovery

# Reset stuck processing jobs back to queued (safe — FOR UPDATE SKIP LOCKED prevents conflicts)
UPDATE public.aivm_jobs
SET    status = 'queued', updated_at = now()
WHERE  status = 'processing'
  AND  updated_at < now() - interval '30 minutes';
 
# Cancel jobs for challenges that are already finalized
npx tsx scripts/ops/cancelTerminalJobs.ts

Reorg recovery

If a reorg deeper than CONFIRMATION_BLOCKS (12) is suspected:

1. Check indexer_state checkpoint values
2. Run scripts/ops/reconcileDemo.ts to reconcile DB state with on-chain state
3. Run scripts/ops/backfillChainOutcome.ts to fix any stale chain_outcome values

22. Production Deployment Checklist

Pre-deployment

• All env vars from .env.example sections 1-5 are set in webapp/.env.local
• DATABASE_URL points to production PostgreSQL with SSL
• LCAI_WORKER_PK and LCAI_FINALIZE_PK wallets are funded with LCAI
• ADMIN_KEY is set (random secret string, ≥32 chars)
• OAUTH_ENCRYPTION_KEY is set (openssl rand -hex 32)
• NEXT_PUBLIC_RPC_URL and LCAI_RPC point to a reliable RPC endpoint
• Contract addresses in webapp/public/deployments/lightchain.json match live deployment

Database

• npx tsx db/migrate.ts completes without errors
• npx tsx scripts/ops/seedStatusIndexer.ts (first deploy only)
• Verify public.schema_migrations shows all migrations applied

Contracts

• ChallengePay.admin() returns the expected admin address
• Treasury.hasRole(OPERATOR_ROLE, <ChallengePay>) returns true
• ChallengePay.dispatchers(<finalize_wallet>) returns true
• ChallengePayAivmPoiVerifier is set as verifier on relevant challenges

Workers

• All 7 workers start without errors
• evidenceCollector logs provider accounts on first poll
• aivmIndexer logs finalization bridge status (ENABLED/DISABLED)
• statusIndexer and claimsIndexer log their ChallengePay addresses

Webapp

• cd webapp && npm run build succeeds
• /explore page loads and shows challenges
• /challenge/<id> page loads for a known challenge
• Admin panel at /admin authenticates correctly with ADMIN_KEY

Post-deployment

• Monitor indexer lag (section 20) for first 10 minutes
• Verify wallet balances are sufficient
• Run a test challenge through the full pipeline (create → evidence → verdict → AIVM → finalize → claim)

23. Competitive Challenges

Competitive challenges rank participants by score rather than applying a pass/fail threshold. The on-chain contract (ChallengePay) still uses binary outcome (winner/loser), so competitive ranking is resolved off-chain before finalization.

Flow

Participants submit evidence during proof window
        │
        ▼ evidenceEvaluator
public.verdicts (score + metadata populated)
        │
        │ [proof deadline passes]
        │
        ▼ challengeDispatcher (competitive mode)
Ranks all verdicts by score DESC, tie-breaks by earliest created_at
Top-N marked pass=true, rest pass=false
        │
        ▼ AIVM job queued → challengeWorker → requestInferenceV2
        │
        │ [Lightchain network: commit → reveal → attest]
        │
        ▼ InferenceFinalized → aivmIndexer
submitProofFor() for each winner → ChallengePay.finalize()

Migration

• 018_verdicts_score_competitive.sql — adds score (numeric) and metadata (jsonb) columns to public.verdicts. The score column stores the evaluated metric value used for competitive ranking (e.g. total steps, total kills). The metadata column stores structured evaluation details (e.g. match IDs, per-day breakdowns).
• 019_seed_demo_challenges.sql — seeds demo competitive and threshold challenges for testing.

Apply with the standard migration runner:

npx tsx db/migrate.ts

Detection

A challenge is competitive if its rule config contains mode: "competitive". The dispatcher checks the following paths in order:

1. proof.params.rule.mode
2. proof.params.mode
3. params.rule.mode
4. params.mode

The topN value (number of winners) is read from the same paths. Defaults to 1 if not specified.

Tie-breaking

When multiple participants have the same score, the tie is broken by created_at on the verdict row — the participant who submitted evidence earliest wins. This is deterministic and auditable.

DB queries for competitive challenges

-- All competitive challenges
SELECT id, subject, status,
       proof->'params'->'rule'->>'mode' AS mode,
       proof->'params'->'rule'->>'topN' AS topN
FROM   public.challenges
WHERE  proof->'params'->'rule'->>'mode' = 'competitive';
 
-- Ranking for a specific competitive challenge
SELECT subject, score, pass, metadata, created_at
FROM   public.verdicts
WHERE  challenge_id = <ID>
ORDER  BY score DESC NULLS LAST, created_at ASC;