SourcAI — Guardrails × Judging Criteria

SourcAI — Guardrails × Judging Criteria: Pitch Reference for Freddy

Purpose: This document maps EVERY judging criterion to specific guardrails, features, and real examples in our agent. Use this to build pitch slides and answer judge questions with precision. Key argument: Our 5-minute processing time is NOT a weakness — it’s the audit trail depth that makes this production-ready, not a toy demo.

THE 5 CRITERIA AND HOW WE WIN EACH ONE

1. FEASIBILITY (25%) — “With a bit more work, would this become production?”

Why we win: We built what Chain IQ would actually deploy. Not a ChatGPT wrapper. A procurement-specific agent with 145 policy rules, 8 escalation triggers, and deterministic guardrails that enforce compliance WITHOUT human supervision.

Specific proof points:

Pitch line: “Every guardrail in SourcAI exists because Chain IQ’s 145 procurement policies demand it. We didn’t invent rules — we encoded yours.”

2. ROBUSTNESS & ESCALATION LOGIC (25%) — “Handle contradictions, rule violations, uncertainty”

Why we win: This is our strongest criterion. We have 12 validation checks, 8 escalation rules, 10 category rules, 8 geography rules, and 5 restricted supplier checks — all enforced consistently.

The 8 Escalation Rules (from policies.json):

Escalation routing is CATEGORY-SPECIFIC — IT escalations go to IT lead, facilities to facilities lead. Never generic. Mentors explicitly validated this.

Real contradiction handling (from actual agent outputs):

1. Budget vs. reality contradiction (REQ-20260319-c9f40c — Smartphones):

   • Requester asked for 30 smartphones at EUR 22,800
   • Agent discovered minimum cost is EUR 26,481 (16.1% shortfall)
   • Instead of failing, generated 4 resolution paths:
     • A: Reduce to 25 units (stays under AT-001, Business-only approval)
     • B: Reduce to 22 Apple units (stays under AT-001, higher quality)
     • C: Revise budget to EUR 26,481 (triggers AT-002, needs Procurement co-approval)
     • D: Revise budget to EUR 30,030 for Apple (highest quality, triggers AT-002)
   • This is what procurement professionals actually do. They don’t say “error”; they say “here are your options.”

2. Insufficient supplier pool (REQ-000001 — Consulting):

   • AT-003 requires 3 quotes, but only 2 suppliers serve Spain for IT PM Services
   • Agent didn’t fake a third supplier — flagged deviation, escalated to Head of Category
   • Documented WHY (Deloitte: no Spain coverage; Infosys: no EU coverage)

3. Past deadline (REQ-000042 — Cloud Compute):

   • Deadline was 2026-03-15, request processed 2026-03-19 (4 days late)
   • Agent didn’t ignore it — flagged as HIGH severity, calculated earliest fulfillment (AWS: 2026-04-01, 17 days late)
   • Suggested incumbent OVHcloud as bridge solution for immediate capacity

4. Preferred supplier can’t serve region (REQ-000002 — Cloud Compute):

   • Requester wanted Azure Enterprise; Swiss Sovereign Cloud (incumbent) doesn’t serve Netherlands
   • Agent excluded incumbent with documented reason, ranked Azure #4 but recommended it anyway with explicit deviation documentation

Binary vs. ranking criteria: The agent distinguishes hard gates (ISO certification required = knockout) from weighted scoring (price, quality, risk/ESG = continuum). Mentors confirmed this is exactly right.

Pitch line: “When a request contradicts itself — budget too low, deadline passed, preferred supplier restricted — SourcAI doesn’t crash. It generates resolution options with trade-off analysis. That’s what a senior procurement specialist does. We automated that judgment.”

3. CREATIVITY (20%) — “Copy paste from what OpenAI does today will fail”

Why we win: We made 5 architectural choices that no other team will have:

1. File-based workspace (not a database)

   • Each request gets its own directory: REQ-{id}/input.md, output.json, status.json, trace.jsonl
   • The agent reads/writes files like a human analyst working through a folder
   • Tim Paridaens (CTO) validated: “Mono-agent with file-based workspace = correct. Multi-agent orchestration creates knowledge management problems.”
   • Why creative: Every other team will use a database or chat memory. We use the filesystem as the knowledge graph.

2. Deterministic guardrails wrapping an LLM core

   • The LLM handles understanding messy text and generating recommendations
   • But thresholds, track classification, and policy enforcement are DETERMINISTIC Python code
   • _reclassify_track() can override the LLM’s classification based on hard budget math
   • Why creative: We don’t trust the AI blindly. We cage it in procurement rules.

3. 3-track system with configurable autonomy

   • Marketplace (fully autonomous) → Technical (agent + approval) → Strategic (agent assists)
   • 80% of requests are tail spend (Marketplace) — these can be processed WITHOUT any human
   • 20% are high-value — these get full audit trail with human approval gates
   • Why creative: We don’t try to automate everything. We automate what SHOULD be automated and escalate what shouldn’t.

4. Historical concentration detection

   • Agent analyzes 590 historical awards to detect single-brand loyalty patterns
   • Example: Apple has 100% of smartphone awards (9/9). Agent flags this as audit risk and recommends Samsung for competitive benchmarking.
   • Why creative: This catches procurement bias that humans miss because they’re too busy.

5. Savings framing as Chain IQ revenue

   • Every recommendation includes savings_vs_most_expensive — this IS Chain IQ’s revenue
   • Agent explicitly writes: “This is the documented savings Chain IQ should record as value delivered”
   • Why creative: We didn’t just build a tool. We built something that directly feeds Chain IQ’s business model.

CTO quote to reference: “The superpower is not the model. The superpower is what happens when capable, reasonable systems are grounded in structured data.” — Daniel Ringsma, Head of AI

Pitch line: “We didn’t build an AI that replaces procurement. We built an AI that thinks like procurement — with 145 rules it can’t break, 8 escalation paths it must follow, and an audit trail that proves every decision.”

4. REACHABILITY (20%) — “If we change the datasets, would it survive?”

Why we win: Our agent is dataset-agnostic. The pipeline reads from CSV/JSON files. Change the suppliers, change the policies, change the categories — the agent adapts.

Proof of transferability:

The agent reads data files at runtime. Nothing is hardcoded. The system prompt in workspace/CLAUDE.md references data generically — “read suppliers.csv”, not “look for Dell in row 12.”

Multi-language support: Agent detects request language (EN, DE, FR, ES, PT, JA) and generates buyer reports in that language. Chain IQ operates in 49 countries.

Multi-currency: Pricing handles EUR, USD, CHF, GBP natively (from pricing.csv). Conversion is handled per-request.

Geography rules (8 regions): CH, DE, FR, ES, Americas, APAC, MEA, LATAM — each with specific compliance requirements (GDPR, LGPD, POPIA, MAS, etc.)

Real proof: We process 304 different requests across IT (laptops, cloud, smartphones), Facilities (furniture), Professional Services (consulting, cybersecurity), and Marketing (SEM, influencer) — all with the same pipeline.

Pitch line: “Give us a new client’s supplier list and policy handbook on Monday. By Tuesday, SourcAI is processing their requests. No retraining. No fine-tuning. Just swap the data files.”

5. VISUAL DESIGN (10%) — “Clarity of comparison view and decision explanation”

What we have:

• Audit report (HTML/PDF): Professional report with all 7 pipeline steps visible, policy citations expandable, scoring breakdowns visualized
• Buyer report: Client-facing recommendation in detected language with savings analysis
• Dashboard: Real-time SSE streaming shows agent working through each step live
• Comparison table: Side-by-side supplier scoring with weighted components visible

Mentor feedback applied:

• PDF output is the HERO (primary action button), not raw JSON
• Policy references are expandable/clickable (e.g., click “AT-003” → see full rule text)
• Reasoning is RESURFACED in the UI, not hidden in backend logs

THE 5-MINUTE ARGUMENT: WHY SPEED ≠ QUALITY

The question judges will ask:

“Your agent takes 5 minutes. Can’t you make it faster?”

The answer (for Freddy):

“Yes, we could make it faster. And we’d be worse.”

Here’s why:

What happens in those 5 minutes:

Total: ~5 minutes of actual procurement analysis.

A human does this in 2 HOURS (mentor-confirmed average). We’re already 24x faster.

The 1-2 minute alternative would require cutting:

• ❌ Historical concentration analysis (590 awards) — judges asked for this
• ❌ Multi-source policy cross-referencing (policies.json + suppliers.csv + pricing.csv) — audit trail breaks
• ❌ Deviation documentation with alternatives — just picks the cheapest, no reasoning
• ❌ Draft clarification messages for escalations — just says “needs human review”
• ❌ Savings framing (vs cheapest, vs most expensive, vs budget) — loses Chain IQ revenue attribution

The result of cutting would be a “fast” system that:

• Fails Robustness (25%) — no escalation logic or contradiction handling
• Fails Feasibility (25%) — no audit trail means EU AI Act non-compliant (procurement = high-risk sector)
• Fails Reachability (20%) — hardcoded shortcuts don’t transfer to new clients

In procurement, a wrong decision costs $100K+. A slow decision costs 2 hours. The math is obvious.

Three killer lines for judges:

1. “A 1-minute agent is a search engine. A 5-minute agent is a procurement analyst. Chain IQ doesn’t need faster Google — they need fewer humans making $100K mistakes.”

2. “Under the EU AI Act, procurement is a high-risk sector. Every automated decision requires an audit trail. Our 5 minutes generates that trail. A 1-minute system can’t — and would be non-compliant in production.”

3. “The 5 minutes saves 2 HOURS of human work per request. At 6,000 requests/month, that’s 60 FTEs — $7M-$15M/year in savings. Nobody is asking those 60 people to work faster. They’re asking us to replace them.”

If pressed further:

“And we ARE optimizing. Marketplace-tier requests (80% of volume) will process in under 2 minutes because they skip the full analysis. The 5 minutes is for Technical and Strategic requests that NEED the depth. The system is smart enough to know the difference.”

NUMBERS CHEAT SHEET (All Mentor-Validated)

JUDGE-SPECIFIC HOOKS

MENTOR QUOTES TO USE IN PITCH

“Traceability is one of the most important things. That’s where things get lost.” — Chain IQ Mentor

“The superpower is not the model. The superpower is what happens when capable, reasonable systems are grounded in structured data.” — Daniel Ringsma, Head of AI, Chain IQ

“If they don’t speak about the disintegration… then I know they haven’t understood what it needs to do.” — Tim Paridaens, CTO, Chain IQ

“The house doesn’t burn down because it’s reckless. It burns down because it’s obedient.” — On AI blindly following policies without common sense (our agent handles this with conditional restrictions, not global bans)

WHAT COULD LOSE US POINTS (AND HOW TO ADDRESS)

SUMMARY: THE PITCH IN 30 SECONDS

“Sift is an autonomous sourcing agent that transforms messy purchase requests into audit-ready supplier comparisons. It enforces 145 procurement policies, routes escalations to the right person, and generates a complete audit trail — all in under 5 minutes. A human takes 2 hours and costs $100-$217 per request. Sift costs $1.35. At scale, that’s 60 FTEs and $7-15M per year in savings. And because every decision is traceable and policy-compliant, it’s production-ready — not just a demo.”

FREDDY’S PITCH SLIDE ADDITIONS — Guardrail Architecture

Added by Freddy during pitch prep (H28+). Maps the 6 challenge questions to real implemented layers with code-backed proof.

THE REAL CHALLENGE — Chain IQ’s 6 Questions

Can your system…

1. Detect contradictions?
2. Enforce hard policy constraints?
3. Handle restricted suppliers?
4. Refuse when risk is too high?
5. Trigger approval workflows?
6. Provide traceable decision logic?

Answer: Yes. All 6. With deterministic guardrails, not LLM promises.

THE 6 LAYERS — Challenge → Layer → Proof

LAYERED DEFENSE DIAGRAM — “The LLM is Caged”

This is the key visual for the pitch slide. The LLM sits at the center, surrounded by 6 concentric layers of deterministic guardrails. The AI recommends; the rules enforce.

┌─────────────────────────────────────────────────────┐
│            LAYER 6: AUDIT TRAIL                     │  ← Every decision logged to 10 files
│  ┌───────────────────────────────────────────────┐  │
│  │         LAYER 5: ESCALATION ROUTING           │  │  ← 8 rules, category-specific
│  │  ┌─────────────────────────────────────────┐  │  │
│  │  │      LAYER 4: SHORT-CIRCUIT REFUSAL     │  │  │  ← Refuse when risk too high
│  │  │  ┌───────────────────────────────────┐  │  │  │
│  │  │  │  LAYER 3: RESTRICTED SUPPLIERS    │  │  │  │  ← Contextual, not global bans
│  │  │  │  ┌─────────────────────────────┐  │  │  │  │
│  │  │  │  │  LAYER 2: POLICY ENGINE     │  │  │  │  │  ← 145 rules, deterministic
│  │  │  │  │  ┌───────────────────────┐  │  │  │  │  │
│  │  │  │  │  │ LAYER 1: DETECTION   │  │  │  │  │  │  ← 12 validation checks
│  │  │  │  │  │                       │  │  │  │  │  │
│  │  │  │  │  │      🤖 LLM CORE     │  │  │  │  │  │  ← Claude Sonnet 4.6
│  │  │  │  │  │   (recommends only)   │  │  │  │  │  │
│  │  │  │  │  │                       │  │  │  │  │  │
│  │  │  │  │  └───────────────────────┘  │  │  │  │  │
│  │  │  │  └─────────────────────────────┘  │  │  │  │
│  │  │  └───────────────────────────────────┘  │  │  │
│  │  └─────────────────────────────────────────┘  │  │
│  └───────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────────┘

Key message: The LLM is powerful but constrained. It can recommend anything — but deterministic Python code validates, reclassifies, and enforces before any output reaches the user.

KILLER NUMBERS FOR THE SLIDE

CODE-BACKED IMPLEMENTATION REFERENCES

SLIDE ONE-LINER (say out loud)

“We didn’t build an AI that replaces procurement. We built an AI that thinks like procurement — with 145 rules it can’t break, 8 escalation paths it must follow, and an audit trail that proves every decision.”

BONUS: CONTRADICTION HANDLING EXAMPLES (for Q&A depth)

Example 1 — Budget vs. Reality (REQ-c9f40c, Smartphones)

• Requester: 30 smartphones at €22,800
• Agent found minimum cost: €26,481 (16.1% shortfall)
• Didn’t fail. Generated 4 resolution paths:
  • A: Reduce to 25 units (stays under AT-001, Business-only approval)
  • B: Reduce to 22 Apple units (stays under AT-001, higher quality)
  • C: Revise budget to €26,481 (triggers AT-002, needs Procurement co-approval)
  • D: Revise budget to €30,030 for Apple (highest quality, triggers AT-002)

Example 2 — Insufficient Supplier Pool (REQ-000001, Consulting)

• AT-003 requires 3 quotes, only 2 suppliers serve Spain for IT PM Services
• Didn’t fake a third supplier. Flagged deviation, escalated to Head of Category
• Documented WHY (Deloitte: no Spain coverage; Infosys: no EU coverage)

Example 3 — Past Deadline (REQ-000042, Cloud Compute)

• Deadline: 2026-03-15, processed: 2026-03-19 (4 days late)
• Didn’t ignore it. Flagged HIGH severity, calculated earliest fulfillment (AWS: 2026-04-01)
• Suggested incumbent OVHcloud as bridge solution

Example 4 — Preferred Supplier Can’t Serve Region (REQ-000002, Cloud)

• Requester wanted Azure Enterprise; incumbent doesn’t serve Netherlands
• Excluded incumbent with documented reason, ranked Azure #4 but recommended with explicit deviation documentation

“When a request contradicts itself — budget too low, deadline passed, preferred supplier restricted — Sift doesn’t crash. It generates resolution options with trade-off analysis. That’s what a senior procurement specialist does. We automated that judgment.”