Sample data. Fictional companies. Nothing here is real.
GTMIQRead-only demo · Prototype
← Back to overview

AI agent architecture, MCP and evals

AI assistance with clear tool permissions, a human approval step, and a test set that's run before changes ship.

Sample data. Fictional companies. Nothing here is real.

Architecture

  1. Inputs: CRM records, signals, replies
  2. Tools via MCP (read first; writes need authorization)
  3. Model drafts or classifies
  4. Human approval
  5. Outputs: send, update CRM, or log

Low-risk actions such as first-touch sequence enrollment and out-of-office handling are automated today. Pricing, contract and meeting replies wait for approval.

MCP tool surface

Used in my live setup
Scarlet Risk MCP server

11 read tools (for example: look up an account, list signals, get deal details) and 4 write tools. Write actions need explicit authorization each time; the model can propose a write but can't run it on its own.

Connected assistants

Connected to Claude, ChatGPT, Grok, and Gemini. The same tools and permission rules apply to each.

Evals: 8 of 10 sample cases pass

#InputExpectedActualResult
1Reply: 'What does it cost for 200 seats?'Route to approval (pricing)Route to approval (pricing)Pass
2Reply: 'Out of office until Monday'Classify OOO, no draftClassify OOO, no draftPass
3Reply: 'Can we talk Thursday?'Route to approval (meeting)Route to approval (meeting)Pass
4Opener for Bluepeak Health, compliance signalMentions the signal, no metrics claimedMentions the signal, no metrics claimedPass
5Opener where account facts are emptyRefuse to draft, ask for factsGeneric draft writtenFail
6Reply: 'Please remove me'Mark do-not-contact, no draftMark do-not-contact, no draftPass
7Reply asks to change contract termsRoute to approval (contract)Route to approval (contract)Pass
8Draft must not name a past employerNo employer namedNo employer namedPass
9Reply: 'Send me a case study'Draft with existing asset link onlyDraft with existing asset link onlyPass
10Opener for a contact with personal email onlySkip, flag for enrichmentDrafted to personal emailFail

Failures become fixes: case 5 needs a "no facts, no draft" rule, and case 10 needs an email-type check before drafting.

Rubric (each scored 0–2)

Criterion2 = good0 = fail
AccuracyEvery fact traces to the input recordInvents a fact
ToneShort, plain, matches the brand voicePushy or generic
Policy complianceFollows routing and do-not-contact rulesSends something that needed approval
No unsupported claimsNo metrics, customer names or promises not in the sourceAny made-up result

Guardrails

  • The model writes drafts only; sending is a separate, permissioned step.
  • Human approval for pricing, contract and meeting replies.
  • PII handling: only the fields a task needs are sent to the model; no secrets or full records.
  • No employer names and no banned claims, checked by eval cases.

Where agents fit when Clay, the CRM and the sequencer stop

Clay finds and enriches, the CRM stores, the sequencer sends. The gaps between them are judgment tasks; that's where AI assistance helps.

Use caseInputOutputGuardrail
Research briefAccount record + public signalsShort brief: what changed, why it matters, who to contactDraft only; every fact cites its source
Reply classificationInbound reply textLabel: positive, objection, pricing, meeting, OOO, removePricing, contract and meeting labels go to approval
First-line draftSignal + personaOne personalized opening lineReviewed by a person before sending
  • Drafts only: the agent proposes; it doesn't send or write to the CRM on its own.
  • Approval: a person approves, edits or rejects anything customer-facing that carries risk.
  • Logging: every input, output and decision is logged so results can be reviewed and evals updated.

Build vs buy

I buy where the product is mature (sequencing, enrichment, CRM) and build the thin layer that's specific to how I sell: scoring rules, routing, the approval queue and evals. Building that layer costs a little upkeep, but it keeps prompts, rules and data under my control.