Project plan
Stages, who does what, what each stage must prove, and what could go wrong
The plan is built so that each stage produces something One Claim can judge before committing to the next, and so that real data is only used once the right approvals are in place. The durations below are our first estimate for a planning conversation; the largest uncertainties are group approval times and Keylane.
Stages
Stage 0: Specification and discovery
Indicative 2 to 3 weeks
Work
- Agree this specification and answer the open questions
- One hour with each of the two triage staff, watching and recording how they decide
- Collect the folder list, email types, category rules, known senders and reference formats
- Collect reply templates and any client handling instructions relevant to triage
- Send the Keylane questions; ask about portal access
- Start the data protection impact assessment and group supplier review
Delivered
- Signed-off specification
- First version of the rule sets, written in plain English
- Draft impact assessment
- Confirmed demonstration scope, audience and date
Done when
- One Claim agrees the rules reflect how triage is actually done
Stage 1: Demonstration
Indicative 4 to 6 weeks, can overlap Stage 0's approvals
Work
- Custodia test Microsoft 365 tenant and mailbox, loaded with invented emails
- Build the platform parts: connector, document preparation, fact extractor, rules engine, rule editor with dry run and test cases, audit log
- Postbot with the Stage 0 rules
- Medical report reader on invented reports
- Mock claims data in the shape of a SiteView export
- Internal testing against the answer set
Delivered
- The demonstration app, usable by One Claim unaided
- Measured cost per email and per report on the invented set
Done when
- All items on the demonstration's done list are met
- One Claim decides whether to proceed to real data
Stage 2: Accuracy test on historic email
Indicative 2 to 3 weeks, after approval gate 1
Work
- One Claim's own Anthropic account and key set up
- Read-only connection to the live mailbox registered by One Claim's administrators
- Dry run over about 500 already-filed emails (or more), compared with where staff filed them
- Review every disagreement with a triage lead; adjust rules and save hard cases as test cases
- Repeat until results are stable
Delivered
- Accuracy report: Category 1 recall, folder agreement, claim match accuracy, rule coverage, review rate, cost per email
- Revised rules and a test case library
Done when
- One Claim sets the targets for shadow running based on these figures
Stage 3: Shadow running
Indicative 4 to 8 weeks
Work
- Postbot processes every new email, read-only; staff triage as normal
- Staff see Postbot's decision alongside their own on the web page
- Weekly review of disagreements; rule changes through the editor with dry runs
- Claims data connected (SiteView export, interface or reporting database)
- Train supervisors on the rule editor
Delivered
- Weekly accuracy reports
- Supervisors able to maintain rules without Custodia
Done when
- Targets met for a set number of consecutive weeks
- Approval gate 2 passed
Stage 4: Assisted filing
Ongoing, widened in steps
Work
- Read-write connection created
- Postbot files one email type first (for example, portal notifications decided by rules alone), then widens type by type
- Everything else continues to the review folder
- Draft replies switched on
Delivered
- Reduced manual triage, measured
- Supervisor dashboard in daily use
Done when
- Each widening step is agreed after a dry run and a week of results
Stage 5: Medical report analysis
Indicative 6 to 8 weeks; can start once Stage 3 is stable
Work
- Confirm route to reports (SiteView, portal link, or manual export)
- Load and check the tariff tables
- Extraction tested on a sample of real reports, each checked by a handler
- Backlog processed through the batch service
- Portfolio reporting agreed with One Claim
Delivered
- Measured extraction accuracy
- Tariff and non-tariff breakdown for the backlog, with evidence
- Ongoing processing of new reports
Done when
- Handlers agree with the classification on the checked sample to an agreed level
After Stage 5, the other applications can be taken one at a time, each with the same pattern: rules drafted, dry run on history, shadow, then live.
Approval gates
| Gate | Before | Needs | Who approves |
|---|---|---|---|
| 1. Real data | Stage 2 | Impact assessment approved; supplier and security review passed; contracts in place; One Claim's Anthropic account; read-only mailbox connection consented | One Claim, group data protection and IT |
| 2. Postbot may act | Stage 4 | Shadow targets met; read-write connection consented; named owner for the rules | One Claim operations lead |
| 3. Health data | Stage 5 | Impact assessment extended to medical reports; zero data retention agreed with Anthropic if required; portal terms checked | One Claim, group data protection |
Who does what
| Party | Responsibilities |
|---|---|
| One Claim sponsor | Decisions on scope, targets and each stage; owner of the benefits |
| One Claim triage staff | Explain current rules; review disagreements in Stages 2 and 3; become the first rule editors |
| One Claim supervisor (rule owner) | Approves and publishes rule changes once live |
| One Claim handlers | Check medical report extractions in Stage 5; give feedback on summaries and flags |
| Group IT | Application registration and consent in Microsoft Entra ID; security review; hosting decision |
| Group data protection | Impact assessment, contracts, retention policy |
| Keylane | Answer interface questions; provide access or exports |
| Custodia | Design, build, testing, documentation, training, support; drafts of the impact assessment and security material |
Workstreams
| Workstream | Main items |
|---|---|
| Platform | Microsoft 365 connector (read-only, then read-write), work queue, document preparation and text extraction, fact extractor, rules engine, rule editor, dry run, test cases, versioning, audit log, web application and sign-in |
| Postbot | Email types, reference formats, known senders and templates, category rules, filing table, reply templates, review queue, dashboard |
| Medical | Report fact schema, injury decision table, tariff tables, cross-check rules, portfolio reports, batch processing |
| Integration | Claims data adapter (mock, then export, interface or reporting database); portal route |
| Governance | Impact assessment, security review, contracts, retention, Anthropic account and data terms, model change procedure |
| People | Rule editor training, staff briefings, supervisor handbook, feedback loop |
Risks
| Risk | Effect | How we reduce it |
|---|---|---|
| Accuracy lower than hoped on real email | Less manual work removed | Stage 2 measures it before any commitment to act; review folder takes anything uncertain; rules improve weekly |
| An urgent email is missed | Missed deadline, cost, complaint | Category 1 is the default when unsure; independent keyword rules; Category 1 recall is the headline measure; staff keep seeing the inbox in early stages |
| No usable SiteView access | Weaker claim matching; some applications not possible | Scheduled export as fallback (option B); design does not depend on the interface |
| Group approvals take longer than planned | Stages 2 onward delayed | Start the impact assessment in Stage 0; demonstration uses invented data so it is not blocked |
| Rules not maintained after go-live | Accuracy drifts as senders and folders change | Named rule owner; editor warns about stale folders and unmatched senders; weekly unmatched report |
| Staff do not trust or use it | Benefits not realised | Triage staff shape the rules; every decision is explained; shadow period before it acts |
| Model changes or retirement | Extraction behaviour shifts | Model version fixed in configuration; any change re-runs the full test case library and a dry run on recent mail before switching |
| Running cost higher than estimated | Budget overrun | Cost per email measured from Stage 1; monthly spend limit set in the Anthropic console; rules take over more email over time |
| Malicious or manipulative email content | Wrong decision | Claude returns facts only, has no actions, and keyword rules run independently; test cases include manipulation attempts |
| Portal terms restrict automated processing | Stage 5 route blocked | Check terms during Stage 0; manual export fallback |
Running it after go-live
- Daily: service health check, review folder cleared by staff, Category 1 queue on the dashboard.
- Weekly: disagreements and corrections reviewed by the rule owner; rule changes dry run and published; unmatched senders report.
- Monthly: accuracy, cost and volume report; review of flagged items and any incidents.
- On any model change: full test case run and dry run before switching.
- Yearly: certificate rotation, impact assessment review, access review.
Custodia would provide support and maintenance under a separate agreement, with the aim that One Claim owns the rules and day-to-day changes.