Rule editor and dry run
One Claim's staff change the rules, and see the effect on real mail before anything moves
Why it matters
Because every decision is made by rules, the rules are the product. They will need changing often: a new credit hire company, a new broker, a renamed folder, a handler leaving, a new type of court letter. If only a developer can change them, Postbot falls behind the business within weeks. The rule editor lets a supervisor at One Claim make those changes, and the dry run shows exactly what a change would do to real mail before it goes live.
What can be edited
| Rule set | What it holds | Example |
|---|---|---|
| Sender lists | Named groups of email addresses and domains | "Credit hire companies": 14 domains |
| Templates | Sender plus subject or body patterns that fix the email type | Portal notification: sender is the portal's address and subject begins "Claim update" |
| Reference formats | Patterns for claim numbers, insurer, broker, court and portal references, registrations | One Claim reference: "OC-" then two digits, a dash and five digits |
| Safety keywords | Words or phrases that force Category 1 whatever else applies | "particulars of claim", "Financial Ombudsman" |
| Category rules | Conditions on the facts that set Category 1 or 2 | Type is credit hire and hire is ongoing: Category 1 |
| Filing table | Email type and client (or broker) to folder and handler | Credit hire, client "Example Logistics": folder "Fleet/Credit Hire", handler team B |
| Thresholds | Confidence levels below which an email goes to review; days that count as "urgent deadline" | Deadline within 5 working days: Category 1 |
| Reply templates | Approved wording with fields filled from the claim | Credit hire acknowledgement requesting the hire agreement |
| Medical rules | The injury decision table, tariff amounts, cross-check thresholds | Physiotherapy sessions above 6 with prognosis under 4 months: flag |
The editor
Rules are built from drop-down choices, not written as code. Each rule reads back as a plain English sentence, so anyone can check it:
When the sender is in Credit hire companies
and the fact hire ongoing is yes
then set category to Category 1, email type to Credit hire notification, and use the filing table.
- Order is visible. Rules run top to bottom within each rule set; safety keywords always run last and can only raise an email to Category 1, never lower it.
- Pick lists come from live data. Folder choices are read from the actual Microsoft 365 mailbox, so a rule cannot point at a folder that does not exist. Handler choices come from the claims data.
- Pattern testing on the spot. When editing a reference format, the editor shows which recent emails it matches and highlights the match, so a pattern that is too broad or too narrow is obvious straight away.
- Problem checks. The editor warns about rules that can never fire because an earlier rule always wins, rules that contradict each other, folders that have been renamed or deleted in the mailbox, and email types with no filing table entry.
- Notes. Each rule carries a note on why it exists and who asked for it.
Dry run
A dry run applies a draft set of rules to real email and reports what would happen, without changing anything in the mailbox. It uses a read-only mailbox connection, so it cannot move, flag, label or draft, even by mistake.
Choosing what to test against
- The current inbox, as it is now
- Any folder, or all folders, over a date range
- The last number of emails received (for example, the last 500)
- Emails staff have already filed, which gives an answer to compare against
- The saved test cases (below)
What it shows
- SummaryHow many emails would be decided, and how: by rules alone, with facts from Claude, or sent to review. Category 1 and 2 counts.
- Changes against the live rulesEvery email whose outcome would differ from the rules currently in use, with before and after side by side: category, folder, handler, claim. This is the main screen for judging a change.
- Agreement with staffFor emails already filed by staff, where the draft rules agree and disagree, and the effect on Category 1 recall. A change that makes Postbot miss urgent email is shown in red at the top.
- UnmatchedEmails no rule decided, grouped by sender and subject, as a prompt for the next rules to write.
- Drill-downOpen any email to see the full trace of which rules were checked and which one decided it.
Cost and speed
A benefit of keeping decisions in rules is that most dry runs cost nothing to run. The facts Claude extracted from an email are stored with the email and the version of the extraction instructions that produced them. Changing a rule re-runs only the rules over the stored facts, which takes seconds for thousands of emails with no call to Claude. Emails that have never been read by Claude, or where the extraction instructions have changed, need a call; the dry run shows how many and an estimated cost first, and asks before spending.
Test cases
Any email can be saved as a test case with the outcome it should get: "this must always be Category 1", "this must file to Fleet/Credit Hire". Hard cases found during shadow running (the court document hidden in a chaser, the email trying to instruct the bot) become permanent test cases. Every dry run and every publish runs them all. A draft that breaks a test case cannot be published until the test is fixed or a supervisor deliberately changes the expected outcome, with a note.
Publishing
- Draft, dry run, publish. Changes are made in a draft. A draft can only be published after a dry run on current mail and a passing run of the test cases.
- Second pair of eyes. Optionally, a different supervisor must approve before publishing. One Claim decides whether this applies to all rule sets or only to category and safety rules.
- Versioned. Every published version is kept with who published it, when, the dry-run results and the notes. Every Postbot decision records the rule version that made it.
- Roll back. Any earlier version can be restored in one step, and that is itself logged.
- Scheduled. A version can be set to go live at a set time, for example a new client from the first of the month.
Who at One Claim should be able to edit rules, and who to publish them? Should publishing need a second person?
Tracing a decision
For any email, live or in a dry run, the trace shows each step in order: references found, sender list and template checks, claim lookup, whether Claude was called and the facts it returned with their evidence, each category rule checked and whether it matched, the safety keywords, the filing table entry used, and the rule version. This is what a supervisor uses to answer "why did this go there?", and what an auditor would look at.