ProductDataAgents

Extraction schemas

When a customer sends us bank statements, there wasn't a standard answer for what should come back. Customers invent their own field names, and if we don't know what someone wants, we don't really have a default to offer them. So I set out to define one for each document type, starting with bank statements.

Fall 2026·Python, Claude, Sensible toolkit, JSON Schema

01Where I started

Look at what customers already built

I knew we had hundreds of configs we'd already built for customers, so instead of designing a schema from scratch I analyzed those: which fields people actually pull from a bank statement, what they call them, and how they structure the output. The end date alone showed up under more than a dozen names.

Roll up the names customers used

end_date89closeDate55statement_end_date54EndDate28Statement Closing Date5closing_date3period_to2start_date90openDate56statementOpenDate54statement_start_date52StartDate28period_from2account_number113accountNumber54PrimaryAccountNumber28Account Number8account_no3account_holder_name56customer_name54merchant41CustomerName28account_owner4

statement_period_end236 configs

statement_period_start282 configs

account_number206 configs

account_holder183 configs

Real field names from our bank statement configs.

02Research

See how everyone else structures it

Then I looked at how banking data standards and other document AI companies handle the same thing, like whether accounts are always a list and whether a transaction has one signed amount or separate debit and credit columns. That turned into a baseline for what a statement should come back as.

03Iterating

My first naming convention was wrong

My first pass named everything entity first, like balance_opening, because it sorted nicely. When I checked it against the configs, nobody had ever written balance_opening, and dozens of them said opening_balance. So I switched to the names people actually use and moved repeating rows into sections, which made the prefixes pointless anyway.

Field names in the wild rolled up to canonical fields

04Proving it

Test each config against real statements

A schema only helps if the configs actually return it. So there's a scorecard that runs each bank's config against every statement we have from that bank and checks that the transactions add up to the totals. Agents built configs for the banks we were missing, and nothing they made went live until it passed.

harness.py

def check_account(a, start, end):
    tx = a.get("transactions") or []
    got = [num(t.get("amount")) for t in tx if num(t.get("amount")) is not None]
    o, c = num(a.get("opening_balance")), num(a.get("closing_balance"))
    if o is not None and c is not None and tx:
        # opening + every transaction should land exactly on closing
        r["reconciles"] = abs(o + sum(got) - c) < TOL and len(got) == len(tx)
    dep = num(a.get("total_deposits"))
    if dep is not None and tx:
        r["deposits_match"] = abs(sum(x for x in got if x > 0) - dep) < TOL
    # and each row's balance should follow from the one before it
    ch = chain_rate(tx)

The check every config has to pass, trimmed from the real harness.

05Beyond banks

Check whether it carries over, then ship it

To see if the shape worked for other documents, I read configs across a lot of other industries. They split into two kinds, ledgers with rows and totals and forms with flat fields, so the guidelines have a version for each. Then the schemas, checks and sample files went into a kit people can download.

Where it is now

New bank configs get built to the schema, and the kit is out for anyone building a document pipeline.