How we built Argus, a self-improving accounting platform

Booking a company’s month correctly takes judgement that tax experts build up over years and rarely write down. Together with the licensed professionals at Integral Tax, we turned that judgement into the evals that built our accounting agent.

Last month we launched Argus, our agentic accounting platform. Argus prepares the bookings for a client’s month. It reads the receipts, invoices and bank statements, works out what happened (a customer paid 2% less because they took the early-payment discount, money moved between two of the company’s own accounts, a payment came from a customer’s parent company instead of the customer), proposes the booking and reconciles each payment with its documents. Whatever needs a human decision gets flagged for the professional at Integral Tax who reviews the month and decides.

Most accounting software is built from rules. An office chair is an expense, unless it costs more than €800 net, in which case it has to be capitalised and depreciated, unless the company sells furniture, in which case it is stock. Every exception adds another rule until, sooner or later, the rules contradict each other.

We took a different route with Argus. The professionals at Integral Tax, who look after their clients’ books every day, wrote down real situations together with the accounting treatment they would apply to each one. These cases are our evals. Whenever we change Argus, a harness runs it against all of them, so we see straight away whether the change made it better or worse. Today the reviewing professional accepts more than 85% of the postings Argus prepares without changing them.

What a tax expert knows

Every posting draws on knowledge an experienced tax expert applies intuitively. Part of it is tax law full of exceptions: a business lunch is only 70% deductible for income tax, but its input VAT is deductible in full, so one receipt splits across two accounts. Part of it is context the document doesn’t contain. A transfer to the managing director can be salary, an expense refund or a loan. The bank line doesn’t say which. The right account for a vendor often depends on how the company booked it last year.

The rest is a firm’s own practice. Where the law allows more than one correct account, as it does for software subscriptions, a firm picks one and uses it for every client. Argus had to learn all of it. We needed a way to measure whether it had.

Before launch: climbing the evals

The cases

Many booking situations have one verifiable answer, which makes them good eval cases. We went through a normal month with the tax professionals at Integral Tax, one scenario at a time. A hotel bill alone carries two VAT rates: 7% for the room and breakfast, 19% for drinks and parking. For each case they wrote down the correct booking and why. The reasoning stays next to the case, so whoever debugs a failing case reads the explanation instead of guessing.

Every company, person, address and bank account in the eval data is invented, with IBANs and tax IDs that still pass validation, so Argus treats them like the real thing. No real company, person or bank account appears anywhere in the eval data. The cases keep the mess of a real month: a taxi receipt with the tip written in by hand, a crooked scan with half the figures unreadable, the same invoice uploaded twice, one direct debit that settles a dozen invoices and a credit note.

Our own eval harness

We built the eval harness in house. Every run tests the whole platform end to end, the same way it runs for clients. Our test suite creates a fresh test company for each case and loads its documents and bank data. Argus works through that company as it would through a client’s month. A grader then compares every booking with the answer the tax professionals wrote and keeps the full trace, so a failing case can be followed step by step back to where it went wrong.

We drive the harness with jeval, a command-line tool we wrote for it. It launches a run over every case or only the cases that failed last time, waits for the run to finish and diffs the result against main. Coding agents use the same tool and start most of the runs. Every change gets its own isolated environment, so many experiments can run at the same time without affecting each other.

The loop

Each round starts from the latest full run on main. We pull the failures, sort them by what went wrong and fix the whole group. Then we rerun and diff against main, rerunning any case that flips to separate a real regression from model noise. A change ships only when the group passes and nothing regresses beyond that noise.

Every fix has to be a general accounting principle. A fix like “invoices from Deutsche Telekom go to telephone costs” is not allowed. It turns one case green and breaks the first time a client buys a phone or a router on a Telekom invoice. Accounting judgement is written the way the tax professionals would explain it, never as special cases in code. Nobody edits an expected booking to get a run to pass either. If a case looks wrong, the tax professionals decide. Their reasoning goes on record.

Early on, Argus was parking ordinary payments on a clearing account and leaving them there. The cause was guidance we’d written for one narrow situation, which the model applied everywhere. We replaced it with clearer guidance on how to classify payments. A whole cluster of failures passed at once.

Later, Argus would not reconcile a card payment with its invoice when the bank line named the card processor instead of the vendor. A more elaborate fix made the score worse, so it never got to main. One line of guidance fixed it: “Missing evidence is not contradicting evidence.” A detail the bank line leaves out is no reason to doubt a reconciliation that otherwise holds.

Every class we fixed also got new cases from the tax professionals, written to fail on the old behaviour, so the eval data got harder as Argus got better. By launch there were several times as many cases as at the start.

After launch: learning from corrections

Corrections become cases

The postings the reviewing professional doesn’t accept unchanged get corrected. Corrections show which kinds of situations Argus still gets wrong, which makes them our most useful guide to new cases.

An agent runs every morning on our agent harness, reads the previous day’s corrections and groups them by kind of mistake. It checks whether the latest Argus still makes each one. If it does, a second agent traces the mistake to its cause.

For each finding, the tax professionals write new cases for that kind of situation. The fix then goes through the same loop as before launch. Once it ships, the new cases stay in the eval data, so every later version of Argus is tested against them.

A model per step

Argus runs as a chain of specialised steps, from reading a document to writing the booking. Each step has its own evals next to the cases for the whole chain, so we can change the model behind one step and see what happens to that step alone. Some steps want more reasoning. Others are just as accurate on a smaller, faster model and a lot cheaper to run. When we think a different model would suit a step better, we try it there. It stays if that step’s evals don’t regress.

What’s next

This is just the beginning. We are taking the same approach to other parts of the work at Integral Tax, with cases from the same professionals. Coding agents do most of the engineering on our internal app platform.

The hardest problems are still ahead. We are looking for builders who want to take them on, next to the people who know the answers. See our open roles.

Die technische Plattform wird von der Integral Services GmbH betrieben. Sämtliche Vorbehaltsaufgaben werden von der Integral Tax GmbH Wirtschaftsprüfungsgesellschaft erbracht.