Why we never let the AI set the price

The computer reads. People and fixed rules decide.

Every software company now says it uses artificial intelligence. Fewer will tell you exactly what theirs is allowed to do. Here is our answer, in full.

On this platform, the AI reads, researches, and prepares. A broker’s email arrives, or a PDF of a remittance advice, or a proposal form filled in three different ways — and the AI turns that paperwork into tidy, labelled information: the occupation, the limit, the turnover, the policy reference. It researches the risk behind a submission and builds the underwriting worksheet — the exposures, what they amount to, and a recommendation for the underwriter to weigh. When cash arrives with a scrappy reference, it suggests where the line probably belongs, and a person confirms. In the product builder, it helps assemble a draft — question sets, wordings, rating tables — for a person to approve before anything runs. And at the other end of the system it drafts English: a quote letter, a plain-language explanation of an outcome. Artificial intelligence helps read the paperwork. It never decides the price.

Notice what is missing from that list: deciding. The worksheet still needs the signature; the suggestion becomes the match only when someone confirms it; and the draft product becomes a product only when someone approves it. And because a recommendation you always follow becomes a decision by another name, every recommendation is written down, and so is what the underwriter did with it.

The deciding is done by rules — written down, numbered, and versioned, the way an underwriting guide is written down — and by people. Ask why a premium moved and the answer is the rule that applied, quoted word for word, with its version number and the date it took effect. Put the same submission through tomorrow and you get the same answer. That sounds unremarkable. It is the property everything else depends on.

Why draw the line so hard? Three reasons.

The first is that this kind of AI can be confidently wrong. It writes fluent, plausible text, and sometimes the fluent, plausible text is mistaken. We plan for that. When the AI misreads a document, the misreading lands in a checking queue where a person looks at it. If the AI could decide, a mistake would land somewhere much worse — in a mispriced policy, discovered at claim time. So we built the system so that when something goes wrong, the result is more checking, never a wrong decision that slips through.

The second is that you cannot audit a guess. A rule is a thing you can inspect: who wrote it, who approved it, what it said last March. A change to a rule is written down like everything else on the platform. Each step is recorded at the time, and nothing is painted over: a correction sits next to what it corrected. That is what you would hand an auditor. No one can produce that file for the inner workings of a language model, and we would rather not sell you a story that says otherwise.

The third is that someone has to own the decision. Insurance runs on written authority — an underwriter agrees cover inside an agreement that says exactly which classes, which limits, which territories. A person can hold that authority and answer for it. A model cannot. Software can get everything ready. Only a person should be able to commit to cover.

Where the market is heading

We did not invent this position, and we make no claim to be braver than the market. The bodies that guide the Lloyd’s market published an AI adoption toolkit in 2026, and its advice comes down to grading AI by how much it is allowed to decide, and governing it in proportion. Regulators in Britain, Europe and Australia have each said their own version of the same thing: use the technology where it helps, keep a person answerable for the outcome, and be ready to explain any decision in plain language. We read all of that as a description of the system we had already chosen to build.

What the choice costs

We will also tell you what this choice costs, because you will notice anyway. Writing rules takes people who understand the class of business — the system cannot improvise an appetite it was never given. And the odd, ambiguous case goes to a person instead of being guessed at, which is slower than guessing. Some companies will promise you the machine does it all, faster. When you hear that promise, we would only suggest one question: ask them to show you the file they would hand an auditor.

That is why the rules engine decides, and the model never does. Or, in the line we use everywhere else on this site: the computer reads; people and fixed rules decide.

Cuttleflow Systems · Field Note · On AI · 33°53′S · 151°16′E · Sydney