Rozes compares company accounts and public information with patterns found in businesses for which offences have been legally established. The Italian startup produces statistical indications of similarity. Its cofounder Jacopo Berti explicitly distinguishes that output from a finding of wrongdoing.
Keep that distinction in view, because it is exactly where the dangerous shortcut begins. An algorithm returns a signal. An office treats the signal as an answer. A company may then have to spend time and money explaining why the answer should never have been treated as an accusation. The machine saves effort on one side of the desk and creates it on the other.
This is a risk of deployment, not an allegation that Rozes has already caused those outcomes. It is also the reason I refuse to accept a sales presentation about efficiency as the end of the argument.
Apparently the accounts existed all along
Company accounts were already filed. Public databases were already there. Artificial intelligence did not discover the legal requirement to deposit financial information. So why is examining it suddenly presented as a revelation?
A system can increase analytical capacity and process information at a scale that manual work cannot match. That does not retrospectively excuse every institutional failure to examine available evidence. “We now have an algorithm” is not an explanation for “we previously failed to do the work”.
What irritates me is the possibility that automation becomes another layer of delegated responsibility. The machine produces the warning, an official endorses it, and nobody owns the difficult task of deciding what it actually establishes. A time clock for the person operating the time clock. At the end of this magnificent chain of productivity, the business still needs somebody to read the bloody documents.
The problem is not that a statistical signal can never be useful. It is that usefulness is being discussed at the receiving institution’s convenience. The organisation being scored has rather more at stake than the attractiveness of the interface.
Ninety per cent is not a conviction rate
Rozes reports precision of 90% among flagged entities. That is a claim about confirmation of the relevant warning under the startup’s assessment, not nine criminal convictions out of ten. The report also says four out of ten risky entities remain undetected. These figures answer different questions: how often a flag is confirmed, and how much risk the system misses.
Neither figure turns a resemblance into proof. Neither tells a buyer that the model will reproduce the same performance in every population or workflow. And a claim about money that might have been recovered had a system been used in the past is not recovered money.
A slide can accommodate those distinctions in a footnote. A decision affecting a company cannot afford to lose them. If an error influences access to an opportunity, an investigation or a business relationship, its cost does not remain inside the performance chart.
For the supplier, it may be an error rate. For the business, it may be a lawyer’s invoice and a question that never quite disappears from the file.
The human stamp is not enough
“The final decision remains human” sounds reassuring until you ask what that person does. Examines the underlying material? Challenges the assumptions? Has the authority and time to reject the score? Or ratifies the output because disagreeing with a supposedly scientific system now requires more explanation than accepting it?
A human signature can formalise an error just as efficiently as a correct decision. Adding a person to the end of an automated chain is not an accountability mechanism if the person merely certifies that the chain ran.
A risk indication must not quietly become presumed guilt. The moment an institution expects a business to disprove an algorithmic resemblance, it risks reversing the practical burden of explanation, whatever the formal legal wording may remain. Presumption of innocence is not a decorative statement to keep on the website while operational practice heads elsewhere.
Before selling suspicion as efficiency, explain who examines the mistakes, how they can be challenged and what prevents a score from acquiring authority it does not possess. Those are not administrative details to settle after adoption. They are the difference between an analytical instrument and a suspicion factory.
The algorithm does not sign an accusation. My concern is the person prepared to sign for it without understanding what the hell it has said.
Raffaele Di Marzio
All my “insane” books on cybersecurity and governance are here 👇 https://www.amazon.it/stores/author/B0FB47T6Q4/allbooks