The deterministic era of software is giving way to a probabilistic future — one where behavior is learned, not coded.
In the year 1975 software error handling was done using error codes. About the year 2000, a majority of software error handling was done by throwing exceptions and using try/catch blocks. Both these categories worked because software was deterministic.
Traditional error handling assumes:
- Errors are discrete and detectable
- Systems behave the same given the same inputs
- There is a correct way to behave
In this era of AI systems, software can now generate vast amounts of realistic data, make inferences that are faster and better than human in some cases, and finally software can learn from data and experiences. Software just doesn’t execute canned instructions, but now has the ability to make autonomous decisions.
This is all well and good, until the decision is wrong!
Previous approaches to errors would be to review the logic and data and then make corrections. This approach does not hold up where the decision is made by AI.
An AI system is not discrete and usually does not give guaranteed repeatable results. An LLM may hallucinate with extreme confidence. A recommendation engine might polarise users when the designers never intended it to be so. A classifier might have performance and behavioural decay but there are no discrete mechanisms to catch this. See my article on the latter topic: “Distributional Shifts: How Data Drift Undermines Model Accuracy and What to Do About It”, https://medium.com/@shirish.ph/distributional-shifts-how-data-drift-undermines-model-accuracy-and-what-to-do-about-it-25b7e969bb65. AI systems do not work in with discrete logic where such discrete logic could reduce these states to be simple enough for correction. AI systems work in continuous spaces and this where new thinking is needed.
The basis of all modern science is methodological reductionism: the scientific attempt to provide an explanation in terms of ever-smaller entities. This has served us well especially in the sciences.
However, if reductionism seeks to understand the whole by analyzing its parts, then holism seeks to understand the whole by studying the interactions and context of the parts. The understanding of the whole is not just structural, but also in the system’s dynamics.
Understanding failure now means modeling relationships, feedback loops, and grokking how small changes amplify into large, often unintended outcomes.
Extant systems like try/catch were devised around deterministic systems, but AI systems are beginning to reveal the inadequacy of such error handling.
Error detection in AI systems is founded on:
- Confidence Thresholds: Assess correctness based on a degree of certainty
- Response Validation: Some form of post process evaluation
- HITL: Human-in-the-loop: Put responses on escalation ladder when there is uncertainty
- Model Behaviour Monitoring: Observe for drift/degradation over time
These new approaches to error detection in software systems suggest massive changes in software methodologies. There is a felt need for more formalism in this domain.
The quality of software can no longer be assured by writing code and tests. AI systems need a new approach with an assumption that AI systems can learn. AI systems are not given to raise errors like traditional software systems do, but they will fail silently by shifts in behaviour or bias injection. It is in the best interest of AI system developers and enterprises that are reliant on these systems to start working on formalisms around error handling.
