"Hallucination is fundamental to how transformer-based language models work. In fact, it's their greatest asset: this is the method by which language models find links between sometimes disparate concepts. But hallucination can become a curse when language models are applied in domains where the truth matters. Examples range from questions about health care policies, to code that correctly uses third-party APIs."
"With agentic AI, the stakes are even higher, as the autonomous bots can take irreversible action-like sending money-on our behalf. The good news is that we have methods for making AI systems follow the rules, and the underlying engines of those tools are also scaling dramatically each year. This branch of AI is called automated reasoning (a/k/a symbolic AI) which symbolically searches for proofs in mathematical logic to reason about the truth and falsity that follow from axiomatically defined policies."
"It is important to understand that we're not talking about probability or best guesses. Instead, this is about rigorous proofs found in mathematical logic via algorithmic search. Symbolic AI uses the foundations originally laid out by predecessors such as Aristotle, Bool, and Frege-and developed in modern times by great minds like Claude Shannon and Alan Turing. Automated reasoning is not just theory: in fact, it enjoys deep industry adoption"
"In the 1990s, it began with proofs of low-level circuits in response to the FDIV bug. Later, it was in safety critical systems used by Airbus and NASA. Today, it is increasingly deployed in instances of neurosymbolic AI. Leibniz AI, for example, is applying formal reasoning in AI for the legal domain, while Atalanta is applying the same ideas to problems in government contracting, and Deepmind's AlphaProof system doesn't generate false arguments in mathematics because it uses the Lean theorem prover."
Hallucination is fundamental to how transformer-based language models work and helps them link disparate concepts. Hallucination becomes problematic in domains where factual accuracy matters, such as health care, API usage, and agentic AI that can take irreversible actions like sending money. Automated reasoning, also called symbolic AI, uses algorithmic search for proofs in mathematical logic to enforce axiomatic policies and determine truth or falsity rather than probabilistic guesses. Symbolic methods have historical roots in Aristotle, Bool, and Frege and modern development by Claude Shannon and Alan Turing. Industry adoption includes circuit proofs after the FDIV bug, safety-critical systems at Airbus and NASA, neurosymbolic deployments, and formal-reasoning projects in legal, government contracting, and mathematics.
Read at Fortune
Unable to calculate read time
Collection
[
|
...
]