"The problem with taxes is all those very small little details matter, and it's not going to get every single little detail right. These models get dramatically better over the course of every six months. But they still give you what is roughly the right answer, and that's not what you want."
"AI can be useful for processing and summarizing large amounts of information, but it struggles with precision in virtually every domain. Chatbots will often fabricate false factual claims, even when asked to summarize a single document. AI programming assistants will slip errors into their code. Image generators produce strange visual artifacts and inconsistencies."
"Only after supplying the models with highly specific instructions, like where each piece of information should go in each IRS document, did the AIs begin to fare better. That, you might argue, defeats the point of using an automated tool in the first place."
The New York Times tested four leading AI chatbots—ChatGPT, Claude, Gemini, and Grok—for tax filing capabilities and found significant shortcomings. All models miscalculated tax amounts owed to the IRS by an average exceeding $2,000. The fundamental issue is that AI excels at processing large information volumes but fails at precision tasks. Tax filing demands exact calculations and meticulous attention to countless small details across highly specific IRS forms. While AI models improve every six months, they provide approximately correct answers rather than the accuracy required for tax compliance. Only when given extremely detailed instructions about where information belongs in specific documents did performance marginally improve, undermining the purpose of automated tax assistance.
Read at Futurism
Unable to calculate read time
Collection
[
|
...
]