"Sikka is currently the CEO of Vianai Systems as well as an Oracle boardmember and advisor to Stanford University on the subject of AI, and has a long history in enterprise tech, having served as the CEO of Infosys and CTO of SAP. He and his co-author - his son Varin - wrote that LLMs and agents based on LLMs hallucinate when ordered to complete tasks that are too complex for their model."
"The paper, which was surfaced by Wired last week, doubts whether AI agents are capable of carrying out multi-step tasks or checking to see if a task was done correctly. "Our argument, in essence, is: if the prompt to an LLM specifies a computation (or a computational task) whose complexity is higher than that of the LLM's core operation, then the LLM will in general respond incorrectly," the paper states."
ServiceNow asserts its automation agents draw on 20 years and 80 billion workflow instances of operational experience, making the underlying language model a small component of the product. Amit Zavery states ServiceNow's agents rely less on an LLM than the agents examined by Vishal and Varin Sikka and that their conclusions do not apply to ServiceNow. Vishal and Varin Sikka contend LLMs and LLM-based agents hallucinate on tasks whose computational complexity exceeds the model's core operation, preventing correct execution of multi-step or high-complexity tasks. ServiceNow emphasizes system-level experience and integration over sole dependence on LLM capability.
Read at Theregister
Unable to calculate read time
Collection
[
|
...
]