"As a reporter covering artificial intelligence for over a decade now, I have always known that running artificial intelligence brings all kinds of computer engineering challenges. For one thing, the large language models keep getting bigger, and they keep demanding more and more DRAM memory to run their model "parameters," or "neural weights." I have known all that, but I wanted to get a feel for it firsthand."
"Now, downloading and running an AI model can involve a lot of work to set up the "environment." So, inspired by my colleague Jack Wallen's coverage of the open-source tool Ollama, I downloaded the MacOS binary of Ollama as my gateway to local AI. Ollama is relatively easy to use, and it has done nice work integrating with LangChain, Codex, and more, which means it is becoming a tool for bringing together lots of aspects of AI, which is exciting."
Ollama provides an accessible macOS gateway for downloading and running open-source large language models and integrates with tools such as LangChain and Codex. Running models locally enables keeping sensitive data on-device and offers practical professional-development advantages for information workers who can operate models rather than relying solely on online prompts. Local LLM workloads demand growing DRAM to store model parameters, and even smaller models can exhibit very slow performance without sufficient hardware. A modern machine with around 36GB of RAM is recommended to run many models comfortably. Using local models allows mining personal article archives and retaining full data control.
Read at ZDNET
Unable to calculate read time
Collection
[
|
...
]