First look: Run LLMs locally with LM Studio

"When you first run LM Studio, the first thing you'll want to do is set up one or more models. A sidebar button opens a curated search panel, where you can search for models by name or author, and even filter based on whether the model fits within the available memory on your current device. Each model has a description of its parameter size, general task type, and whether it's trained for tool use."

"Downloads and model management are all tracked inside the application, so you don't have to manual wrangle model files like you would with ComfyUI."

"To have a conversation with an LLM, you choose which one to load into memory from the selector at the top of the window. You can also finetune the controls for using the model-e.g., if you want to attempt to load the entire model into memory, how many CPU threads to devote to serving predictions, how many layers of the model to offload to the GPU, and so on. The defaults are generally fine, though."

LM Studio enables model setup through a sidebar-curated search panel with filters for name, author, and device memory compatibility. Model entries display parameter size, general task type, and tool-use training status. Downloads and model management are handled inside the application, avoiding manual file handling. A top-of-window selector loads chosen models into memory. Inference configuration options include attempting full model memory loading, setting CPU thread counts for serving predictions, and offloading specific model layers to the GPU. Reasonable defaults are provided to simplify initial use.

Read at InfoWorld