Ollama 0.40.11 Hands-On: Running Local LLMs on Windows

extract

Ollama 0.40.11 gives Windows users a desktop chat application and a local model runtime for other software. Ollama has also moved beyond its original command-line interface, adding a searchable model selector, image attachments, and links to coding tools. Some controls we needed for testing still live in PowerShell and model configuration files. We installed Ollama on our HP Z4 Rack G5 running Windows 11 Pro for Workstations, with 128GB of RAM and an NVIDIA RTX 5000 Ada Generation GPU. Using our existing Jan V3.5 4B and Jan V2 VL High GGUFs, we tested generation speed, chat, images, online search, and the local API. Ollama ran our local models well, though importing the GGUFs and collecting performance data took us out of the desktop chat. It is the third application in our local LLM series, after Jan and Open WebUI, and the results feed our Best Local LLM Tools page. Test System Installing Ollama and Getting Started Installing Ollama on our Windows workstation was a welcome change from some of the setup headaches we encountered with other applications in this series. We downloaded the standard Windows installer, launched the application, and didn’t have to install a separate inference runtime or track down CUDA packages. A quick PowerShell check confirmed that Ollama 0.40.11 was running and that its local service was already responding at http://127.0.0.1:11434. After we got past the welcome screen, Ollama opened to a familiar chat interface with recent conversations, Apps, and Settings in the left sidebar. The message field keeps the attachment button, Web Search, and model selector together. We could start chatting with our imported local models without creating an account, although Ollama asked us to sign in when we later tried enabling Web Search during testing. Browsing and Adding Models Ollama keeps model selection fairly simple, with a dropdown that shows both installed models and others available to download. We could see our imported Jan models and Qwen3-VL alongside several cloud-based options, which were identified by a small cloud icon. There’s also a search field for finding other models, although we would have liked to see more information here, such as model sizes, quantization, and hardware requirements. Jan provides those details through its model hub, while Ollama keeps most of that information out of the main chat interface. Getting our two benchmark models into Ollama took more work because we wanted to use the exact same GGUF files from our previous reviews, keeping the performance tests as comparable as possible. While Ollama lets you download models directly through its desktop interface, we couldn’t find an option to import GGUF files already stored on the workstation. We had to use PowerShell instead, creating a Modelfile for each model that pointed to its existing GGUF and included the sampling parameters used in our earlier tests. We then used the ollama create command to add them to Ollama’s model library. Both imports worked without any issues, with Jan V3.5 appearing as jan-v35-sr and Jan V2 VL High as jan-v2-sr. We also downloaded Qwen3-VL 4B directly through Ollama for a separate image-analysis test. After importing the models, we checked how Ollama identified them and whether their settings had carried over properly. Jan V2 VL High was recognized as an 8.2-billion-parameter Qwen3-VL model using Q4_K_M quantization, which matched our original file. Jan V3.5 was the exception, as the GGUF we imported was named Jan-v3.5-4B-Q4_K_XL.gguf, but Ollama reported its quantization as Q4_K_M. We left the original file unchanged; the likely reason is that Q4_K_XL is Unsloth’s dynamic quantization, built mostly from Q4_K_M layers, and the file’s metadata records that base type. Model Settings and GPU Use Ollama’s Settings page includes cloud access, automatic updates, network access, a model-storage path, and a context-length slider, which showed 32K when we first inspected it. Our performance tests explicitly requested 8,192 tokens through the API, so the benchmark results below used that context, not the desktop default. Because Ollama doesn’t offer the same level of model configuration through its desktop interface as Jan, we made most adjustments through Modelfiles and API requests. To keep our benchmarks comparable, we used the same settings as our previous tests: temperature 0.8, Top K 40, Top P 0.95, Min P 0.05, and a repeat penalty of 1.0. We also requested a batch size of 2,048, though we couldn’t independently verify whether Ollama applied it during generation. GPU configuration was different as well, since there was no desktop setting equivalent to the 100 GPU layers we used in Jan. Ollama handled GPU offloading automatically, and a quick check using ollama ps confirmed that both imported models were running entirely on the RTX 5000 Ada. We verified that report against NVIDIA’s own readings. Jan V3.5 increased VRAM allocation by several gigabytes after loading, and a live reading reached 93 percent GPU utilization during generation. Jan V2 also reported full GPU residency and showed 96 percent utilization in our diagnostic run. At least on this workstation, we didn’t have to work through a separate CUDA variant or GPU-layer setup to get accelerated generation. Local Generation Performance For generation testing, we used the same prompt from our published Jan hands-on: “Explain RAID 5 and RAID 6 to someone building their first NAS. Include the main advantages and disadvantages of each, in about 400 words.” We disabled Web Search, set the context to 8,192 tokens, and included only that single user message in each API request. We discarded one warm-up response per model and then collected three measured runs. Both models completed all three measured runs without generation failures, and GPU utilization stayed high throughout testing. Ollama required no additional CUDA configuration or manual GPU-layer adjustments to get both models running with full GPU acceleration, which was not the case in Open WebUI, where we added CUDA runtime DLLs by hand. Note: Output-token counts, decode speeds, and GPU memory figures are medians of three measured runs after one discarded warm-up run. GPU memory is the median of the peak GPU-wide memory use recorded during each measured run. *The Jan V3.5 GGUF filename identifies Q4_K_XL, while Ollama’s model information reports Q4_K_M; see the import notes above. Jan V3.5 recorded 145.48, 145.73, and 145.44 tokens/sec across its measured runs, while Jan V2 recorded 92.87, 92.22, and 92.97 tokens/sec. Both sets stayed close from run to run, and the measured GPU memory peaks were unchanged within each set. Jan V2 produced considerably more reported output tokens, apparently including reasoning activity, so those totals are not a direct comparison of visible answer length. On the same workstation and model files, those are the fastest results in this series so far. Jan V3.5 decoded 18 percent faster than in Jan itself (123 tokens/sec) and 26 percent faster than in Open WebUI Desktop (115.32), and Jan V2 was 13 percent ahead of Jan (82) and 24 percent ahead of Open WebUI (75.06). Each application runs its own llama.cpp build and manages GPU offload its own way, so the comparison reflects the complete package each one ships, not the model alone. We did not publish prompt-processing medians because repeated requests produced very short timings that caching may have affected. Nor can runtime settings be identical in every respect: Jan exposed GPU layers directly, while Ollama managed offload automatically. Using Ollama for Local Chat With the models installed, we selected Jan V3.5 or Jan V2 from the dropdown and entered the RAID prompt. Answers appeared with headings and lists, while recent conversations stayed in the sidebar. Ollama didn’t display generation rate or token count beneath responses, so we collected those figures via its API. Managing conversations in Ollama was easy enough, since right-clicking an earlier conversation brought up options to rename or delete it, with a Share option available under More tools. We could also go back and edit prompts we’d already sent, although there weren’t as many options for the model’s responses. The only button we found was Copy, which stays visible below each answer even when you’re not hovering over it. We couldn’t find a way to regenerate a response or view its generation statistics from the chat window, at least in the version we tested. Ollama also has an Apps section with coding assistants, including Codex CLI, Claude Code, OpenCode, and Cline. We wanted to see how these integrations worked, so we clicked Codex CLI to see what setup involved. Ollama did not open a configuration screen; it copied a launch command to the clipboard and told us to paste it into a terminal. We didn’t complete the full installation or run Codex, but it gave us a better idea of how Ollama connects local models to external applications. Web Search Ollama places a globe button alongside the attachment control in Chat, but selecting it showed a message that Web Search requires an Ollama account. After signing up for a free account and enabling the feature, we used our existing test question asking for the latest stable Open WebUI release as of October 15, 2026, with a source citation. We ran into another issue when testing Ollama’s Web Search feature: the application tried to retrieve results but returned a “search API error (status 402)” message instead. HTTP 402 is the Payment Required code, which suggests the search service needs a paid plan, though Ollama did not say so. The model suggested the problem might be related to account pricing or usage limits, but we couldn’t determine the exact cause. Since our local models were working normally, we decided not to spend more time troubleshooting the search service. It was still disappointing that we couldn’t complete the same Web Search test we’d attempted with the other applications in this series. Multimodal Input Our first image test used the same Jan V2 VL High GGUF that worked with a separate vision projector in Jan. Ollama accepted the language-model file, but our import procedure did not associate the accompanying mmproj file. When we tried to attach the NVIDIA screenshot to a Jan V2 conversation, the desktop application displayed “This model does not support images” before we could send the prompt. We then tried Qwen3-VL 4B, downloaded through Ollama, using the same prompt asking for the GPU model, driver version, VRAM, utilization, temperature, and power. That model accepted the screenshot and finished with the correct six values: RTX 5000 Ada Generation, driver 597.06, 32,760 MiB, 0 percent utilization, 39C, and 40W. Its response was not perfect throughout, since it briefly gave an incorrect 8 percent utilization reading before correcting that value in the final summary. In Ollama, we couldn’t paste an image directly from the Windows Snipping Tool into the chat window like we could with the other applications. Instead, we had to save the screenshot first, then upload it using the + button next to the message field. That worked without issues, and Ollama’s Qwen3-VL model analyzed the image successfully. Our imported Jan V2 model could not accept the same attachment, which limited our ability to repeat the original multimodal test. Local API Server Ollama’s local API was available as soon as its Windows service started, without launching a separate server from the desktop interface. A request to http://127.0.0.1:11434/v1/models returned our installed Jan V3.5 and Qwen3-VL models, and a request to /v1/chat/completions using Jan V3.5 generated a one-sentence explanation of RAID 6. The response used slightly imprecise RAID 6 parity wording, which was a model-output issue, not an API failure. The working endpoint allowed us to query local models directly from PowerShell without using desktop chat. What Went Wrong The most frustrating issue we encountered was getting our imported Jan V2 VL High model to work with images. We had no problems importing the GGUF or running text prompts with full GPU acceleration, but things changed when we tried attaching our NVIDIA screenshot. Ollama told us the model didn’t support images because our import didn’t include the separate vision projector that Jan V2 requires. We could still complete the image-analysis test using Ollama’s Qwen3-VL download, which worked without issues. This showed that Ollama could process images, but getting our existing Jan V2 model to do the same wasn’t as simple as importing its GGUF file. To use the Web Search feature, we had to create an Ollama account before we could enable it. Once we signed in and submitted our test question, the service returned an HTTP 402 error and no search results. We couldn’t determine what caused the error, and there wasn’t much reason to keep changing account settings or trying different plans without knowing whether that would help. We also noticed how often we had to return to PowerShell for tasks like importing our existing models, adjusting their settings, and collecting performance measurements. The desktop application was easy to use for everyday chatting, but it didn’t offer the same level of control we had through Jan’s interface. Aside from these issues, Ollama gave us very little trouble during testing: both models imported successfully, all six measured benchmark runs completed, and the local API worked as expected. Conclusion Our testing with Ollama 0.40.11 generally went well, especially getting local models running on the RTX 5000 Ada. Both of our imported GGUFs worked with full GPU acceleration without requiring any additional CUDA setup, and performance led the series, with Jan V3.5 reaching a median of 145.48 tokens/sec and Jan V2 VL High reaching 92.87 tokens/sec, the fastest results in this series and 13 to 18 percent ahead of Jan on the same workstation. Once everything was configured, we had no problems moving between the desktop chat interface and the local API, and all measured benchmark runs completed without issues. While we liked how simple Ollama was for everyday chatting, we would have preferred more control over individual models in the desktop application. Having to use PowerShell and create Modelfiles to import our existing GGUFs added a few extra steps, especially compared with Jan, where most of those settings are available directly in the interface. This won’t necessarily bother someone already familiar with running local models, but newcomers who want to manage everything through a graphical interface may find Ollama limiting. We also ran into a few issues with Web Search and our imported vision model, though neither affected normal text generation. What we liked most about Ollama was how easily it could serve local models to other applications without requiring us to configure a separate server. The desktop chat is easy to get started with, while the local API and support for external coding tools let users do a lot more than chat with local models. We would still like to see better model-management controls in the desktop interface, but Ollama performed well on our workstation and gave us a dependable way to run both of our test models with GPU acceleration. Leaderboard: Ollama holds Best Overall Local LLM Runtime on our Best Local LLM Tools page, where these results now sit beside our Jan and Open WebUI numbers.