[note May '26 - this post is a bit out of date now. By far the easiest way forward is to install LM studio and run your models there. Hook them up to VS code with the 'Continue' add on, or install opencode, as I've detailed in this post.] I've found VC Code copilot to... Continue Reading →
My experience training a local LLM (AI chatbot) on local data…
The user encountered challenges while attempting to use various methods to feed information into local Large Language Models (LLMs) via RAG (Retrieval-Augmented Generation). They explored methods such as Nvidia Chat with RTX, Ollama with Python scripts, and Ollama with Open-webui. Results varied, with some methods providing inaccurate or incomplete outputs. Comparatively, Microsoft Co-pilot, running GPT4-Turbo, significantly outperformed the local methods.