If you’ve been skimming the tech news this week, you probably saw the headlines about Microsoft’s latest push into consumer AI. Rumor has it they want users to become practically "addicted" to Scout, their upcoming AI personal assistant designed to embed itself deep into your daily digital workflow. It’s the classic Big Tech playbook: build a closed-source, highly proprietary assistant, tie it intimately to your OS, and lock you into their cloud ecosystem under the guise of "frictionless productivity."
As developers, our immediate reaction to this shouldn't just be skepticism about vendor lock-in or privacy (though those are highly valid). Instead, we should be asking: "How do we build this ourselves?"
We don't need a black-box assistant telemetry-mining our development environments. We have the open-source tools, the local LLMs, and the architectural patterns to build our own highly customized, context-aware AI assistants that run entirely on our own hardware. Today, we are going to bypass the corporate hype and build a local, context-aware developer assistant using Ollama, LangChain (TypeScript), and local vector embeddings. By the end of this post, you'll have a running assistant that can securely parse your local codebase and answer queries without sending a single byte of data to an external server.
The Architecture of a Privacy-First Developer Assistant
To build an assistant that actually understands your project context (unlike generic chat web interfaces), we need to implement a Retrieval-Augmented Generation (RAG) pipeline. Instead of training a model from scratch—which is computationally prohibitive—we feed our local model relevant snippets of our codebase alongside our prompt.
Here is how our local architecture looks:
+-----------------------------------------------------------------+
| Your Codebase |
| (TypeScript files, Markdown docs, Configuration files, etc.) |
+-----------------------------------------------------------------+
|
v [AST Parser / Document Splitter]
+-----------------------------------------------------------------+
| Text Chunks (Tokens) |
+-----------------------------------------------------------------+
|
v [Local Embedding Model: mxbai-embed-large]
+-----------------------------------------------------------------+
| Vector Database (HNSWLib) |
+-----------------------------------------------------------------+
|
[User Query] ---> [Vector Similarity Search] ---> [Context Retrieval]
|
v
+----------------------------+
| Local LLM (Ollama: Llama3) |
+----------------------------+
|
v
[Contextual Answer]
By running everything locally, we achieve three major wins:
- Zero Latency to Cloud: No waiting on rate-limited APIs or internet bottlenecks.
- Absolute Privacy: Your proprietary source code, API keys, and architectural drafts never leave your local machine.
- Extensibility: You decide exactly what files are indexed and how the model behaves via system prompts.
Setting Up Your Local Environment
Before writing our TypeScript control loop, we need to spin up our local LLM and embedding engine. We'll use Ollama, an incredibly efficient tool for running open-source LLMs locally via llama.cpp.
Step 1: Install and Run Ollama
Download Ollama for your operating system. Once installed, fire up your terminal and pull the two models we will need: our generative model (Llama 3) and our embedding model (mxbai-embed-large).
# Pull the LLM for chat and code generation
ollama run llama3
# Pull the high-performance embedding model
ollama pull mxbai-embed-largeOllama will now host a local web server at http://localhost:11434, exposing an API we can interact with programmatically.
Building the Assistant: Step-by-Step Code
Let's initialize a Node.js project using TypeScript. We'll use LangChain's community packages to orchestrate our document loading, vector storage, and model querying.
Step 2: Project Initialization
mkdir local-scout
cd local-scout
npm init -y
npm install typescript @types/node tsx --save-dev
npx tsc --initNow, install the required LangChain and vector store dependencies:
npm install @langchain/community @langchain/core @langchain/ollama hnswlib-nodeStep 3: Writing the Indexing and Query Pipeline
Create a file named assistant.ts. We will write a script that reads all Markdown and TypeScript files from a targeted directory, indexes them into a local vector database, and allows us to run context-rich queries against them.
import { Ollama, OllamaEmbeddings } from "@langchain/ollama";
import { HNSWLib } from "@langchain/community/vectorstores/hnswlib";
import { RecursiveCharacterTextSplitter } from "langchain/text_splitter";
import { DirectoryLoader } from "langchain/document_loaders/fs/directory";
import { TextLoader } from "langchain/document_loaders/fs/text";
import * as path from "path";
// Initialize our Local Ollama instances
const ollamaLLM = new Ollama({
baseUrl: "http://localhost:11434",
model: "llama3",
temperature: 0.2, // Low temperature for more deterministic, accurate technical answers
});
const ollamaEmbeddings = new OllamaEmbeddings({
baseUrl: "http://localhost:11434",
model: "mxbai-embed-large",
});
async function buildAndQueryAssistant() {
const targetDocsDir = path.join(__dirname, "docs");
console.log("š Loading local documentation and codebase files...");
// Load local text/markdown files containing our project documentation
const loader = new DirectoryLoader(targetDocsDir, {
".md": (filePath) => new TextLoader(filePath),
".txt": (filePath) => new TextLoader(filePath),
});
const rawDocs = await loader.load();
if (rawDocs.length === 0) {
console.error("❌ No documents found in the ./docs folder. Please add some markdown files.");
return;
}
console.log(`š Loaded ${rawDocs.length} documents. Splitting into chunks...`);
// Split documents into manageable chunks for the LLM's context window
const splitter = new RecursiveCharacterTextSplitter({
chunkSize: 800,
chunkOverlap: 100,
});
const docs = await splitter.splitDocuments(rawDocs);
console.log(`⚡ Generating embeddings and saving to local vector store...`);
// Vectorize the text chunks and load them into our memory-mapped vector store
const vectorStore = await HNSWLib.fromDocuments(docs, ollamaEmbeddings);
// Define a prompt template that enforces strict contextual answering
const query = "How do we handle user authentication and token rotation in this project?";
console.log(`\nš User Query: "${query}"`);
// Search our local vector DB for relevant context chunks
const searchResults = await vectorStore.similaritySearch(query, 3);
const contextText = searchResults
.map((doc, idx) => `[Context ${idx + 1}]:\n${doc.pageContent}`)
.join("\n\n");
const systemPrompt = `
You are a highly capable local software engineering assistant.
Use the following pieces of context from our codebase/documentation to answer the user's question.
If you do not know the answer based on the context, state that you don't know. Do not make things up.
---
CONTEXT:
${contextText}
---
QUESTION: ${query}
ANSWER:
`;
console.log("š¤ Querying local Llama3 instance...");
const response = await ollamaLLM.invoke(systemPrompt);
console.log("\n=================== ASSISTANT RESPONSE ===================");
console.log(response);
console.log("==========================================================");
}
buildAndQueryAssistant().catch(console.error);Running the Code (And Seeing It in Action)
To see this actually work, create a docs folder in your project root and add a markdown file detailing a fictional or real architectural flow for your application.
mkdir docs
echo "# Authentication Architecture
Our application uses JSON Web Tokens (JWT) for authentication.
Access tokens have a lifespan of 15 minutes.
Token rotation is managed via HTTP-only secure cookies containing the refresh token.
When the access token expires, the client calls the '/api/auth/refresh' endpoint to get a new pair of tokens." > docs/auth_spec.mdNow, run your assistant with tsx:
npx tsx assistant.tsIn your console, you'll see the loader parsing your directory, embedding the content, performing a vector search to pull the most relevant details from auth_spec.md, and feeding that exact context to Llama 3 to formulate a highly targeted, accurate response—all locally.
Why This Beats Proprietary "Addictive" Ecosystems
While tech giants try to hook consumers on cloud services that monetize their daily workflows, developer tools require precision, reproducibility, and security. Building your own stack offers key advantages that cloud-native agents like Scout cannot match:
1. Exact Context Injection
System-wide assistants try to ingest everything from your calendar to your cat photos. Our local tool can be tailored down to a specific subdirectory, a single branch, or even a database schema. You feed the model only what is relevant to the task at hand.
2. Deterministic Tooling
By controlling the temperature parameter in our code (keeping it low, around 0.1 to 0.3), we minimize hallucinations. Microsoft's consumer assistants are optimized for creative engagement (keeping you clicking and scrolling); we optimize for engineering accuracy.
3. No Subscription Fees or Telemetry
Once you download Ollama and your weights, you have zero marginal costs. You can run millions of queries, compile huge codebases into your vector store, and maintain absolute confidentiality.
Conclusion: The Future of Developer Tooling is Local
Instead of waiting to see how intrusive "Scout" or other OS-integrated assistants become, we have the agency to build our own automated workspaces right now. Utilizing the power of open-source models like Llama 3 and embedding utilities like LangChain, you can easily implement a powerful developer assistant that keeps your IP safe, fits cleanly into your CLI terminal, and helps you write better code.
As developers, our superpower is building the tools we want to see in the world. Let the consumer market get hooked on proprietary clouds—we'll keep compiling, indexing, and optimizing our own local models.
Over to You!
Have you experimented with running local LLMs like Llama 3 or Mistral for your development environment? What's your favorite way to index codebases? Let me know in the comments below, or share your thoughts on Twitter/X by tagging @sysseder!