If you've been anywhere near the software engineering space over the last two years, you’ve probably felt a creeping sense of imposter syndrome. Day in and day out, we are bombarded with terms like embeddings, latent space, vector databases, and transformer architectures. We import @langchain/community or call OpenAI's API, and things just... work. But as developers, we hate magic. We want to know what’s happening under the hood.
The secret under the hood of almost all modern engineering breakthroughs—from LLMs and computer graphics to search indexing and game engines—isn't complex calculus. It's Linear Algebra.
This week, Sheldon Axler’s legendary textbook, Linear Algebra Done Right (now in its brand-new, open-access fourth edition), shot to the top of Hacker News. It sparked a massive discussion in the dev community: Why do so many of us struggle with linear algebra, and how do we actually learn it in a way that makes us better engineers?
Today, we're going to dive into why linear algebra is the ultimate developer superpower, how the traditional way of teaching it failed us, and how you can apply its core concepts to real-world code right now.
The Determinant Problem: Why Traditional Math Classes Failed Us
If you took linear algebra in university, you probably remember a soul-crushing barrage of rote calculations. You spent hours computing the determinants of $3 \times 3$ matrices by hand, performing row reduction algorithms (Gaussian elimination), and memorizing formulaic steps to find eigenvalues.
Axler’s book became a cult classic because it takes a radically different approach: it avoids determinants almost entirely until the very end.
Instead of viewing linear algebra as a series of spreadsheet manipulation rules, Axler treats it as the study of linear maps on vector spaces. It’s clean, highly visual, and deeply structural. For a programmer, this is the equivalent of moving from writing spaghetti code with raw multidimensional arrays to designing clean, object-oriented, or functional abstractions.
When you view linear algebra through the lens of a developer, vectors aren't just lists of numbers; they are states. Matrices aren't just grids; they are functions (or transformations) that map one state to another.
From Pixels to LLMs: Where the Code Meets the Math
To understand why this matters, let’s look at three practical areas where modern software developers run face-first into linear algebra.
1. Vector Search and Semantic Similarity
If you are building a modern search engine, a recommendation system, or a Retrieval-Augmented Generation (RAG) pipeline for an AI agent, you are using vector databases like pgvector, Pinecone, or Milvus.
When you generate a text embedding, you are mapping a string of text to a high-dimensional vector space (often 1536 dimensions for OpenAI's text-embedding-3-small). To find "similar" documents, you don't do SQL LIKE queries. You calculate the cosine similarity between two vectors.
Mathematically, the cosine similarity between vector $A$ and vector $B$ is defined as:
similarity = (A · B) / (||A|| * ||B||)Where A · B is the dot product, and ||A|| is the norm (magnitude) of the vector. Here is how we implement this from scratch in TypeScript to understand exactly what the CPU/GPU is doing:
// A simple implementation of Vector operations for Semantic Search
type Vector = number[];
function dotProduct(a: Vector, b: Vector): number {
if (a.length !== b.length) {
throw new Error("Vectors must be of the same dimension");
}
return a.reduce((sum, val, i) => sum + val * b[i], 0);
}
function magnitude(a: Vector): number {
return Math.sqrt(a.reduce((sum, val) => sum + val * val, 0));
}
export function cosineSimilarity(a: Vector, b: Vector): number {
const denom = magnitude(a) * magnitude(b);
if (denom === 0) return 0; // Avoid division by zero
return dotProduct(a, b) / denom;
}
// Example usage: Comparing three documents
const docQuery = [0.12, 0.85, -0.04]; // "How to scale Postgres"
const docPostgres = [0.15, 0.82, -0.01]; // "PostgreSQL scaling strategies"
const docCooking = [-0.78, 0.11, 0.54]; // "Best chocolate chip cookie recipe"
console.log(cosineSimilarity(docQuery, docPostgres)); // Output: ~0.996 (Highly similar)
console.log(cosineSimilarity(docQuery, docCooking)); // Output: ~0.021 (Unrelated)
When you run this code, you are executing linear algebra. Scale this up to millions of vectors, and you need specialized indexing structures like HNSW (Hierarchical Navigable Small World) graphs, which are built entirely on vector space geometry.
2. Graphics, CSS Transforms, and Game Dev
Ever wondered how CSS transform: matrix3d(...) works, or how game engines rotate a 3D camera?
A 3D point is represented as a 3-element vector (or 4-element homogeneous coordinate). To rotate, scale, or translate that point, we multiply it by a transformation matrix. If you chain multiple operations together—say, spinning a character's sword while the character is running inside a moving train—you don't calculate each step sequentially. You multiply the transformation matrices together first:
CombinedMatrix = Translation * Rotation * ScaleBecause matrix multiplication is associative (but not commutative!), you can pre-calculate this single CombinedMatrix on the CPU and send it to the GPU, which applies it to millions of vertices in parallel. This is why GPUs exist: they are massively parallel matrix multiplication engines.
Linear Algebra "Done Right" for Developers
If you want to master linear algebra today without getting bogged down in the academic academic-style hand-calculations of the 1980s, here is your roadmap.
Step 1: Build a Mental Model with Visuals
Before you open any textbook, go to YouTube and watch Grant Sanderson’s playlist, The Essence of Linear Algebra on his 3Blue1Brown channel. It is, without exaggeration, the finest educational resource on the internet. Sanderson explains vectors as arrows in space and matrices as transformations of that space. Once you actually see what a matrix multiplication does to a grid, formulas like eigenvalues make intuitive sense.
Step 2: Dive into Axler's "Linear Algebra Done Right"
Now that the fourth edition is open-access, you can download the PDF for free legally. Read it with a notebook next to you. Because it focuses on linear maps and vector spaces, it aligns beautifully with how programmers think about types, inputs, outputs, and abstractions.
Step 3: Implement It in Code
Don't just read—code. Write your own tiny matrix library. Implement matrix addition, multiplication, and transposition. If you are a Python developer, dig into NumPy and understand how vectorization works under the hood to bypass Python's Global Interpreter Lock (GIL).
Here is a quick Python example showing the difference between standard loops and vectorized NumPy operations. Vectorization allows your CPU to use SIMD (Single Instruction, Multiple Data) instructions to execute operations in parallel.
import numpy as np
import time
# Create two massive arrays (10 million elements)
size = 10_000_000
a = np.random.rand(size)
b = np.random.rand(size)
# Traditional Python loop (Slow!)
start = time.time()
dot_loop = sum(x * y for x, y in zip(a, b))
print(f"Loop time: {time.time() - start:.4f} seconds")
# Vectorized NumPy dot product (Blazing fast!)
start = time.time()
dot_np = np.dot(a, b)
print(f"NumPy time: {time.time() - start:.4f} seconds")
On average, the NumPy vectorization is over 100 times faster. Understanding why this works requires knowing that linear algebra operations map directly to contiguous memory layouts and CPU/GPU hardware capabilities.
Conclusion
As software engineers, our tooling is moving higher and higher up the abstraction stack. But the infrastructure powering this stack is digging deeper into mathematics. If you want to move beyond being a consumer of API wrappers and become someone who designs, optimizes, and deploys cutting-edge systems, understanding the math is no longer optional.
Sheldon Axler’s Linear Algebra Done Right isn't just for mathematicians; it's a guide to thinking clearly about multidimensional systems. Download the book, watch some 3Blue1Brown, and start looking at your code through the lens of vector spaces.
Over to you: Have you tried reading Axler’s book? What’s your biggest roadblock when trying to learn the math behind modern AI? Let’s chat in the comments below!