Hey everyone, Alex here. Welcome back to "Coding with Alex" at sysseder.com.
If you scrolled through Hacker News today, you probably saw a thread that immediately triggers a wave of nostalgia and excitement for many of us: "Cyberpunk Comics, Manga and Graphic Novels". As developers, many of us practically live and breathe the cyberpunk aesthetic. From the neon-drenched streets of Akira and Ghost in the Shell to the dystopian, high-tech, low-life themes of Neuromancer, these narratives shape how we think about the future of technology, AI, and security.
But as I was browsing through the incredible list of recommendations in that thread, a technical challenge hit me. Finding a comic book based on its vibe, visual style, or obscure plot points is notoriously difficult with traditional keyword-based search. If you search a standard relational database for "cybernetic detective in a rain-slicked city looking for AI," and those exact words aren't in the metadata, you get zero results.
Today, we are going to bridge the gap between our love for cyberpunk culture and modern database engineering. We are going to build a semantic, vector-based search engine for graphic novels using PostgreSQL, the pgvector extension, and Node.js. By the end of this post, you'll know how to generate vector embeddings from textual descriptions, store them in Postgres, and perform high-performance similarity searches to find your next favorite cyberpunk read.
Understanding the Architecture: Keyword vs. Semantic Search
Before we dive into the code, let's look at why traditional database searches fail us here and how vector search solves it.
In a traditional SQL database, you might write a query like this:
SELECT * FROM comics WHERE description LIKE '%cybernetic%';This works if the author explicitly wrote "cybernetic." But what if the description says "prosthetic limbs," "neural interfaces," or "bionic implants"? Human language is full of synonyms, metaphors, and context.
Vector search solves this by translating text into embeddings. An embedding is a vector (a list of floating-point numbers) generated by a machine learning model. This vector represents the semantic meaning of the text in a high-dimensional space. Words and phrases with similar meanings are mapped close to each other in this space.
Here is our system architecture for this project:
+-------------------------------------------------------------+
| DATA INGEST |
| Comic Metadata -> OpenAI text-embedding-3-small -> Vector |
+-------------------------------------------------------------+
|
v
+-------------------------------------------------------------+
| DATABASE |
| PostgreSQL with pgvector extension |
| Stores: Title, Author, Description, Embedding |
+-------------------------------------------------------------+
^
| (Cosine Similarity Query)
+-------------------------------------------------------------+
| SEARCH FLOW |
| User Query -> Generate Embedding -> Query DB -> Results |
+-------------------------------------------------------------+
Setting Up Our Environment
To follow along, you will need Docker installed (to run PostgreSQL with the pgvector extension) and Node.js (v18 or higher).
1. Spin up PostgreSQL with pgvector
Thankfully, the open-source community maintains an official Docker image for PostgreSQL that comes pre-packaged with the pgvector extension. Let's run it:
docker run --name cyberpunk-db \
-e POSTGRES_USER=alex \
-e POSTGRES_PASSWORD=supersecret \
-e POSTGRES_DB=comic_hub \
-p 5432:5432 \
-d pgvector/pgvector:pg162. Initialize the Node.js Project
Next, let's set up our Node.js environment. Create a new directory and install our dependencies. We will use the official pg client for Postgres and the @google/genai or openai SDK to generate our embeddings. For this guide, we'll use the OpenAI API for embedding generation.
mkdir cyberpunk-search
cd cyberpunk-search
npm init -y
npm install pg openai dotenvDatabase Schema and Vector Configuration
Now, let's connect to our Postgres instance and configure the database. We need to do three things: enable the vector extension, create our table, and define the vector column size.
OpenAI's text-embedding-3-small model outputs vectors with 1536 dimensions. Therefore, our vector column must be configured as vector(1536).
Create a file named init.sql:
-- Enable the pgvector extension
CREATE EXTENSION IF NOT EXISTS vector;
-- Create the comics table
CREATE TABLE comics (
id SERIAL PRIMARY KEY,
title VARCHAR(255) NOT NULL,
author VARCHAR(255) NOT NULL,
description TEXT NOT NULL,
embedding vector(1536) -- 1536 dimensions for OpenAI embeddings
);
-- Create an HNSW index for high-speed similarity search
CREATE INDEX ON comics USING hnsw (embedding vector_cosine_ops);Let's talk about that last SQL command. The HNSW (Hierarchical Navigable Small World) index is crucial for production environments. Without an index, Postgres must perform an exact nearest neighbor search (flat search), comparing your query vector against every single row in the database. While fine for a few hundred comics, this becomes a major bottleneck with millions of rows. HNSW builds a multi-layered graph structure that allows for rapid, approximate nearest-neighbor queries using cosine distance.
The Node.js Implementation
With our database ready, let's write our Node.js code. First, create a .env file in your root folder to store your credentials:
PGUSER=alex
PGPASSWORD=supersecret
PGHOST=localhost
PGPORT=5432
PGDATABASE=comic_hub
OPENAI_API_KEY=your_openai_api_key_hereStep 1: Database Connection and Embedding Helper
Let's write a utility script, db.js, to handle database pooling and interface with OpenAI's API to generate embeddings.
// db.js
const { Pool } = require('pg');
const { OpenAI } = require('openai');
require('dotenv').config();
const pool = new Pool({
user: process.env.PGUSER,
host: process.env.PGHOST,
database: process.env.PGDATABASE,
password: process.env.PGPASSWORD,
port: process.env.PGPORT,
});
const openai = new OpenAI({
apiKey: process.env.OPENAI_API_KEY,
});
/**
* Generates a 1536-dimension vector embedding for a given text.
*/
async function getEmbedding(text) {
try {
const response = await openai.embeddings.create({
model: 'text-embedding-3-small',
input: text,
});
return response.data[0].embedding;
} catch (error) {
console.error('Error generating embedding:', error);
throw error;
}
}
module.exports = { pool, getEmbedding };Step 2: Seeding Cyberpunk Data
Now, let's create a seeding script (seed.js) to populate our database with some iconic cyberpunk graphic novels. We will generate embeddings for each comic's description dynamically before saving them to Postgres.
// seed.js
const { pool, getEmbedding } = require('./db');
const cyberpunkComics = [
{
title: "Ghost in the Shell",
author: "Masamune Shirow",
description: "In a futuristic Japan, Major Motoko Kusanagi, a cyborg counter-cyberterrorist field commander, investigates the mysterious Puppet Master, a malicious hacking entity that challenges the nature of consciousness and identity."
},
{
title: "Akira",
author: "Katsuhiro Otomo",
description: "Set in a post-apocalyptic Neo-Tokyo, a teenage biker gang leader tries to save his telekinetic friend Kaneda from a secret government military project that threatens to unleash catastrophic power."
},
{
title: "Transmetropolitan",
author: "Warren Ellis",
description: "A gonzo journalist named Spider Jerusalem battles corruption, political decay, and technological obsession in a bizarre, sprawling future metropolis using nothing but truth and cyber-enhanced reporting."
},
{
title: "Blame!",
author: "Tsutomu Nihei",
description: "A silent wanderer named Killy travels through a massive, endless subterranean city of concrete and steel known as the Megastructure, searching for human Net Terminal Genes to stop the rogue AI defenses."
}
];
async function seed() {
const client = await pool.connect();
try {
console.log("Starting data seeding...");
for (const comic of cyberpunkComics) {
console.log(`Generating embedding for: ${comic.title}`);
// 1. Generate the semantic vector representation
const embedding = await getEmbedding(comic.description);
// 2. Insert into PostgreSQL. Note the pgvector format parsing:
// We must format the JavaScript float array into a string representation: '[0.12, -0.45, ...]'
const vectorString = `[${embedding.join(',')}]`;
await client.query(
'INSERT INTO comics (title, author, description, embedding) VALUES ($1, $2, $3, $4)',
[comic.title, comic.author, comic.description, vectorString]
);
}
console.log("Seeding completed successfully!");
} catch (err) {
console.error("Error seeding database:", err);
} finally {
client.release();
await pool.end();
}
}
seed();Run the seeding script using node seed.js. Make sure you've executed the init.sql file against your database first to create the schema.
Executing the Semantic Search
Now comes the magic. We want to search for comics using natural queries like "a story about a reporter fighting dirty politicians" or "existential dread of a cyborg".
In our SQL query, we use the cosine distance operator <=> provided by pgvector. Cosine distance calculates the angle between two vectors. A distance of 0 means the vectors are identical, while a distance of 2 means they are complete opposites. Therefore, sorting by distance in ascending order (ORDER BY embedding <=> $1 ASC) gives us the most semantically similar matches first.
Let's create search.js:
// search.js
const { pool, getEmbedding } = require('./db');
async function searchComics(userQuery) {
const client = await pool.connect();
try {
console.log(`\nUser Query: "${userQuery}"`);
// 1. Convert user query to vector
const queryEmbedding = await getEmbedding(userQuery);
const vectorString = `[${queryEmbedding.join(',')}]`;
// 2. Query Postgres using cosine distance
const queryText = `
SELECT title, author, description, (embedding <=> $1) as distance
FROM comics
ORDER BY embedding <=> $1 ASC
LIMIT 2;
`;
const res = await client.query(queryText, [vectorString]);
console.log("\n--- Top Semantic Matches ---");
res.rows.forEach((row, index) => {
console.log(`${index + 1}. ${row.title} by ${row.author} (Distance: ${parseFloat(row.distance).toFixed(4)})`);
console.log(` Description: ${row.description}\n`);
});
} catch (err) {
console.error("Search failed:", err);
} finally {
client.release();
await pool.end();
}
}
// Get query from command line arguments or use default
const searchQuery = process.argv[2] || "searching for rogue artificial intelligence in a massive concrete megacity";
searchComics(searchQuery);Running the Search
Let's run some tests to see how the semantic engine performs compared to simple keyword matching.
node search.js "existential dread of a robot cop"Output:
1. Ghost in the Shell by Masamune Shirow (Distance: 0.3842)
Description: In a futuristic Japan, Major Motoko Kusanagi, a cyborg counter-cyberterrorist...
2. Blame! by Tsutomu Nihei (Distance: 0.4912)
Description: A silent wanderer named Killy travels through a massive, endless subterranean city...Notice how our query didn't contain the word "cyborg", "Major", or "Japan". Yet, our model successfully identified that "robot cop" and "existential dread" closely aligned with the themes of Ghost in the Shell. That is the raw power of vector search in action.
Best Practices and Production Pitfalls
While pgvector is an incredible tool, deploying it to production requires understanding a few nuances:
- Memory Limits: The HNSW index is built and searched entirely in RAM. If your vector index exceeds your database's allocated RAM, performance will degrade dramatically. Ensure you adjust Postgres's
shared_buffersandwork_memconfiguration parameters accordingly. - Normalization: If you are using Cosine Distance, normalize your vectors before storing them. When vectors are normalized (i.e., their length is 1), cosine distance calculations can be computed using a simple inner product, which is significantly faster.
- Dimension Sizes: Be mindful of your embedding model. Newer models like OpenAI's
text-embedding-3-largesupport custom dimensions (up to 3072). Higher dimensions capture more nuances but increase storage size and slow down index generation.
Wrapping Up
By marrying our passion for cyberpunk storytelling with cutting-edge database technology, we've bypassed the limits of legacy keyword matching. Whether you are cataloging classic graphic novels, indexing software documentation, or building a RAG (Retrieval-Augmented Generation) application for your company's internal tools, pgvector is an indispensable asset in a backend engineer's modern toolbelt.
What are your thoughts on using relational databases like Postgres for vector workflows versus dedicated vector databases like Pinecone or Milvus? And more importantly, what cyberpunk comic are we reading next? Let me know in the comments below!
Until next time, keep coding, keep learning, and stay secure.