We’ve all been there. You’re building an internal tool, a smart home integration, or a quirky office app, and you need to solve a deceptively simple problem: Is that person in the camera feed actually who they say they are? Or, more specifically, in the case of a recent project trending on Hacker News: Is It Greg?
While the "Is It Greg?" project started as a lighthearted, highly specific solution to a local problem, it shines a spotlight on a massive trend in modern software engineering: local, privacy-first, low-latency edge AI. A few years ago, if you wanted to build a facial verification or image classification system, you had to spin up an expensive AWS Rekognition pipeline, deal with API latencies, worry about egress costs, and navigate the sticky ethical waters of sending user biometric data to third-party cloud providers.
Today, things are different. Thanks to lightweight neural networks, highly optimized runtimes like ONNX and TensorFlow Lite, and powerful web standards, we can run sophisticated image classification and facial recognition directly on the edge—even entirely within the client’s browser. In this post, we’re going to dissect how to build your own version of "Is It Greg?" (or "Is It a System Administrator?", "Is It My Dog?", etc.) using modern, developer-friendly open-source tools.
The Architecture of Local Image Recognition
Before we write any code, let's look at the architecture of a modern, local image recognition system. Unlike heavy cloud-based setups, a local pipeline needs to be lean, fast, and resource-efficient. Here is how the data flows from a user's camera to a classification decision:
[Camera/Video Stream]
│
▼
[Frame Capture (HTML5 Canvas / OpenCV)]
│
▼
[Pre-processing (Resizing, Normalization, Grayscale)]
│
▼
[Local Inference Engine (ONNX Runtime / TensorFlow.js)]
│
▼
[Feature Extraction (Embeddings) / Classification (Is it Greg? Yes/No)]
│
▼
[Application State & UI Update]
To achieve this locally, we typically rely on two core components:
- The Model: A lightweight convolutional neural network (CNN) or a vision transformer (ViT). For edge devices and browsers, models like MobileNetV3 or EfficientNet are the gold standard. They sacrifice a tiny fraction of accuracy for massive gains in speed and reduced memory footprints.
- The Runtime: A library that can execute the model on local hardware, leveraging GPU acceleration (like WebGL or WebGPU in the browser, or CoreML/NNAPI on mobile devices) without needing a Python backend.
Step-by-Step: Building a Local Verification Engine
Let’s build a practical, browser-based image classifier. We’ll use JavaScript and TensorFlow.js to load a pre-trained MobileNet model, feed it a webcam stream, and determine if the object or person in front of the camera matches our target profile.
Step 1: Setting Up the HTML Structure
First, we need a simple interface to display our webcam feed and show the classification results in real-time. Here is our clean, boilerplate HTML:
<div class="scanner-container">
<h1>Identity Verification Portal</h1>
<div class="video-wrapper">
<video id="webcam" autoplay playsinline width="640" height="480"></video>
<canvas id="overlay" width="640" height="480"></canvas>
</div>
<div class="results-panel">
<p>Status: <span id="status">Loading Model...</span></p>
<h2 id="prediction">Analyzing...</h2>
</div>
</div>
Step 2: Accessing the Camera Safely
Modern browsers are highly restrictive about camera permissions. We need to request access to the user's media devices securely, ensuring we handle errors gracefully if the user denies access or is on an insecure origin (HTTP vs. HTTPS).
async function setupWebcam() {
const video = document.getElementById('webcam');
try {
const stream = await navigator.mediaDevices.getUserMedia({
video: { width: 640, height: 480, facingMode: "user" },
audio: false
});
video.srcObject = stream;
return new Promise((resolve) => {
video.onloadedmetadata = () => {
resolve(video);
};
});
} catch (error) {
console.error("Camera access denied or unavailable:", error);
document.getElementById('status').innerText = "Camera Error";
throw error;
}
}
Step 3: Loading the Local Model and Running Inference
Now, let's load the machine learning model. Instead of calling an external API, TensorFlow.js downloads the model weights once and caches them in the user's browser. Inference happens entirely on the client's GPU.
// Import TensorFlow.js and MobileNet via CDN or npm
// <script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs"></script>
// <script src="https://cdn.jsdelivr.net/npm/@tensorflow-models/mobilenet"></script>
let model;
async function initApp() {
document.getElementById('status').innerText = "Loading MobileNet...";
// Load the model locally in the browser
model = await mobilenet.load({ version: 2, alpha: 1.0 });
document.getElementById('status').innerText = "Model Active. System Ready.";
const video = await setupWebcam();
video.play();
// Start our real-time inference loop
runInference(video);
}
async function runInference(video) {
const predictionElement = document.getElementById('prediction');
async function predictLoop() {
// Classify the current video frame
const predictions = await model.classify(video);
if (predictions.length > 0) {
// Check for our target "Greg" (or whatever label you are scanning for)
const topPrediction = predictions[0];
const probability = (topPrediction.probability * 100).toFixed(1);
// Let's assume we are verifying if the subject is a "coffee mug" for this demo
const isTarget = topPrediction.className.toLowerCase().includes("mug");
if (isTarget) {
predictionElement.innerHTML = `<span style="color: #2ecc71">TARGET IDENTIFIED (${probability}%)</span>`;
} else {
predictionElement.innerHTML = `<span style="color: #e74c3c">UNKNOWN SUBJECT (${topPrediction.className})</span>`;
}
}
// Request the next animation frame for a smooth 30-60 FPS loop
requestAnimationFrame(predictLoop);
}
predictLoop();
}
// Start the application when DOM is loaded
window.addEventListener('DOMContentLoaded', initApp);
Why Local-First ML is a Game Changer for Devs
The "Is It Greg?" philosophy is highly practical. As software engineers, we often default to over-engineering. We build microservices, queue systems, and data pipelines for tasks that could easily be handled with a tiny library and a few lines of JavaScript. Here is why local edge classification is becoming the default choice for modern apps:
1. Absolute Privacy by Design
When you handle biometric data, facial features, or video feeds of people's private spaces, compliance frameworks like GDPR and CCPA become a massive headache. By keeping the video stream and the machine learning model entirely within the client's runtime environment, zero data leaves their device. Your servers never see the images, which means your liability drops to zero.
2. Low Latency, No Network Dependence
Network latency is the enemy of great UX. Round-tripping an image to a cloud server takes anywhere from 200ms to several seconds depending on the user's connection. A optimized ONNX or TFJS model running via WebGL/WebGPU can run inference in under 16 milliseconds (60 FPS), enabling real-time interactions, smooth overlays, and instant feedback.
3. Zero Infrastructure Costs
Cloud AI APIs are expensive. Running a dedicated GPU instance on AWS or paying per-call API rates can quickly bloat your monthly bill if your application scales. By delegating the compute load to the user's processor (CPU/GPU), your infrastructure costs scale linearly at exactly $0.00.
Going Beyond: Custom Training with Transfer Learning
You might be asking: "MobileNet is great for identifying generic objects like 'mug' or 'cat', but how do I make it recognize a specific person, like my coworker Greg?"
This is where Transfer Learning comes in. You don't need to train a massive model from scratch. Instead, you can freeze the lower layers of a pre-trained model (which already know how to identify basic shapes, edges, and textures) and train a tiny classifier on top of those embeddings using just 10 to 20 photos of your target subject. Tools like KNN Classifier in TensorFlow.js allow you to do this directly in the browser in real-time. You can have a "Train" button that captures frames of "Greg", saves the vector embeddings locally, and immediately begins classifying him with high accuracy.
Wrapping Up: Simplicity Wins
The "Is It Greg?" project is a great reminder that software development is at its best when we build highly focused, incredibly efficient solutions to real-world problems. Whether you are building smart security systems, UX improvements, or just a fun office hack, the tools to run powerful AI locally on the edge are ready for production today.
What are you building that could benefit from local, low-latency image recognition? Are you ready to ditch the heavy cloud APIs and migrate your classification pipelines to the edge? Let’s chat in the comments below!
Happy coding, friends! — Alex