If you've been hanging around developer forums lately, you’ve probably noticed a shift in how we talk about Large Language Models (LLMs). The initial awe of "look, it wrote a basic React component!" has worn off. Today, we are asking much harder questions: How do we actually use these probabilistic engines to solve deterministic, highly complex problems? How do we scale them beyond single-prompt chats?
This week, a fascinating project caught the eye of the developer community on Hacker News: solving 20 open or notoriously difficult Erdős mathematics problems using 20 Codex (and modern LLM) accounts running in parallel. For the uninitiated, Paul Erdős was one of the most prolific mathematicians of the 20th century, famous for posing elegant but deceptively brutal problems in combinatorics, graph theory, and number theory.
Solving these isn't a matter of copy-pasting a prompt. It requires a fundamental shift in system architecture: treating LLMs not as "all-knowing oracles," but as highly parallelizable, noisy heuristic generators coupled with rigid, deterministic verifiers. As software engineers, DevOps practitioners, and system architects, this pattern is incredibly important. It points directly to the future of agentic workflows and automated coding tools. Let's dive into how this architecture works, why it succeeds where single prompts fail, and how you can apply these patterns to your own engineering challenges.
The Architecture: Generator-Verifier Loops
If you ask GPT-4 to solve a novel mathematical proof or find a bug in a highly complex, multithreaded database engine, it will often fail. It hallucinated a library, missed a edge case, or simply lost its train of thought halfway through the token generation. This is because LLMs lack a "system 2" thinking process—they cannot pause, reflect, and run test cases in their heads before speaking.
To solve 20 Erdős problems, researchers didn't just ask the AI for the answers. They built a distributed system based on the Generator-Verifier (or Predict-Evaluate) loop. The architecture looks like this:
+---------------------------------------------------------+
| |
| Orchestrator |
| |
+------------+--------------------+----------------------+
| ^
| 1. Prompt/Task | 4. Score/Feedback
v |
+------------+----+ +--------+--------+
| | | |
| LLM Generators | ---> | Code Verifiers |
| (20x Parallel) | | (Python/Z3/SAT) |
| | | |
+-----------------+ +-----------------+
|
| 3. Execution/Proof Check
v
+----+----+
| Success |
+---------+
In this architecture, the workload is split into two distinct execution environments:
- The Generator (Probabilistic): A massive swarm of parallel LLM instances running in parallel. Their job is to propose candidate solutions, code implementations, or mathematical counterexamples. They are optimized for high throughput and high temperature (creativity).
- The Verifier (Deterministic): A local sandbox running fast, deterministic code. This could be a Python runtime, a unit test runner, a SAT solver (like Z3), or a compiler. Its job is to ruthlessly execute, verify, and grade the generator's output.
Why Parallelism is the Secret Sauce
Why run 20 accounts in parallel? It comes down to search space exploration. In mathematics and complex software engineering, the state space of potential solutions is vast. If a model has a 1% chance of generating a working solution on any single run, the probability of getting a correct answer in 1 attempt is, well, 1%.
But what if we run 1,000 attempts in parallel across dozens of concurrent API threads? The probability of success jumps exponentially:
P(success) = 1 - (1 - p)^n
Where p is the single-shot success rate (0.01) and n is the number of parallel runs (1000). Suddenly, your probability of finding a working solution climbs to over 99.99%. By scale-out parallelization, we transform a flaky, unreliable AI into an incredibly robust problem-solving engine. The bottleneck is no longer the intelligence of the model; it is our ability to orchestrate, parallelize, and verify at scale.
Building a Mini-Generator-Verifier in Python
Let's bring this down to earth. How do we build this type of system? Suppose we want to solve a complex coding puzzle or find a specific algorithmic implementation that satisfies a set of strict conditions. We can write an orchestrator that spins up parallel requests to an LLM, parses the code, runs it against a local test suite, and feeds back failures to iterate.
Here is a simplified, practical implementation using Python's concurrent.futures to show how you can run parallel generator tasks and verify them deterministically.
import concurrent.futures
import openai
import sys
import subprocess
# The system prompt tells the LLM to output ONLY raw Python code
SYSTEM_PROMPT = """
You are an expert algorithmic generator.
Provide ONLY valid Python code containing a function named 'solve(n)'.
Do not include markdown blocks, explanation text, or usage examples.
"""
# Let's say we want to find a function that calculates the nth Fibonacci
# number, but with a highly specific constraint (e.g., must be O(log n) time).
PROBLEM_PROMPT = """
Write a Python function 'solve(n)' that calculates the nth Fibonacci number.
It must run in O(log n) time complexity using matrix exponentiation.
"""
def query_llm_generator(prompt_text):
"""Queries the LLM for a potential code solution."""
client = openai.OpenAI()
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": prompt_text}
],
temperature=0.7 # High temperature for diverse approaches
)
return response.choices[0].message.content.strip()
def verify_solution(code_string):
"""Deterministically verifies the generated code against unit tests."""
# Clean up markdown code blocks if the LLM ignored instructions
if "```python" in code_string:
code_string = code_string.split("```python")[1].split("```")[0]
elif "```" in code_string:
code_string = code_string.split("```")[1].split("```")[0]
test_harness = f"""
{code_string}
# Deterministic verification suite
try:
assert solve(0) == 0, "Failed test for n=0"
assert solve(1) == 1, "Failed test for n=1"
assert solve(10) == 55, "Failed test for n=10"
assert solve(30) == 832040, "Failed test for n=30"
print("SUCCESS")
except Exception as e:
print(f"FAILED: {{e}}")
"""
try:
# Run the code in a restricted subprocess for safety
result = subprocess.run(
[sys.executable, "-c", test_harness],
capture_output=True,
text=True,
timeout=5
)
if "SUCCESS" in result.stdout:
return True, code_string
else:
return False, result.stdout + "\n" + result.stderr
except Exception as e:
return False, str(e)
def run_parallel_search(workers=10):
"""Orchestrates parallel generation and verification."""
print(f"Starting parallel search with {workers} workers...")
with concurrent.futures.ThreadPoolExecutor(max_workers=workers) as executor:
# Submit multiple parallel generator jobs
futures = {executor.submit(query_llm_generator, PROBLEM_PROMPT): i for i in range(workers)}
for future in concurrent.futures.as_completed(futures):
worker_id = futures[future]
try:
candidate_code = future.result()
print(f"[Worker {worker_id}] Generated candidate solution. Verifying...")
success, output = verify_solution(candidate_code)
if success:
print(f"\n🎉 SUCCESS! Worker {worker_id} found a working solution:")
print("-" * 40)
print(output)
print("-" * 40)
# Cancel other running tasks if we found our answer
executor.shutdown(wait=False, cancel_futures=True)
return
else:
print(f"[Worker {worker_id}] Verification failed: {output.strip()}")
except Exception as exc:
print(f"[Worker {worker_id}] Generated an exception: {exc}")
if __name__ == "__main__":
run_parallel_search(workers=5)
From Code Generation to System Verification
While the Erdős project focused on mathematics, this paradigm is highly applicable to modern software development and platform engineering. Here are a few places where this generator-verifier pattern is already changing the game:
1. Automated Vulnerability Patching
Instead of manually writing a patch for an identified security vulnerability, a security agent can spin up 20 parallel LLM instances to rewrite the vulnerable code block. A local test suite—including static analysis tools (like SonarQube or Snyk) and unit tests—verifies the patches. Only the patches that pass security compilation, don't break existing tests, and maintain performance are presented to developers as PRs.
2. IaC & Cloud Architecture Validation
Writing complex Terraform modules or Kubernetes manifests can be error-prone. A developer platform can spin up parallel generation threads to write infrastructure code based on natural language requirements. The verifier runs tools like tflint, tfsec, or kubeval in a localized container to assert compliance before committing.
3. Database Query Optimization
Need to optimize a slow SQL query with multiple joins? Generate 10 alternative queries using an LLM swarm, run them against a staging database instance, analyze the EXPLAIN ANALYZE execution paths, and automatically select the query with the lowest cost and index usage.
Conclusion & The Shift to Orchestration
The lesson from the Erdős project is clear: we are moving from prompt engineering to system orchestration. Writing a better prompt is a linear improvement; building a parallelized, self-correcting system of models coupled with deterministic verifiers is an exponential leap.
As developers, our roles are evolving. We aren't just writing the code ourselves, nor are we just copying and pasting from ChatGPT. Increasingly, our job is to design the orchestrators, build the testing sandboxes, write the rigorous verifiers, and manage the feedback loops that allow these models to safely and reliably do the heavy lifting for us.
Have you experimented with multi-agent workflows, code verifiers, or parallel LLM processing in your development pipelines? How do you handle safety when running AI-generated code in sandboxes? Let me know in the comments below, or share this article with your DevOps team to start the conversation!