There’s definitely truth to this. I’ve worked with APIs that completely overwhelm agents, especially with older LLMs. Even with current models, a large surface area of tools can push them to wander rather than act decisively. That said, I’m not fully convinced that API-to-MCP is inherently flawed. My position is more nuanced. I’ve quipped “maybe your API is just bad,” but my fuller thought is that it has less to do with whether an API is good or bad and more to do with what kind of experience we want an agent to have.

The reality is that nearly everything about AI engineering is fairly new, including our approaches to designing agent experience. I don’t know if there’s an official term for “Agent Experience” yet, and I’m not trying to coin one here. I imagine someone somewhere is already thinking about Agent Experience (AX?) as a new design discipline that sits beside the traditional UX we’ve built for humans. And refocusing our attention on AX helps us move beyond blanket prescriptions toward a better understanding of when API-to-MCP works and when it doesn’t.

Why Some API Surfaces Fail When Mapped Directly into MCP

The clearest failure mode happens when someone points an MCP generator at a very large, SDK-oriented API. These APIs often contain hundreds of operations, deeply nested resources, pagination helpers, batch endpoints, and internal administrative actions. They were designed for typed programming languages where developers rely on autocompletion, documentation, and compiler checks. They were not designed to be reasoned over by a language model.

If you expose this entire surface directly to an agent, the model has too many choices. Tool selection becomes noisy, the agent explores irrelevant operations, and the number of back-and-forth calls grows. In these situations, where no one intentionally designed for the agent’s experience, the critique is valid. It doesn’t surprise me that a design inherited from assumptions that only make sense for developers may not work well for agents.

But Not All APIs Look Like SDKs

Many real-world APIs are much smaller and built to mirror the task flows that humans perform through a UI. These UI-shaped APIs tend to be pragmatic and intuitive. They have just enough operations to support the interface without producing an overwhelming number of choices. When an agent’s role resembles a task a human might perform in the product, these UI-shaped APIs often align well with what the agent needs.

This is where API-to-MCP becomes an elegant, low-friction pattern. The API already expresses the steps in a workflow: one operation to create something, another to update it, another to attach related items, and another to finalize the task. There’s no obscure domain model to infer, no arbitrary distinctions to decode. The agent can reason about the operations the same way a human would if they were using the UI.

A Concrete Example: My Reporter Application

I saw this firsthand while building my Reporter application. The API behind Reporter was originally structured for a simple UI: create a report, add findings, attach actions, leave comments, and update status. It wasn’t large, and it wasn’t designed as a formal SDK surface. It simply mirrored how a human completed reporting tasks.

When I exposed that API through MCP, the agent handled the full workflow with ease. It created reports, added findings, attached actions to each one, and inserted comments in relevant places. It understood how to reference IDs returned from previous calls and could chain those operations effectively. I didn’t have to invent agent-specific abstractions or redesign anything. The shape of the API already matched the shape of the work.

When API-to-MCP Is Appropriate

API-to-MCP is a valid and often effective pattern when the API surface is small and corresponds to meaningful tasks, when the agent’s responsibilities resemble human workflows, when resource boundaries are clear and easy to reason about, and when operations compose naturally. It also works well when the agent doesn’t need hidden backend capabilities and when we can support the agent with context that explains relationships or expected sequences.

This is when you can introduce agentic value immediately with what you already have. If the API is already a clean representation of domain capabilities, use it. Investing in API design pays dividends twice. A well-structured API serves your human users through the UI, and it serves your agents through MCP. The same clarity that makes an API pleasant for frontend developers to work with makes it navigable for an agent. If your API is a mess, both audiences suffer. If it’s clean, both benefit.

The Real Question: What Experience Should the Agent Have?

The success or failure of API-to-MCP is not determined by whether the API was created for humans or for SDKs. The key question is whether the API surface supports the experience we want the agent to have.

Humans have UX. Agents need AX. UX is supported and expressed through the UI—the visual layer that guides users through interactions, constrains their choices, and provides feedback. AX needs its own supporting structures, but we don’t yet have a fully developed equivalent of the UI for agents. What we have is MCP, which exposes capabilities and schemas, but doesn’t actively guide the agent through workflows the way a UI guides a human.

In the absence of that guiding layer, a smaller, more directed API surface compensates by reducing cognitive load. It limits the agent’s choices and makes the workflow clearer through the shape of the tools themselves. This is exactly what I observed with my Reporter application. The API was clear enough that the agent could navigate the workflow without needing additional guidance structures. The tool surface itself provided enough direction.

Designing AX means asking thoughtful questions. Does this agent work more like a human completing a task? Or is it acting as a system-level orchestrator? Does it need broad domain coverage or a narrow, purpose-built surface? Does it need to understand relationships across resources, or will those be supplied as context? These decisions shape whether an existing API is enough, or whether we need a curated agent-native layer.

A real Agent Experience guiding layer might look more like:

• Workflow orchestration hints

• Contextual tool filtering

• Sequential guidance

• State awareness that limits what’s shown when

Context engineering is a major part of AX as well. It includes how we prime the agent, guide its sequences, and help it interpret responses. Good context complements the tool surface, working together to shape the agent’s understanding and behavior. This deserves its own deeper discussion, but for now, it’s important to recognize context as a key component of agent experience design.

Looking Forward

API-to-MCP should not be dismissed by default. Yes, some APIs are too broad or too developer-oriented to expose directly. But others map precisely to the tasks we want agents to handle. A poor experience in MCP is not evidence that the pattern is inherently flawed. It might indicate API design issues that humans feel too, or it might mean that the agent deserves a different interface altogether.

As AX becomes a more mature discipline, the question will shift from “Should I wrap my API?” to something more architectural: “Does my API already represent the work I want this agent to do?” When the answer is yes, API-to-MCP can be a clean and effective solution for that particular agent’s needs. When the answer is no, we can design agent-native layers that help the agent understand the domain more clearly.

But the tool surface is just one piece of agent experience. Context engineering, workflow guidance, state management, and other dimensions of AX are still emerging areas of thought. We’re early in understanding how to design well for agents, and there’s room for experimentation and new ideas.

Either way, our goal is to create environments where agents can act with increased reliably, interpretably, and purpose. Thinking deliberately about agent experience gives us a framework to make those decisions with confidence.

In a recent post, I argued that separating vectors from relational data creates a separation tax. This shows up as costs you pay in correctness, complexity, performance, and observability. My main thesis is that when embeddings and metadata live together in one database, retrieval becomes a bounded act of reasoning under policy constraints, not just simple recall.

Today, I’m releasing an update to the CRDB Tuning Report Generator that demonstrates this pattern in action with semantic similarity search and access control guardrails all running in a single CockroachDB query.

What’s New

This update adds two major features that work together to prove the co-location thesis:

1. Semantic Similarity Search

Reports and findings now get OpenAI embeddings automatically when created. The system can then find semantically similar content without relying on keyword matching.

Frontend:

• “Find Similar” button on every report detail page

• Dialog showing top 5 similar reports with similarity scores

• Real-time semantic search across existing reports

Backend:

• Auto-embedding via OpenAI text-embedding-3-small (1536 dimensions)

• Vector similarity using CockroachDB’s native VECTOR(1536) type

• Single-query retrieval with the <-> distance operator

Database:

-- Co-located embeddings with metadata
CREATE TABLE reports (
  id UUID PRIMARY KEY,
  title STRING,
  report_text STRING,
  embedding VECTOR(1536),  -- Semantic representation
  customer_id UUID,        -- Access control
  region STRING,           -- Compliance boundary
  pii_flag BOOL,          -- Content guardrail
  status STRING,          -- Workflow state
  ...
);

2. Access Control Demo

Three demo users showcase how guardrails work alongside similarity search:

• Alice Johnson (analyst_alice) – Can only see Acme Corp reports

• Bob Smith (analyst_bob) – Can only see Globex Industries reports

• Charlie Davis (admin_charlie) – Admin with full access to all customers

Switch between users in the UI and watch the similarity results change based on their access level. Same semantic query, different results—because access control happens before retrieval, not after.

Why This Matters: The Single Query

With separated systems, you’d do this:

# Retrieve all, then filter 
all_similar = vector_db.search(embedding, limit=100)
# Now fetch metadata for each to check access...
allowed = []
for report in all_similar:
    metadata = relational_db.get(report.id)
    if user_has_access(user_id, metadata.customer_id):
        allowed.append(report)
    if len(allowed) >= 5:
        break
return allowed

With co-located vectors, it’s one query:

-- Filter and retrieve in one serializable snapshot
WITH authorized AS (
  SELECT customer_id 
  FROM user_access 
  WHERE user_id = $user_id
)
SELECT 
  r.id, 
  r.title, 
  r.customer_id,
  r.embedding <-> $query_embedding AS distance
FROM reports r
WHERE r.customer_id IN (SELECT * FROM authorized)
  AND r.embedding IS NOT NULL
  AND r.status IN ('published', 'in_review')
  AND r.pii_flag = FALSE
ORDER BY distance
LIMIT 5;

What you gain:

• Correctness – Access control enforced before retrieval; no “oops” moments

• Simplicity – One database, one query, one transaction, one truth

• Performance – Prefilter by indexed customer_id, then compute distance on small candidate set

• Observability – EXPLAIN ANALYZE shows the full plan: filters, joins, and vector distance together

Code Change Highlights

The similarity search integrates cleanly with the existing agentic architecture:

Main API (FastAPI) now includes:

• GET /api/v1/reports/{id}/similar – Semantic search with access control

• embedding_service.py – Generates embeddings via OpenAI

• Auto-embedding on report/finding creation

Frontend (React + TypeScript):

• User selector in top navigation (demo users: Alice, Bob, Charlie)

• “Find Similar” button triggers similarity search dialog

• Results filtered by selected user’s access level

• Automatic redirect when switching users on detail pages

Database (CockroachDB):

• VECTOR(1536) columns for embeddings

• Access control via user_access and customers tables

• Single-query retrieval with <-> operator

• ACID transactions guarantee consistency

Get the Code

Repository: github.com/kikiya/agentic-tuning-reporter

git clone https://github.com/kikiya/agentic-tuning-reporter.git
cd agentic-tuning-reporter

Quick Start

See the complete SIMILARITY_SEARCH.md guide. Quick version:

# 1. Install dependencies
cd backend && pip install -r requirements.txt

# 2. Add OpenAI key to backend/.env (You'll need this to generate embeddings)
OPENAI_API_KEY=sk-proj-your-key-here

# 3. Run database migration
cockroach sql --file=schema-add-embeddings.sql

# 4. (Optional) Load demo users for access control
cockroach sql --file=test-data-access-control.sql

# 5. Restart services
cd backend && python main.py

# 6. Test it!
# Open http://localhost:5173 → Any report → Click "Find Similar"
# Switch between users (Alice/Bob/Charlie) to see access control

Core Demonstrations

1. Co-Location Pattern – Embeddings + metadata + guardrails in one database

2. Single-Query Retrieval – Similarity search with access control in one plan

3. ACID Guarantees – Embedding updates atomic with metadata changes

4. Access Control – Customer isolation, PII filtering, region boundaries

5. Observability – EXPLAIN ANALYZE shows the full retrieval plan

6. Demo Architecture – User switching to showcase access control in action

Key Takeaways

From the original “Separation Tax” post:

“When semantics meet relations under ACID, retrieval becomes a bounded act of reasoning as opposed to simple recall. Co-locating vectors with metadata lets you express hypotheses as predicates and distances in the same query plan—what should be visible, which era matters, which regions and roles apply, how to weigh similarity against severity—and settle it with a single, auditable transaction.”

This code release demonstrates that thesis in practice. The result isn’t just “the model said these are similar,” but rather “the database proved these are the most similar items you’re allowed to see, under one serializable snapshot.”

While this is a proof-of-concept with areas for enhancement, the core pattern is pretty solid.

Performance Notes

Cost: OpenAI embeddings cost ~$0.02 per 1M tokens. For typical reports (500-2000 words), this is negligible—a few cents for thousands of reports.

Query Performance:

• With proper indexes (customer_id, status, pii_flag)

• Prefiltering reduces candidates before distance calculation

• Exact distance search is fast on filtered sets (<1000 rows)

• Add ANN indexes when dataset grows large (CockroachDB 24.x+ supports ivfflat)

Contributing & Feedback

This is a learning resource built to help others navigate co-located vector architectures. If you:

• Find bugs or improvements

• Have questions about the implementation

• Want to share how you adapted it

• Built something similar

Open an issue or PR! I’m building this in public to demonstrate practical patterns for “AI as a collaborator” systems with deterministic foundations.

What’s Next

I’ll continue exploring the intersection of:

• Agentic architectures with stateful reasoning

• Co-located vectors for bounded retrieval

• Guardrails that enforce correctness at the database layer

• Human-in-the-loop workflows with AI assistance

Links

• Original “Separation Tax” Blog Post

• GitHub Repository

• Similarity Search Guide

• Base App Quick Start

In a previous post, I argued that agentic applications need deterministic foundations.
The short version: if the top of your stack is intentionally non-deterministic (creative, adaptive AI), the foundation has to be rock-solid.

However, non-determinism isn’t the only cost you pay when you separate vector and relational data in RAG applications. You eventually encounter a separation tax on correctness, simplicity, performance, and observability.

The Cost of Separation

When I first added RAG, I reached for a vector database for embeddings and kept CockroachDB for relational data. On paper, it looked fine:

• structured metadata in CRDB,

• embeddings in a vector store,

• application/agents orchestrating both.

In practice, separation shows up as four recurring costs:

1) Correctness drift (not just dual-writes)

Yes, dual-writes can leave embeddings stale, but another issue is policy and retrieval diverging:

• Permissions/region/PII flags live in relational tables, while retrieval happens elsewhere.

• You end up filtering after retrieval, which means the model may see candidates it shouldn’t or reason from an incomplete snapshot.

2) Complexity

Two systems = two schemas, two backups, two monitoring planes, two incident playbooks.

• You need glue for sync, retries, and idempotency.

• Every feature that touches knowledge now crosses a boundary and doubles in complexity.

3) Performance taxes you can’t index away

• More hops: app → vector DB → app → relational DB (then post-filter).

• Bigger candidate sets: vague prefiltering in the vector tier, precise constraints later.

• Worse tail latency: small percentiles dominate user perception, and each hop compounds it.

4) Observability gaps

You can’t EXPLAIN an end-to-end plan across two engines.

• You see half-plans and infer the rest.

• Fingerprinting and regression tracking break across the boundary.

Once you move out of POC territory, these can become serious headaches. As with any problem, there are many approaches you could take. In my case, I reached for simplicity.

With a relational database that supports Vector types, I’m able to simplify my designs so retrieval, policy, and state coexist to coordinate.

From Separation to Co-Location (What We’ll Build Next)

In my real applications, I was not using the Reporter. But I’ll stick with that as an example, just to keep the context.

Imagine you’re a consultant for a company that specializes in tuning databases. We could update the Reporter App such that it has a feature to perform similarity searches. This would enable you to explore actions and outcomes taken on systems similar to one you’re tuning. But your company may have clients who only want to share data with consultants named on the project. So we’ll need to account for that as well.

For this update, we want to design the system such that:

• Embeddings and metadata live together in a relational database,

• Guardrails (PII, region, role) are enforced inside the retrieval query,

• Writes are atomic (report text + embedding update in a single transaction),

• And observability (e.g., EXPLAIN ANALYZE) applies to the whole plan.

In the next section, we’ll walk through a minimal schema and the core query patterns that make this work:
filter → (join) → order by vector distance, all under one serializable snapshot.

It’s a simplified use case, but I think you’ll find the pattern useful in broader scenarios.

The Co-Located Reporter (Concept)

The Reporter already:

• ingests cluster metrics via API,

• creates structured records (findings, actions, comments) via MCP tools,

• persists everything in CockroachDB,

• and keeps a human reviewer in the loop.

Now we add memory:

1) Embed past reports/findings so the agent can find similar situations.
2) Keep permissions, region/compliance flags, and PII markers alongside those embeddings.
3) Query for “similar cases I’m allowed to see” in one SQL plan.

Minimal Schema (relational + vector, with guardrails)

CREATE TABLE reports (

  id            UUID PRIMARY KEY DEFAULT gen_random_uuid(),

  customer_id   UUID NOT NULL,

  title         STRING,

  created_by    STRING,

  created_at    TIMESTAMP NOT NULL DEFAULT now(),

  pii_flag      BOOL NOT NULL DEFAULT FALSE,

  region        STRING,                 -- e.g., 'US', 'EU'

  crdb_version  STRING,

  severity      STRING,                 -- e.g., 'low'|'moderate'|'high'

  report_text   STRING,

  embedding     VECTOR(1536)            -- co-located semantic representation

);

-- Per-user access control for consultants/analysts

CREATE TABLE user_access (

  user_id    UUID NOT NULL,

  report_id  UUID NOT NULL,

  PRIMARY KEY (user_id, report_id)

);

-- Optional content flags for finer guardrails

CREATE TABLE content_flags (

  report_id UUID NOT NULL,

  flag      STRING NOT NULL,            -- e.g., 'pii', 'restricted_customer'

  PRIMARY KEY (report_id, flag)

);

Observability note: because this all lives in one database, EXPLAIN ANALYZE and query fingerprints apply to the whole retrieval+policy plan.

One Query: Similar Reports I’m Allowed To See

WITH authorized AS (

  SELECT report_id AS id

  FROM user_access

  WHERE user_id = $1

)

SELECT r.id,

       r.title,

       r.customer_id,

       r.region,

       r.severity,

       vector_distance(r.embedding, $2) AS score

FROM   reports AS r

JOIN   authorized a USING (id)

WHERE  r.pii_flag = FALSE

  AND  r.region = $3                  -- region boundary (e.g., match analyst region)

  AND  r.severity IN ('moderate','high')

ORDER  BY score

LIMIT  10;

This is retrieval + guardrails + business filters in a single serializable snapshot:

• If a report is flagged PII or out-of-region, it never becomes a candidate.

• If access is revoked, the join eliminates it before scoring.

• No post-filter “oops” moments.

The Write Path: Atomic, Not Eventual

When the agent (or human) updates a report’s text and re-embeds it, both changes commit or roll back together:

BEGIN;

UPDATE reports

SET report_text=$1,
    embedding=$2,
    severity=$3
WHERE id=$4

COMMIT;

One transaction = one truth.

Performance: Filter → (Join) → Order by Distance

Start with exact search + strong filters:

• Constrain by region, severity, customer, time windows.

• Project only needed columns.

• Order by vector distance last (on a small candidate set).

As data grows, add ANN indexing; until then, filters + exact distance are fast and fully accurate.

Why Co-Location Changes the Shape of RAG

• Correctness: retrieval and policy run under the same plan/snapshot.

• Simplicity: one schema, one backup, one monitoring plane.

• Performance: prefilter with indexes; compute distance on fewer rows.

• Observability: EXPLAIN ANALYZE shows the whole picture, not just half.

In other words, RAG becomes a relational capability the Reporter can rely on deterministically.

When semantics meet relations under ACID, retrieval becomes a bounded act of reasoning as opposed to a simple recall. Co-locating vectors with metadata lets you express hypotheses as predicates and distances in the same query plan—what should be visible, which era matters, which regions and roles apply, how to weigh similarity against severity—and settle it with a single, auditable transaction.

The result isn’t just “a model said so,” but “the system proved it under one snapshot.” That’s the qualitative shift: rather than just fetching context, the agent reasons over state and the database provides the guardrails, evidence, and durability that make those judgments more trustworthy.

This “AI slop” is a new slur for our super-charged machine age. It’s what we call the weird-fingered images, the off-kilter text, the uncanny attempts at art or writing that feel like someone pressed generate and walked away.

But beneath the meme is something more revealing: a human defense mechanism and a little bit (maybe a lot?) of ego.

I think we call it slop because we feel simultaneously threatened and superior.

The Slur of the Machine Age

When we label something as AI slop, we’re not just critiquing the output. In reality, we’re trying to reassert our place in a changing world. It’s a psychological recoil against the idea that machines might now share our creative space.

Every major shift in technology has birthed its insults. Early digital art was “soulless.” Word processors were “lazy writing.” Even photography, when new, was dismissed as a “mechanical trick.” “AI slop” joins that lineage. It’s an emotional shorthand for “this thing doesn’t make me feel special anymore.”

We Crave Relevance

Human beings need to matter. For many of us, our sense of identity is deeply tied to what we do rather than how we feel. Work is how we prove worth. So when machines start doing the things that once defined us, we begin to feel displaced and unmoored.

But instead of naming that loss directly, we redirect it outward. We mock, we label, we diminish. “AI slop” becomes a socially acceptable way to express technological grief.

It’s easier to sneer at the product than to sit with the discomfort of perceived redundancy.

We’re Better Than That

We also can’t imagine being redundant because, naturally, we feel more valuable than the tools we wield. In the past, we praised the author, not the pen. You’ve probably never considered which computer or operating system your favorite novelist uses.

But with Gen AI, it feels like the awe has shifted to the tool, leaving less for the human. Where it’s obvious that a word processor doesn’t create stories, Gen AI seems to blur that line. And because of that illusion, we’re on high alert. We find ourselves constantly scanning for ways to prove we’re still better than the machine.

When Humans Err, It’s Humanity. When AI Errs, It’s Slop.

We romanticize our mistakes. A typo is “authentic.” A misstep is “part of the process.” But when a model does it, we call it “slop.”

We don’t just measure machine performance by output. We measure it by emotion. Does it make us feel diminished or inspired?

The irony is that many AI “failures” look a lot like human drafts: first tries, rough sketches and unedited thoughts. The difference is that when we make something messy, we call it creativity. When it does, we call it trash.

Furthermore, we shame the human who dared to use it.

Who Calls It “Slop” — and Why

Many of the strongest reactions come from highly experienced people who have spent years mastering craft, process, or precision. The ones who know what “good” actually means. Theirs is an earned skepticism in many ways.

I remember being in one of my first corporate jobs after university, working on a mission-critical application. The senior architect on the project had designed a remarkably clean, consistent pattern for building new services. When I was assigned to build one, I followed his structure exactly and got working code.

During the code review, he glanced at my work and said, “You can always tell when someone copy/pastes to start.”

He didn’t mean it cruelly, and I don’t think he realized how it landed. Up until that moment, I was proud. I’d followed his pattern, learned from it, and built something real. But what I heard in his tone wasn’t pride in having created a system others could replicate. What I heard was uncertainty.

And I understand that now.

When you’ve spent years building a foundation and suddenly someone (or something) can produce competent results by following your pattern, it stirs complicated emotions. Pride and fear coexist. I don’t think that the architect resented me. But I do wonder if he felt like he was watching his expertise become invisible through success.

That’s what’s happening to many professionals today.

Humans have spent generations producing patterns, frameworks, and cultural blueprints. Now AI, trained on those very patterns, can generate something good enough in seconds. It’s dizzying. You can either cry “slop!” or step back and marvel that our collective work has reached a point where a machine can echo it at all.

And many people do have reason to pause. I think our discomfort with GenAI doing our work, especially creative work, stems from a deep human desire to feel that our contributions still matter in a world that increasingly automates expression.

We’ve been here before, watching the tools we built start to outpace us, or at least look like they do. And yet, if we’re honest, it’s rarely the tools themselves that scare us.

The Real Threat Isn’t the Machine

What really unnerves us are the people who might use those tools against us. The manager who sees automation as a shortcut. The founder who values efficiency over expertise. The peer who leans on the tool instead of the team.

“AI slop” becomes a rallying cry: Don’t replace us. Don’t sell out. Don’t pretend the machine can do what we do.

It’s a human-to-human warning disguised as a critique of code. Calling out “slop” is a way of reminding each other not to become complicit in our own obsolescence.

But the truth is, AI isn’t the replacement. No tool has ever replaced human intent; it simply amplifies whatever direction we point it in.

If I used spell check and Grammarly to collect a bunch of grammatically correct sentences, you wouldn’t call it an essay. Grammarly can’t write one for you. And although Gen AI can generate an essay, it’s rarely inspiring on its own.

When the direction is thoughtful, the outcome can be extraordinary. When it’s careless, it looks like slop. That’s because we humans are still learning how to use AI.

We might consider judging work by the product itself, not by the tool used to create it.

From Shame to Standard

Eventually, the cultural shame around using AI will fade. It will be as ordinary as using spellcheck or autocomplete.

Just as spellcheck doesn’t make you a writer, AI doesn’t make you an expert. The tool doesn’t replace the need for judgment, intent, or taste.

Instead of being so fixated on identifying AI slop, we should turn our focus to cultivating humans who can shape what it produces with discernment and curiosity instead of defensiveness or fear.

Learning to use GenAI to create tasteful work will be the mark of creative maturity in the AI era. When it comes to writing code, or doing other non-creative work (I know code is creative in its own way. Don’t fight me) I think GenAI will have much less resistance to acceptance. Actually when I think about it, developers using coding agents have mostly come to accept AI assistance with some level of joy. Even if resistant at first.

So Who’s Really Being Sloppy?

Before we call the next image or paragraph “AI slop,” it’s worth pausing. What are we really reacting to? Are we offended by the machine, or what it reflects back about us?

Maybe it’s neither. Maybe it’s just the natural awkwardness of a species learning to collaborate with its own invention.

Whatever it is, we should learn to name it honestly. We can learn to identify and articulate whether it’s disappointment in the product, discomfort with the process, or resistance to change. Some work will need to be rejected. Some will just need another round of human collaboration and iteration.

We are not (yet :P) being replaced by AI. But I do feel like we are being reshaped through our relationship with it.

And the sooner we stop shouting “slop,” the sooner we can start shaping what’s next.

In a previous post, I walked through how I transformed a manual reporting workflow into an agentic one by giving an AI agent the same tools a human analyst would use to create CockroachDB performance reports.

Today, I’m releasing the full working code so you can explore, learn from, or adapt it for your own projects.

What You’ll Find

This is a complete, working proof-of-concept demonstrating:

• FastAgent for agentic workflows

• MCP (Model Context Protocol) integration with FastMCP 2.0 (Using FastAPI to generate FastMCP)

• A simple React frontend with AI-powered and manual report creation

• CockroachDB integration for cluster metrics

Note: This is a proof-of-concept built to demonstrate a simple agentic architecture pattern. There are areas for improvement (like configurable time ranges for metrics collection), but the core concepts are ready and the code is clear enough to learn from and build upon.

Get the Code

Repository: github.com/kikiya/agentic-tuning-reporter

git clone https://github.com/kikiya/agentic-tuning-reporter.git

cd agentic-tuning-reporter

Quick Start

The repo includes a QUICKSTART.md that gets you running in ~5 minutes:

1. Setup database – One script creates everything

2. Start backend services – Main API (8001) + Agent Service (8002)

3. Launch frontend – React UI on port 5173

Then click “New Report” → “Quick Analysis” and watch the AI agent:

• Gather cluster metrics

• Analyze performance

• Create structured findings

• Generate actionable recommendations

• Save everything to the database

All automatically. All editable afterward.

Architecture Highlights

Three Services Working Together

Main API – FastAPI service with:

• Reports CRUD operations

• Cluster metrics endpoints (topology, schema, CPU, slow queries)

• Direct CockroachDB connection

Agent Service – Separate FastAPI service with:

• AI-powered report generation

• FastAgent + LLM integration

• MCP server that exposes Main API as tools

Frontend – React + TypeScript with:

• “Quick Analysis” (AI-generated reports)

• “Custom Report” (manual creation)

• Both types are fully editable

Model Context Protocol for tools

The key insight from the blog post was giving the agent the same API access as a human. Here’s how:

# backend/mcp/reporter_mcp.py

from main import app

from fastmcp import FastMCP

# Convert entire FastAPI app to MCP tools

mcp = FastMCP.from_fastapi(app=app)

Every endpoint becomes a tool the agent can call. No manual wrapping needed.

What You Can Learn

This repo demonstrates:

1. Agentic Architecture – An introduction into how to structure apps where AI agents act autonomously

2. MCP Integration – Practical example of Model Context Protocol

3. Tool Design – What makes good tools for AI agents

4. Human-in-the-Loop – AI generates, humans refine via the UI

5. Service Separation – Why the agent service is separate from the main API

Use Cases

This pattern works for any workflow where you want AI to:

• Gather data from multiple sources

• Analyze and synthesize information

• Create structured outputs

• Save results for human review

Examples:

• Security audit reports

• Code review summaries

• Customer support analysis

• Financial report generation

• Research synthesis

Documentation

The repo includes:

• README.md – Comprehensive setup and architecture guide

• QUICKSTART.md – Get running in 5 minutes

• backend/AGENT_SERVICE.md – Deep dive on agent architecture

• backend/mcp/README.md – MCP server explanation

What You’ll Need

• CockroachDB running locally (or remote cluster)

• Python 3.12 (required for FastAgent)

• Node.js 16+

• Anthropic API key (or your choice of model. FastAgent supports most majors)

• 30 minutes to explore

Key Takeaways

As I wrote in the original post:

“What I love most is that the original pattern of direct value coupling still holds. Every model improvement or new MCP tool instantly enriches the agent’s output without code rewrites.”

This architecture is:

• Extensible – Add new tools by adding API endpoints, or new MCP servers

• Maintainable – Clean separation of concerns

• Flexible – Swap models, change prompts, no rewrites

• Human-centric – AI assists, humans decide

Known Limitations & Future Improvements

This is a proof-of-concept. Some areas for enhancement:

• Time Range Selection – Currently uses default time windows for metrics. Could add UI controls for “last hour”, “last day”, “custom range”

• Report Scheduling – No automated report generation on a schedule

• Multi-Cluster Support – Currently targets one cluster at a time

• Advanced Filtering – Could add more granular filtering for slow queries and metrics

• Report Templates – Could support different report types (weekly summary, incident analysis, etc.)

The core pattern is solid and these are just natural extensions as you adapt it to your needs.

Contributing & Feedback

This is a learning resource. If you:

• Find bugs or improvements

• Have questions about the architecture

• Want to share how you adapted it

Open an issue or PR! I’m building this in public to help others navigate the shift from “AI as a feature” to “AI as a collaborator.”

What’s Next

I’ll continue exploring agentic architectures—how agents, tools, and humans can work together as adaptive systems. Follow along at kikia.io or watch the repo for updates.

Links:

• 

  Original Blog Post

• 

  GitHub Repository

• 

  QUICKSTART Guide

Built with

to demonstrate agentic architecture patterns. Not production-ready, but production-inspired.

I’ve been working closely with Gen AI for a few years now. If you’ve been following along, you’ll recall I first used Gen AI as a partner in deconstructing data and ideas in my creative process (I wrote more about that here).

When OpenAI introduced the API, I began using ChatGPT programmatically. Then came “Functions/Tools,” which expanded my programmatic use (I talk about that here). Finally, Model Context Protocol arrived, giving LLMs an entirely new level of power. (A bit on my first app re-imaged as agentic here).

As the ecosystem evolved, so did my approach to building applications with LLMs. At first, my projects didn’t need persistence. But soon enough, data became the central to my functionality, and that shift changed major elements in my stack.

From zero to 100 on persistence

My first AI app was simple: a report generator that gathered system details, sent them as a formatted prompt, and had the LLM create a detailed report and actionable analysis. That didn’t require much tech: Node/Express, some HTML and CSS, the OpenAI API, and voila! No persistent data, no complex stack.

But that phase didn’t last long.

As my apps grew more complex, data stopped being incidental and became more central to my functionality. Apps weren’t just calling LLM APIs, agents were managing state, working with metadata, and contextual data to make better decisions. I started to think more about how to give my agents more of the right context at the right time.

Eventually retrieval-augmented generation (RAG) entered the picture: I wanted agents to surface relevant context on demand, not just query a database.

And that brought me to the next stage.

Evolving without introducing unnecessary complexity

One of my approaches to RAG leveraged embeddings, which to me meant a vector database. So I reached for Pinecone while still leaning on CockroachDB for relational data. initially, it looked like success to me! I was so caught up in the novelty of my apps new super powers that I lost sight of classic principled architecture.

By using two different data stores, I had revived the notorious dual-write problem. Ugh! One piece of data lived in CRDB, its semantic twin lived in Pinecone, and my agents were managing both. Sometimes they succeeded. Sometimes they left things inconsistent, like updating a record in CRDB but leaving the embedding stale in Pinecone.

It’s bad enough when you have hard coded instructions performing dual-writes. But this new age “soft coded” agent was amplifying risk. How quickly had I gone from a principled architect to a person asking non-deterministic agents to perform a dual-write?!

Thankfully I found myself and refined my approach.

The solution wasn’t to make agents more careful. Agent instructions and guardrails are a valid part of hardening your system. But I had to make a more fundamental change to strengthen the system. I decided to simplify the architecture itself. CockroachDB now supports vector embeddings, which meant I could co-locate both relational metadata and vector data in the same system. Of course, I acknowledge there are other ways to solve for dual-writes! This was the simplest way to completely avoid dual-writing in my solution.

For example: I could now design a system to, store both support ticket metadata (customer, severity, etc) and embeddings of past tickets in CRDB. One query could return both related cases and the structured context for escalation. I could do this without dual writes and with other performance benefits I’ll dive into a little deeper in different material.

This simplification worked but there was another foundational reason for using CRDB as my apps DB.

Limiting non-determinism at the foundation

I glossed over this earlier but the importance of CRDB’s consistent data principles can’t be overstated in the context of building agentic apps. Think about how CRDB handles concurrency and anomalies. By default, it uses serializable isolation. When you’re already introducing a degree of uncertainty by design in your stack, you can’t afford race conditions or corrupted states in your foundation.

In a weaker isolation model, two agents might interleave updates and leave a nonsense value like truefalse. With CRDB, one transaction wins cleanly, keeping the system correct. That stability matters when agents participate in state management. In my opinion it matters when introducing state to any system but especially one where you have this extra area of non-determinism.

My takeaway

AI applications are new, fun, and powerful. But they also run risk to your user’s Trust by introducing doubt. It doesn’t even have to be AI’s fault that a data anomaly occurred, especially if you’re using a database with lower levels of transaction isolation. But the user’s perception is everything and blaming AI for any off experience is just easy default behavior. You want to be able to infuse AI into your solutions without weakening or damaging the trust you’ve built with your customers. And since building agentic applications means embracing some uncertainty at the top of the stack, that only works if you eliminate uncertainty at the bottom.

If consistent data is critical to your application’s core function and you have agents that manage or rely on that data, a rock-solid data layer is just non-negotiable. For me, that’s what CockroachDB provides: strong data consistency, resilience across failures, and a simplified architecture for both relational and vector data.

In the last post, we looked at how AI reshaped teamwork. I discussed how humans stayed in the loop while friction melted away. Now, it’s time to reimagine the Reporter itself.

Let’s start with its purpose. The report was never for reporting’s sake—it’s meant to drive action. People need to share it, review it, prune or expand recommendations, maybe even track and execute actions directly from it. That simple workflow carries a lot of potential.

For now, though, I’ll keep things simple to show what happens when we give the Reporter more power, more agency, without losing control.

From Human Workflow to Agentic Workflow

Before we automate, let’s think about the regular, human-first flow.

Imagine you have a person responsible for analyzing cluster metrics and building a report. That person has access to the CockroachDB Cluster API and a web application where reports can be created, reviewed, and edited.

I’ve built a small demonstration app to show this process:

The analyst uses the Cluster API to gather metrics, then creates findings in the web application.

The web app is backed by a Python FastAPI service exposing these endpoints:

GET     /api/v1/reports                    → List Reports
POST    /api/v1/reports                    → Create New Report
GET     /api/v1/reports/{report_id}        → Get Report
PUT     /api/v1/reports/{report_id}        → Update Existing Report
DELETE  /api/v1/reports/{report_id}        → Delete Existing Report

GET     /api/v1/reports/{report_id}/findings       → List Findings
POST    /api/v1/reports/{report_id}/findings       → Create Finding
PUT     /api/v1/findings/{finding_id}              → Update Finding
DELETE  /api/v1/findings/{finding_id}              → Delete Finding

GET     /api/v1/findings/{finding_id}/actions      → List Actions
POST    /api/v1/findings/{finding_id}/actions      → Create Action
PUT     /api/v1/actions/{action_id}                → Update Action

GET     /api/v1/reports/{report_id}/comments       → List Comments
POST    /api/v1/reports/{report_id}/comments       → Create Comment

This allows me to build a structured report.

So what if I want the LLM itself to act as the reporter?
I can create an agent and give it the same API access as the human—just through tools.

Building the Agent

Let’s start with a minimal setup using the FastAgent library.

import asyncio
from mcp_agent.core.fastagent import FastAgent

# Create the application
reporter = FastAgent("Report Generator")

# Define the agent
@reporter.agent(instruction="""You are an expert at database performance analysis and optimization""")
async def main():
    # use the --model command line switch or agent arguments to change model
    async with reporter.run() as agent:
        await agent.interactive()

if __name__ == "__main__":
    asyncio.run(main())

At this stage, it’s in interactive() mode so I can test before weaving it into the full app.

I also need two files:

• fastagent.config.yaml – defines default model, logging, and MCP servers

• fastagent.secrets.yaml – securely stores model API keys and environment variables

Example Configuration

# fastagent.config.yaml
default_model: sonnet35

logger:
  level: "debug"
  type: "console"
  progress_display: true
  show_chat: true
  show_tools: true
  truncate_tools: true

# fastagent.secrets.yaml
openai:
  api_key: <your-openai-key>
anthropic:
  api_key: <your-anthropic-key>

If I start it in interactive mode, I get a chat interface like this:

Feeding it the same prompt as before produces a similar structured report—almost.
Previously, the structure was enforced by a tool call:

const tools = [
  {
    type: "function",
    function: {
      name: "generateRecommendations",
      description: "Generates structured recommendations for a CockroachDB cluster",
      strict: true,
      parameters: { /* JSON schema */ }
    }
  }
];

Stuffing the JSON schema into the agent prompt is brittle because some models still prefix their response (“Here is your JSON response:”) and break the UI contract.

So now we’ll empower the agent with real tools using Model Context Protocol (MCP).

Empowering the Agent with MCP

MCP (Model Context Protocol) is an open-source standard that connects an LLM to internal or external systems.
In this case, I want the agent to access the Reporter Service API to create reports the same way a human would.

If APIs connect UIs to services, think of MCP as connecting agents to tools.
An MCP server sits between the agent and those systems, exposing structured functions it can call.

Let’s define one using FastMCP:

from fastmcp import FastMCP

mcp = FastMCP("Example Server")

@mcp.tool
def temp_to_celsius(f: float) -> float:
    """Convert a temperature from Fahrenheit to Celsius"""
    return round((f - 32) * 5.0 / 9.0, 2)

if __name__ == "__main__":
    mcp.run()

Each @mcp.tool makes a function callable by an agent.
For our app, instead of writing every endpoint by hand, FastMCP 2.0 introduces a shortcut that can wrap a full FastAPI app into an MCP server:

from main import app
from fastmcp import FastMCP

# Convert to MCP server
mcp = FastMCP.from_fastapi(app=app)

if __name__ == "__main__":
    mcp.run()

Now every endpoint in the Reporter API becomes a callable tool.

Next, register the server in the FastAgent configuration:

# fastagent.config.yaml
mcp:
  servers:
    reporter:
      command: "python"
      args: ["reporter_mcp.py"]

And add necessary environment variables:

# fastagent.secrets.yaml
mcp:
  servers:
    reporter:
      env:
        DATABASE_URL: "cockroachdb+psycopg://root@insecure-database:26257/tuning_reports?sslmode=disable"

Finally, wire it into the agent definition:

@reporter.agent(
  instruction="You are an expert at database performance analysis and optimization.",
  servers=["reporter"]
)

Now, when I prompt:

“Use the reporting tools to create a fictional crdb tuning report.”

the agent does exactly that—creating structured records via the MCP tools.

It even added comments automatically! That’s the power of giving an agent access to well-documented tools.

Wiring It All Together

There’s one last piece: context. The agent still needs cluster metrics. We can expose CockroachDB Cluster API endpoints through the same FastAPI app and include credentials in the secrets config.

Then we create an API endpoint that generates reports using the agent.

The Agent Service (snippet)

 async def generate_report_with_agent(database: str, app: str, custom_prompt: Optional[str] = None) -> str:
    reporter = FastAgent("Report Generator")
    prompt = custom_prompt or f"""
Please analyze the CockroachDB cluster performance for the '{database}' database and '{app}' application...
"""
    @reporter.agent(
        instruction="You are an expert at database performance analysis and optimization.",
        servers=["reporter"]
    )
    async def generate():
        async with reporter.run() as agent:
            return await agent.send(prompt)
    return await generate()

The API Endpoint

@app.post("/generate-report", response_model=GenerateReportResponse)
async def generate_report(request: GenerateReportRequest):
    try:
        report = await generate_report_with_agent(
            database=request.database,
            app=request.app,
            custom_prompt=request.prompt
        )
        return GenerateReportResponse(success=True, report=report)
    except Exception as e:
        raise HTTPException(status_code=500, detail=f"Failed to generate report: {e}")

This lives in a separate agent_main.py, isolated from the core Reporter APIs.

To test, I start a CRDB cluster and load it with test data for my “bookly” database and app:

curl -X POST "http://localhost:8002/generate-report"   -H "Content-Type: application/json"   -d '{"database": "bookly", "app": "bookly"}'

Refreshing the app shows the new report instantly—no UI changes required!

A UI button could easily call this same endpoint to create either a standard or a custom-prompted report.

The Pattern Holds: Direct Value Coupling

What I love most is that the original pattern of direct value coupling still holds. Every model improvement or new MCP tool instantly enriches the agent’s output without code rewrites.

In the beginning, I had to stuff all context into a single prompt. Now, the agent fetches its own data, calls structured APIs, and builds a shareable, actionable report. The reporter didn’t just get smarter, it gained agency. And the human in the loop kept theirs: reviewing, editing, and managing the AI-generated report assets with intention.

What emerged was a structured architecture that supported both the machine acting with autonomy, and the human maintaining authorship. This is our triad working together again: application code orchestrates structure, the AI agent performs the analysis, and the human refines the outcome. Each doing their best part, in concert.

Although this is the last part in the original From Prompts to Programs to Agents journey, I’ll continue exploring what agentic architecture really means — how agents, tools, and humans can work together as an adaptive system.

I’m all about building AI that works with humans, not just for humans or in place of them. I’ll share more code and technical reflections in future posts.

Before I dive into the agentic phase with MCPs and tools, I need to pause. I was moving too fast. There’s a big part of my journey that I haven’t shared yet, and it’s just as important as the code: the role of humans in the loop.

Why this matters

Early on, I identified the human in the loop as a key pattern for AI workflows. What I want to explore in this post is how one human in the loop experience revealed how AI could help bridge people with different specialties. It smoothed friction across cross-functional teams and reduced capacity pressure on teammates.

Instead of leaning entirely on a technical expert or slogging through scattered Google searches, a non-expert could now get enough grounding to push forward faster and with less burden on others.

Lamont and the Tombstones

Back in the early GenAI days, I worked with Lamont, a creative video producer and editor. Like me, he was bullish on AI and a big dreamer. And like any great dreamer, he knew how to spot opportunity.

His first AI project came before AI video tools even existed. He combined his illustration, animation, and editing skills with AI’s ability to help him grasp technical concepts. The idea was brilliant: a Halloween-themed video about tombstones — not just the spooky kind, but the technical kind. In databases, tombstones mark deleted rows. He turned that into a horror short.

Lamont had the creative vision, the storytelling skill, and the production chops. But he wasn’t a database expert. He used AI not to tell the story for him — he already had that gift — but to understand the technical side well enough to weave it into the script. Then he iterated on details with AI before pulling me in as the technical reviewer.

Before AI vs. After AI

Before AI, Lamont would have pulled me in at the very beginning: research, metaphors, breaking things down simply. With AI, he didn’t need me there. He could use ChatGPT — which he nicknamed “Chatty” — to clarify concepts, explore metaphors, and test his understanding. He could draft a script, refine it, and only then pull me in to check technical accuracy.

The workflow shifted:

• Before AI: Human (creative) → Human (technical explainer) → Draft → Human (technical review)

• With AI: Human (creative) + AI (technical explainer) → Draft → Human (technical review).

Our conversations changed too. Instead of going back and forth on every technical detail just so he could learn, I might only need to explain why AI needed to be steered differently or clarify one subtle point. Meanwhile, he could dream up content ideas a mile a minute and work through them with his indefatigable AI helper.

Why this was an aha moment

It clicked for me that GenAI didn’t just change individual workflows. It changed team workflows. It lowered friction in cross-functional collaboration. Lamont didn’t have to worry about pulling me away from other work or me losing time to context switching. He could self-serve on the technical scaffolding and save me for the expert check.

That’s revolutionary. We often think of AI as a replacement for rote tasks. But in this case, AI expanded Lamont’s creative autonomy while deepening the collaboration between us. He could dream bigger because he had a tireless partner to accelerate his research. And I could step in more surgically where my expertise actually mattered.

Humans, AI, and roles

Looking back, it’s clear the instinct to know where to place AI and where to place people is just as important as writing the code. It’s about roles:

• Application code → structure.

• AI → assistant, explainer, technical researcher.

• Human experts → reviewers, editors, creators.

Lamont embodied that instinct. Even where he was the creative expert, he still invited feedback. He never treated AI or himself as the final authority. That kind of collaboration mindset is what makes cross-functional AI workflows powerful.

A future glimpse

The funny part is, Lamont was early to so much of this. He was experimenting with content workflows, avatars, and synthetic voices long before ElevenLabs made them mainstream. He knew that if you design workflows early, they’ll only get better with time as the tools mature.

Eventually, he learned creative prompting and AI tools so well, he was able to build out entire High Quality (slop-free) content pipelines in a fraction of time it would typically take. If you are interested in seeing more of Lamont’s work, check out www.belton.ai

It wasn’t about replacing people. It was about putting creativity into hyperdrive. And participating in his process helped me see how AI could strategically smooth the friction between teams and disciplines. That was an aha moment I still carry.

The pivot to agents

This reflection sets up the next phase. Once I saw how AI changed not just my workflow, but our workflow, I started to imagine architectures that formalized this: humans in the loop, AI assistants orchestrating context, and applications providing structure. That’s what Model Context Protocol and agents made possible.

So before we go back to code, remember: the technical evolution is only half the story. The human patterns matter just as much. And this is the part of the story that made me ready to embrace the agentic phase.

In the last post, I talked about learning to extract meaningful value from ChatGPT even when it was limited by hallucinations, short memory, context limits, and all. The point wasn’t perfection, it was usefulness. That mindset carried directly into my first programmatic app. It didn’t need to be flawless to prove something powerful: that AI could integrate into a workflow and improve the product in ways I’d never seen before.

From Conversational to Programmatic

This post begins the programmatic phase. When the ChatGPT APIs arrived I was excited to try it out and coded my first AI‑infused app. It was a simple proof of concept, but it reframed how I think about AI’s role in traditional stacks.

It was during this phase that I moved from chatting in the web UI to coding in my IDE. Using GenAI programmatically opened a new world where I could repeatably produce useful content as part of a workflow. My first programmatic workflow implementation was simple, but what I learned from that proof of concept was more enlightening than I expected.

I built a tiny app that generated a CockroachDB tuning report. The surprising part was not that it worked. It was that the report kept getting better every time the underlying model leveled up—with me changing almost nothing. Swap the model name in one line, redeploy, and the “product” improved. The “reporter” got smarter, so the report got better.

This pointed me to a new way of viewing the relationship between AI and traditional stacks. You can find the precise seam in your workflow where an LLM fashions some deliverable, then let model progress flow straight into user value.

The App, in 60 Seconds

Goal: given cluster metadata and a slice of workload evidence, produce specific, actionable tuning recommendations with example SQL and expected impact.
Shape: tiny HTTP service with one endpoint. Front‑end optional. Returns JSON or HTML.
LLM role: expert “reporter.” It must call a tool that enforces a strict JSON schema for recommendations. No rambling essays.

High‑Level Flow

[Client] → GET /report/recommendations
            ↓
        Express Router
            ↓
   gatherClusterData()      → cluster JSON
            ↓
   generateReport()         → OpenAI chat.completions with tool schema
            ↓
  parsed { recommendations } → JSON/HTML report

Minimal Stack

• Node 18+, ESM modules

• Express for routing

• Axios for optional REST calls (e.g., CRDB API)

• OpenAI SDK v4 (ESM)

• dotenv for config

Managing Context (Early Days)

In this first iteration, all context was stuffed directly into a single prompt. Everything the LLM needed—cluster topology, schema, workload samples—had to be included inline. There was no memory, no selective fetching, no orchestration. Just one big prompt, every time.

It worked, but it was rigid. If I wanted a new signal (like CPU usage), I had to update gatherClusterData and stuff more JSON into the prompt. Context stuffing was necessary—without it, the model would only give generic advice. With it, I could get precise, cluster‑specific insights.

Code Walk‑Through (Trimmed)

app.js — wire up the router

// app.js (ESM)
import express from 'express';
import path from 'path';
import { fileURLToPath } from 'url';
import reportRoutes from './reportRoutes.js';

const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);

const app = express();
app.use(express.static(path.join(__dirname, 'public')));
app.use('/report', reportRoutes);

app.listen(3000, () => {
  console.log('Server listening on 3000');
});

gatherClusterData.js — inputs the reporter needs

import axios from 'axios';
import minimist from 'minimist';

const args = minimist(process.argv.slice(2));
const hostIp = args['host-ip'] || '127.0.0.1';
const COCKROACHDB_API_URL = `http://${hostIp}:8080/api/v2`;
const axiosConfig = { /* auth, headers, timeouts */ };

export async function getClusterTopology() {
  const res = await axios.get(`${COCKROACHDB_API_URL}/nodes/`, axiosConfig);
  return res.data.nodes.map(n => ({
    node_id: n.node_id,
    locality: n.locality?.tiers?.map(t => `${t.key}=${t.value}`).join(', ') || 'Unknown'
  }));
}

// also: getDatabaseSchema(db), getCPUUsage(), getSlowStatements()
export async function gatherClusterData(databaseName) {
  try {
    const topology = await getClusterTopology();
    const schema = await getDatabaseSchema(databaseName);
    const cpuUsage = await getCPUUsage();
    const slowStatements = await getSlowStatements();
    return { topology, schema, cpuUsage, slowStatements };
  } catch (e) {
    console.error('Error gathering cluster data:', e);
    return { topology: [], schema: {}, cpuUsage: [], slowStatements: [] };
  }
}

openaiClient.js — make the model call and force structured output

import OpenAI from 'openai';
import dotenv from 'dotenv';
dotenv.config();

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

const tools = [
  {
    type: 'function',
    function: {
      name: 'generateRecommendations',
      description: 'Structured CockroachDB tuning advice',
      strict: true,
      parameters: {
        type: 'object',
        properties: {
          recommendations: {
            type: 'array',
            items: {
              type: 'object',
              properties: {
                section: { type: 'string' },
                details: {
                  type: 'array',
                  items: {
                    type: 'object',
                    properties: {
                      title: { type: 'string' },
                      rationale: { type: 'string' },
                      actions: { type: 'array', items: { type: 'string' } },
                      sql_examples: { type: 'array', items: { type: 'string' } },
                      expected_impact: { type: 'string' }
                    },
                    required: ['title', 'actions']
                  }
                }
              },
              required: ['section', 'details']
            }
          }
        },
        required: ['recommendations']
      }
    }
  }
];

export async function generateReport(clusterInfo) {
  const messages = [
    { role: 'system', content: 'You are an expert in CockroachDB performance.' },
    { role: 'user', content: `Analyze this cluster and produce structured JSON via the tool.\n${JSON.stringify(clusterInfo, null, 2)}` }
  ];

  const completion = await openai.chat.completions.create({
    model: process.env.OPENAI_MODEL || 'o3-mini-2025-01-31',
    messages,
    reasoning_effort: 'high',
    tools,
    tool_choice: { type: 'function', function: { name: 'generateRecommendations' } },
    store: true
  });

  const toolCalls = completion.choices[0].message.tool_calls;
  const args = JSON.parse(toolCalls[0].function.arguments);
  return args; // { recommendations: [...] }
}

reportRoutes.js — one endpoint

import express from 'express';
import { gatherClusterData } from './gatherClusterData.js';
import { generateReport } from './openaiClient.js';

const router = express.Router();

router.get('/recommendations', async (req, res) => {
  try {
    const db = req.query.db || 'defaultdb';
    const clusterInfo = await gatherClusterData(db);
    const result = await generateReport(clusterInfo);
    res.json({ ok: true, ...result });
  } catch (err) {
    console.error(err);
    res.status(500).json({ ok: false, error: err.message });
  }
});

export default router;

The Knobs I Used

• Determinism when needed: pinned temperature near 0.

• Schema drift: enforced strict: true, fail closed if the tool call broke.

• Reasoning effort: experimented with low, medium, and high once reasoning models arrived.

The Challenges

• When OpenAI deprecated “functions” in favor of “tools,” I had to refactor. It was mildly inconvenient, but the idea still worked.

• When the reasoning models arrived, the call signature. I easily adapted thanks to the OpenAI platform playground.

• Testing non‑OpenAI models wasn’t straightforward. I could swap models in one provider (OpenAI), but not providers entirely. Unless I built a new client, I was basically locked in with OpenAI.

Why This Felt Different Than “Faster Chips”

Sometimes your product gets better because hardware or dependencies improve, but that value doesn’t always flow directly into your product’s core. Faster chips make things smoother. New frameworks bring new features—if you write the code to take advantage.

This experiment was different. It was direct value coupling where the deliverable was the model’s output. When GPT‑4 → o1 → o3 improved, the report improved along the same axis: nuance, noise filtering, actionable SQL. All I did was switch the model name. No new plumbing. No feature rewrites. That’s what made it feel profound.

What felt missing? The human in the Loop

In the conversational phase, the human orchestrated everything: gathering context, prompting, writing reports.

In the programmatic phase, I automated orchestration. The LLM produced structured outputs, but the human was absent from the loop. I only had the machine + LLM.

The ideal future isn’t removing humans, but shifting roles:

The human/traditional app code/LLM triad keeps everyone playing their best part and this journey series is dedicated to telling that story.

Recap of My Journey

• Conversational phase: ChatGPT as a creative partner and deconstruction tool.

• Programmatic phase: this project—a proof that LLMs could reliably call tools and improve the product directly.

• Agentic phase: now, with MCPs, context is managed better. Agents fetch what they need, call tools dynamically, and free me from provider lock‑in (Claude, Gemini, OpenAI, etc.).

Takeaway

Find the seam where your product’s “reporter” lives, and let the model roadmap do part of the lifting. Although this story centers on a small proof of concept, the principle scales: the LLM doesn’t need to sit at the center of your app; you just need to find the right place, where improvements flow directly into value.

This was one phase of my journey. In the next, I’ll reimagine this app in the agentic phase—with MCP agents granting more autonomy to the LLM while keeping humans in the loop.

This is part two of a series tracing my AI journey. Part one covered the conversational phase, this post focused on the programmatic phase, and next up I’ll dive into the agentic phase. However, before I dive into the agentic phase, there’s another lesson that shaped my thinking just as much as writing my first lines of AI code: the human in the loop.

In 2022, ChatGPT didn’t know the news, forgot conversations, and sometimes hallucinated yet I still made it one of the most useful tools in my workflow. Using it early and often sharpened my instincts for how to collaborate with AI in practice, not in theory.

I’ve always loved the bleeding edge of technology. Not for novelty’s sake, not to sit on a shelf at CES, but because what others might see as futuristic, I want to put to work. My favorite moments in tech are when one architectural approach quietly eclipses another by doing the same job in a more elegant, efficient, beautiful way.

That’s why, when ChatGPT burst onto the scene in the fall of 2022, I picked it up and began testing its limits immediately! I even gave it a name: Gina.

When ChatGPT Knew Everything and Nothing

In the beginning, Gina didn’t know much about the world. In fact, she knew nothing past September 2021 — the training cutoff at the time. If you asked about current events, she would come up empty. But if you wanted to dive into history, philosophy, or technical concepts, she would happily oblige.

Debating policy without worrying about offense? Gina was there. Clarifying an obscure cuneiform text? She leaned in. Exploring concurrency and memory management across programming languages? She had plenty to say. She could even help debug code problems to an extent. She wasn’t always accurate, but she was always chatty, fun, and useful.

Looking back, I realize that period was defined by deconstruction. Gina excelled at breaking things down with me: dense academic passages, technical white papers, abstract philosophical ideas, and engineering principles. Her generation fed into my generation, and that loop helped me refine my understanding and rebuild ideas in new ways.

For example, once I fed her a translation of a Thomas Aquinas passage and asked her to turn it into a children’s story with forest creatures. Other times, I’d paste in sections from technical papers and ask her to strip them down to first principles. It was play and work at once. This kind of experimentation sharpens your grasp of both the tool and your own thinking.

Learning the Boundaries

The more I talked with Gina, the more I discovered her boundaries. As I pushed against boundaries, I learned how to prompt better. Through practiced prompt engineering, I figured out what phrases worked best, when to be open-ended and when to be prescriptive, when to stay conversational and when to ask for structured output.

One of my earliest takeaways was that whether conversing or creating, context was key.

I realized that to get better answers, I had to provide better context. At the time, context meant including any assumptions, details, things I tried, or even pasting blocks of text from documents, web pages, or code into the chat. This was how I brought in information from after the training cutoff. It also helped me zoom in on specific problems. Providing raw material allowed Gina to deconstruct it with me, often more effectively.

Conversations themselves sometimes hit a wall. I noticed Gina would forget things we had already discussed. I’d ask, “Don’t you remember I said…?” only to realize that I had bumped into what’s now called the context window.

I learned to work around this by summarizing. I started condensing our conversations, replaying the highlights back to Gina, and sometimes even asking her to summarize for me. For the early models, it often helped to summarize and then start a fresh conversation with that recap. This forced me to clarify my own understanding. The limits pushed me to compact ideas and keep the thread tight.

Another limitation was hallucination. Gina sometimes delivered confident but inaccurate statements. But I never treated her as a final authority. She was a tutor, a coach, a study partner. When I needed definitive answers, I went to primary sources: the original text, the dictionary, the official documentation.

For me, hallucinations weren’t dealbreakers because I wasn’t asking for absolute truth. I was asking for perspective, structure, and possibilities. That distinction mattered.

If I wanted to know the similarities between concurrency in Java and Rust, Gina might not get every detail right, but she gave me enough scaffolding to refine my own thinking. If I wanted to reimagine Aquinas as a woodland tale, she gave me a story to work with. Thinking, creating, and problem solving all became more fun with my new unpretentious partner.

Practicing deconstruction developed instinct

As I said earlier, those early days with ChatGPT were all about deconstruction. Breaking down concepts, reframing ideas, and reconstructing them into new forms. I worked through policy debates, software engineering principles, theological concepts, and more. The point wasn’t the final answer, it was the process.

What others dismissed as novelty, I was putting to serious work.

And that work left an imprint. There’s a difference between knowing about a technology and knowing it from experience. Reading about large language models gave me intellectual understanding, but using them daily rewired my instincts.

I think working with a technology in its infancy imprints something on how you approach it as it grows. Because I had used ChatGPT early and often, I developed a kind of muscle memory around it. I knew its rhythms, its quirks, the way it handled context, the way it faltered. That hands-on experience gave me a foundation I couldn’t have gotten from articles or demos alone.

It meant that as the models evolved, I evolved too. My proficiency grew not because I read more, but because I practiced more.

That’s why I think of those early months not just as the period of deconstruction, but as the time when my instincts for working with AI were formed.

From Prompts to Programs

That was the foundation: using Gina as a conversational partner to help me dismantle and rebuild ideas. Technology will always have shortcomings. Gen AI is no different. Models have context limits, they hallucinate, they miss the headlines of the day. But the measure of a tool isn’t whether it’s flawless. It’s whether you can put it to work today.

I didn’t ask Gina to be perfect. I asked her to be useful. And when she was, I extracted surprising amounts of value for a fraction of what that kind of intellectual companionship would have cost in any other form.

That was the period of deconstruction. Next came something even more exciting: when ChatGPT stopped being just a conversation partner and became a programmable collaborator.

APIs, programmatic workflows, agents all quickly followed.