My First AI‑Infused Application: The One‑Line Upgrade (Part 2)
From Prompts to Programs to Agents (Part Two)
In the last post, I talked about learning to extract meaningful value from ChatGPT even when it was limited by hallucinations, short memory, context limits, and all. The point wasn’t perfection, it was usefulness. That mindset carried directly into my first programmatic app. It didn’t need to be flawless to prove something powerful: that AI could integrate into a workflow and improve the product in ways I’d never seen before.
From Conversational to Programmatic
This post begins the programmatic phase. When the ChatGPT APIs arrived I was excited to try it out and coded my first AI‑infused app. It was a simple proof of concept, but it reframed how I think about AI’s role in traditional stacks.
It was during this phase that I moved from chatting in the web UI to coding in my IDE. Using GenAI programmatically opened a new world where I could repeatably produce useful content as part of a workflow. My first programmatic workflow implementation was simple, but what I learned from that proof of concept was more enlightening than I expected.
I built a tiny app that generated a CockroachDB tuning report. The surprising part was not that it worked. It was that the report kept getting better every time the underlying model leveled up—with me changing almost nothing. Swap the model name in one line, redeploy, and the “product” improved. The “reporter” got smarter, so the report got better.
This pointed me to a new way of viewing the relationship between AI and traditional stacks. You can find the precise seam in your workflow where an LLM fashions some deliverable, then let model progress flow straight into user value.
The App, in 60 Seconds
Goal: given cluster metadata and a slice of workload evidence, produce specific, actionable tuning recommendations with example SQL and expected impact.
Shape: tiny HTTP service with one endpoint. Front‑end optional. Returns JSON or HTML.
LLM role: expert “reporter.” It must call a tool that enforces a strict JSON schema for recommendations. No rambling essays.
High‑Level Flow
[Client] → GET /report/recommendations
↓
Express Router
↓
gatherClusterData() → cluster JSON
↓
generateReport() → OpenAI chat.completions with tool schema
↓
parsed { recommendations } → JSON/HTML report
Minimal Stack
Node 18+, ESM modules
Express for routing
Axios for optional REST calls (e.g., CRDB API)
OpenAI SDK v4 (ESM)
dotenv for config
Managing Context (Early Days)
In this first iteration, all context was stuffed directly into a single prompt. Everything the LLM needed—cluster topology, schema, workload samples—had to be included inline. There was no memory, no selective fetching, no orchestration. Just one big prompt, every time.
It worked, but it was rigid. If I wanted a new signal (like CPU usage), I had to update gatherClusterData and stuff more JSON into the prompt. Context stuffing was necessary—without it, the model would only give generic advice. With it, I could get precise, cluster‑specific insights.
Code Walk‑Through (Trimmed)
app.js — wire up the router
// app.js (ESM)
import express from 'express';
import path from 'path';
import { fileURLToPath } from 'url';
import reportRoutes from './reportRoutes.js';
const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);
const app = express();
app.use(express.static(path.join(__dirname, 'public')));
app.use('/report', reportRoutes);
app.listen(3000, () => {
console.log('Server listening on 3000');
});
gatherClusterData.js — inputs the reporter needs
import axios from 'axios';
import minimist from 'minimist';
const args = minimist(process.argv.slice(2));
const hostIp = args['host-ip'] || '127.0.0.1';
const COCKROACHDB_API_URL = `http://${hostIp}:8080/api/v2`;
const axiosConfig = { /* auth, headers, timeouts */ };
export async function getClusterTopology() {
const res = await axios.get(`${COCKROACHDB_API_URL}/nodes/`, axiosConfig);
return res.data.nodes.map(n => ({
node_id: n.node_id,
locality: n.locality?.tiers?.map(t => `${t.key}=${t.value}`).join(', ') || 'Unknown'
}));
}
// also: getDatabaseSchema(db), getCPUUsage(), getSlowStatements()
export async function gatherClusterData(databaseName) {
try {
const topology = await getClusterTopology();
const schema = await getDatabaseSchema(databaseName);
const cpuUsage = await getCPUUsage();
const slowStatements = await getSlowStatements();
return { topology, schema, cpuUsage, slowStatements };
} catch (e) {
console.error('Error gathering cluster data:', e);
return { topology: [], schema: {}, cpuUsage: [], slowStatements: [] };
}
}
openaiClient.js — make the model call and force structured output
import OpenAI from 'openai';
import dotenv from 'dotenv';
dotenv.config();
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
const tools = [
{
type: 'function',
function: {
name: 'generateRecommendations',
description: 'Structured CockroachDB tuning advice',
strict: true,
parameters: {
type: 'object',
properties: {
recommendations: {
type: 'array',
items: {
type: 'object',
properties: {
section: { type: 'string' },
details: {
type: 'array',
items: {
type: 'object',
properties: {
title: { type: 'string' },
rationale: { type: 'string' },
actions: { type: 'array', items: { type: 'string' } },
sql_examples: { type: 'array', items: { type: 'string' } },
expected_impact: { type: 'string' }
},
required: ['title', 'actions']
}
}
},
required: ['section', 'details']
}
}
},
required: ['recommendations']
}
}
}
];
export async function generateReport(clusterInfo) {
const messages = [
{ role: 'system', content: 'You are an expert in CockroachDB performance.' },
{ role: 'user', content: `Analyze this cluster and produce structured JSON via the tool.\n${JSON.stringify(clusterInfo, null, 2)}` }
];
const completion = await openai.chat.completions.create({
model: process.env.OPENAI_MODEL || 'o3-mini-2025-01-31',
messages,
reasoning_effort: 'high',
tools,
tool_choice: { type: 'function', function: { name: 'generateRecommendations' } },
store: true
});
const toolCalls = completion.choices[0].message.tool_calls;
const args = JSON.parse(toolCalls[0].function.arguments);
return args; // { recommendations: [...] }
}
reportRoutes.js — one endpoint
import express from 'express';
import { gatherClusterData } from './gatherClusterData.js';
import { generateReport } from './openaiClient.js';
const router = express.Router();
router.get('/recommendations', async (req, res) => {
try {
const db = req.query.db || 'defaultdb';
const clusterInfo = await gatherClusterData(db);
const result = await generateReport(clusterInfo);
res.json({ ok: true, ...result });
} catch (err) {
console.error(err);
res.status(500).json({ ok: false, error: err.message });
}
});
export default router;
The Knobs I Used
Determinism when needed: pinned temperature near 0.
Schema drift: enforced
strict: true, fail closed if the tool call broke.Reasoning effort: experimented with low, medium, and high once reasoning models arrived.
The Challenges
When OpenAI deprecated “functions” in favor of “tools,” I had to refactor. It was mildly inconvenient, but the idea still worked.
When the reasoning models arrived, the call signature. I easily adapted thanks to the OpenAI platform playground.
Testing non‑OpenAI models wasn’t straightforward. I could swap models in one provider (OpenAI), but not providers entirely. Unless I built a new client, I was basically locked in with OpenAI.
Why This Felt Different Than “Faster Chips”
Sometimes your product gets better because hardware or dependencies improve, but that value doesn’t always flow directly into your product’s core. Faster chips make things smoother. New frameworks bring new features—if you write the code to take advantage.
This experiment was different. It was direct value coupling where the deliverable was the model’s output. When GPT‑4 → o1 → o3 improved, the report improved along the same axis: nuance, noise filtering, actionable SQL. All I did was switch the model name. No new plumbing. No feature rewrites. That’s what made it feel profound.
What felt missing? The human in the Loop
In the conversational phase, the human orchestrated everything: gathering context, prompting, writing reports.
In the programmatic phase, I automated orchestration. The LLM produced structured outputs, but the human was absent from the loop. I only had the machine + LLM.
The ideal future isn’t removing humans, but shifting roles:
RoleFunctionApplication codeorchestrates structureLLMreporterHumanreviewer/editor
The human/traditional app code/LLM triad keeps everyone playing their best part and this journey series is dedicated to telling that story.
Recap of My Journey
Conversational phase: ChatGPT as a creative partner and deconstruction tool.
Programmatic phase: this project—a proof that LLMs could reliably call tools and improve the product directly.
Agentic phase: now, with MCPs, context is managed better. Agents fetch what they need, call tools dynamically, and free me from provider lock‑in (Claude, Gemini, OpenAI, etc.).
Takeaway
Find the seam where your product’s “reporter” lives, and let the model roadmap do part of the lifting. Although this story centers on a small proof of concept, the principle scales: the LLM doesn’t need to sit at the center of your app; you just need to find the right place, where improvements flow directly into value.
This was one phase of my journey. In the next, I’ll reimagine this app in the agentic phase—with MCP agents granting more autonomy to the LLM while keeping humans in the loop.
This is part two of a series tracing my AI journey. Part one covered the conversational phase, this post focused on the programmatic phase, and next up I’ll dive into the agentic phase. However, before I dive into the agentic phase, there’s another lesson that shaped my thinking just as much as writing my first lines of AI code: the human in the loop.