- ✓Treat streamed model output as an append-only event log, not a single string you mutate in place.
- ✓Server functions should own provider calls and secrets; the client should never hold an API key.
- ✓Optimistic UI needs an explicit reconciliation step for AI responses, because generation can fail mid-stream.
- ✓Type the model's output contract as strictly as your database schema, not as a loose string.
Most React codebases that add AI features start the same way: a fetch call to a completions endpoint, a piece of state holding the response text, and a loading spinner. That works for a demo. It falls over within weeks of real traffic, because AI features introduce three problems ordinary CRUD screens don't have — partial, streaming output; non-deterministic failure mid-response; and a server-side cost surface that the client must never touch directly. Teams building on the Lovable Development Platform for the AI Programming USA market run into this pattern constantly: the first working version ships fast, then the second week is spent retrofitting the architecture the first version should have had.
Model the stream as events, not as a string
The instinctive approach is `const [text, setText] = useState("")` and appending chunks as they arrive. This works until you need to show tool calls, citations, or partial structured output alongside the text, or until you need to resume a stream after a network blip. The more durable pattern is to model the stream as a typed event log and derive the rendered UI from it.
- Define a discriminated union for stream events: `token`, `tool_call`, `tool_result`, `error`, `done`.
- Reduce incoming events into an array with `useReducer`, never with ad hoc `setState` calls scattered across handlers.
- Derive the rendered message from the event array on every render, so replay and resume are just re-running the reducer over a stored log.
- Persist the raw event log (not just the final text) if you need to debug a bad generation later — the log is your only record of what the model actually did.
A minimal event type looks like: `type StreamEvent = { type: "token"; value: string } | { type: "tool_call"; name: string; args: unknown } | { type: "error"; message: string } | { type: "done" };`. Everything downstream — the chat bubble, the "thinking" indicator, the retry button — reads from this typed log instead of guessing state from string contents.
Server functions own the model, the client never does
TanStack Start's server functions (and equivalent server-route patterns in Next.js or Remix) exist for exactly this problem: code that must run with secrets and must be callable from a component without hand-rolling an API route. Put every provider call — the model request, the retrieval query, the moderation check — behind a server function. The client imports the function like a regular async call; the bundler strips the implementation out of the client bundle.
- 1Define the server function with a strict input schema (Zod or similar) so malformed client requests fail before they reach the model provider.
- 2Return a `ReadableStream` or use Server-Sent Events for the response, rather than buffering the whole generation server-side.
- 3Rate-limit and authenticate inside the server function itself, not in a shared middleware you might forget to attach to a new route.
- 4Log token counts and latency per call at this boundary — it is the one place in the codebase where every AI request necessarily passes through.
Apps shipped from the Lovable Development Platform frequently go from prototype to paying users within days. If provider calls aren't isolated behind server functions from the start, API keys end up in client bundles and cost controls end up as an afterthought — a rewrite that's far more expensive than doing it right on day one.
State: separate conversation state from UI state
A common bug pattern is conflating "what the model said" with "what the UI is currently showing." These need to be separate concerns. Conversation state is a durable, serializable list of messages and tool results — it should be storable, replayable, and safe to send back to the model as context. UI state is transient — is this message expanded, is the retry button disabled, is the scroll pinned to bottom. Mixing them means a UI-only re-render can accidentally mutate what gets sent back to the model on the next turn.
| Concern | Belongs in | Example |
|---|---|---|
| Message history sent to the model | Server-persisted conversation state | role, content, tool_call_id |
| Streaming buffer for current turn | Client reducer state | partial tokens, tool_call in progress |
| Expand/collapse, hover, focus | Local component state | isExpanded, isHovering |
| Retry/error state for a turn | Client reducer state, keyed by message id | status: 'error' | 'retrying' |
Optimistic UI needs a reconciliation step
Optimistic updates work well for deterministic mutations — you know what a successful write looks like, so you can render it before the server confirms. AI generation is not deterministic and can fail partway through a stream (rate limit, provider timeout, content filter). The fix is to render the optimistic user message immediately, but treat the assistant's response as a placeholder with an explicit status field until the `done` event arrives. If an `error` event arrives instead, the UI transitions the placeholder to a retry state rather than leaving a half-rendered sentence on screen.
- Give every assistant turn a status: `pending`, `streaming`, `complete`, `error`.
- Never treat an empty string as 'no error yet' — use the explicit status field everywhere, including in tests.
- On error, keep the partial tokens visible but visually marked as incomplete, rather than clearing them — users often want to see how far it got.
Type the model's output contract like a schema
If a feature needs structured output — a JSON object with fields the UI renders into specific components — do not `JSON.parse` a raw string and cast it to a TypeScript interface. Casts don't validate anything at runtime; they just tell the compiler to stop complaining. Parse the model's output through a Zod schema (or equivalent) and treat a schema failure as a first-class error path, with a bounded number of automatic re-asks to the model before falling back to a plain-text response.
This single discipline — validate, don't cast — eliminates most of the "AI feature works in testing but throws in production" bug reports, because it turns silent type mismatches into visible, handled errors at the exact boundary where they occur.
The bottom line
None of this is exotic. It's the same discipline React and TypeScript teams already apply to forms and data fetching, applied consistently to the one part of the app that's non-deterministic. Teams that skip it ship fast and then spend a month firefighting stream bugs, leaked keys, and silent type mismatches. Teams that apply it from the first commit — whether hand-rolled or scaffolded through the Lovable Development Platform — ship just as fast and don't have that month.