Building a Chatbot

A production chat experience end to end: a conversational prompt, a thin backend, and a streaming React frontend.

This guide builds a real-time chatbot on PromptJuggler: your user types a message, the answer streams into the page token by token, and the conversation remembers itself. It takes one prompt, two small backend endpoints, and one React component.

1. The prompt

Create a prompt (say assistant) with read-write memory — it sees the conversation so far and appends its replies to it (see Memory & Threads). Give it your system instructions and an input variable for the user’s message, then publish it to production.

2. The backend

Two endpoints, both thin. One mints the stream credential, one triggers runs. Your API key stays here; the browser never sees it.

import { PromptJuggler } from '@promptjuggler/sdk';
import { randomUUID } from 'node:crypto';

const pj = new PromptJuggler(process.env.PROMPTJUGGLER_API_KEY);

// The stream credential: scoped to one thread, valid 24h. Authenticate your
// own user first — whoever holds this can read that thread's stream.
app.get('/api/chat/:thread/stream-token', async (req, res) => {
  res.json(await pj.createStreamToken(req.params.thread));
});

// One message: trigger a run on the thread. Returns immediately; the answer
// arrives on the stream.
app.post('/api/chat/:thread/messages', async (req, res) => {
  const run = await pj.runPrompt('assistant', 'production', {
    message: req.body.message,
  }, { thread: req.params.thread });
  res.json({ runId: run.id });
});

The thread ID is yours to choose — generate a UUID per conversation (randomUUID()), keep it in your session or database, and pass the same one to both endpoints. The thread doesn’t need to exist before the first run; minting a stream token for it first is exactly the right order.

3. The frontend

Connect before sending the first message — a fresh subscription starts at the live tip, so open the stream first and you see every token from the beginning.

import { useState } from 'react';
import { usePromptJugglerStream } from '@promptjuggler/browser/react';

export function Chat({ threadId }: { threadId: string }) {
  const [draft, setDraft] = useState('');
  const { connected, runs } = usePromptJugglerStream(
    {
      getToken: () =>
        fetch(`/api/chat/${threadId}/stream-token`).then((r) => r.json()),
    },
    [threadId],
  );

  const send = async () => {
    setDraft('');
    await fetch(`/api/chat/${threadId}/messages`, {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({ message: draft }),
    });
  };

  return (
    <div>
      {Object.entries(runs).map(([runId, run]) => (
        <div key={runId} className={run.status === 'failed' ? 'error' : ''}>
          {run.text}
          {run.status === 'streaming' && <Cursor />}
        </div>
      ))}
      <input value={draft} onChange={(e) => setDraft(e.target.value)} />
      <button onClick={send} disabled={!connected}>
        Send
      </button>
    </div>
  );
}

That’s the whole loop: send triggers a run, the run streams into runs[runId].text, and status flips to done when it finishes. Reconnects, retries, and workflow segments are handled inside the hook.

When a terminal state carries gapped: true, the streamed text may be incomplete — fetch the run through your backend (pj.getPromptRun(runId)) and swap in the authoritative output. With connect-before-send this is rare, but handling it is what makes the UI trustworthy.

Showing the user’s side

The stream carries the model’s output; your user’s messages you already have locally. Render them optimistically from your own state, keyed however you like — the runs map and your local messages interleave naturally by time. On a full page reload, rebuild the transcript from your backend (the thread’s runs via the API, or your own message store) and let the stream take over for new messages.

Structured data alongside the prose

Sometimes the answer isn’t only text. An assistant that pulls up records wants to stream its reply and hand the frontend the ids to render as cards. Add an emit tool to the prompt — a tool whose schema-validated arguments are the result — and its payloads arrive on the same stream as run.data, beside the text you already render.

Give the tool a payload schema (say { records: [{ id, title }] }) and tell the model to emit when it has matches. Then render the data next to the prose:

{Object.entries(runs).map(([runId, run]) => (
  <div key={runId}>
    {run.text}
    {run.data?.map((item, i) => (
      <RecordCard key={i} {...item.payload} /> // payload matches your schema
    ))}
  </div>
))}

run.data is the ordered list of { tool, payload } the model emitted this run, maintained for you exactly like text — cards appear the moment the model produces them, while the prose keeps streaming. The same payloads are on the run’s emitted field if you need the authoritative copy after a gapped terminal state.

Showing what the model is doing

If the prompt has tools, the user watches a pause with nothing to explain it — a knowledge search or an HTTP call can take seconds. run.transcript is the same run as an ordered sequence — prose, tools, emit payloads in position — so you can render the activity instead of a blank gap:

{Object.entries(runs).map(([runId, run]) => (
  <div key={runId}>
    {run.transcript?.map((item, i) => {
      switch (item.type) {
        case 'text':
          return <Markdown key={i}>{item.content}</Markdown>;
        case 'tool':
          return <ToolChip key={i} name={item.name} status={item.status} />;
        case 'data':
          return <RecordCard key={i} {...item.payload} />;
      }
    })}
    {run.status === 'streaming' && <Cursor />}
  </div>
))}

A tool chip appears the moment the model calls the tool, with status: 'pending', and flips to ok or error when it comes back. transcript replaces text and data in your render — it contains both — and it is the same shape pj.getPromptRun(runId) returns, so the same component renders a live run and a reloaded one. See Transcript for the full shape.

Workflows instead of a single prompt

Swap runPrompt for runWorkflow and nothing else changes — every prompt node streams into the same connection. If the workflow has internal lanes (research, tool use, synthesis), put the user-facing node on a dedicated channel and pass channels: ['support'] to the hook so only that lane renders.