Transports

Run the AIEngine in-process or behind an HTTP/SSE endpoint. The same Chat API works either way.

Transports

A Transport decides where the LLM runs. useInferaGraphChat() doesn't care: it consumes the same AsyncIterable<ChatEvent> shape regardless of whether the events come from in-process AI calls or from an SSE stream over HTTP.

inProcessTransport

Default. Runs the AIEngine in the same JS context as the renderer. Fastest path for prototypes, demos, and apps with browser-safe key strategies.

httpTransport

Browser-side proxy. POSTs the message to your endpoint and parses an SSE response back into ChatEvents. Recommended for production — keys, cache, and the LLM stay server-side.

In-process

Pass an llm prop and the in-process transport is built automatically. You only need inProcessTransport() directly when you want to share an engine or wrap it.

Implicit (recommended)

import { InferaGraph } from '@inferagraph/core/react';
import { openaiProvider } from '@inferagraph/openai-provider';

// Default. The browser-bound AIEngine calls the LLM directly.
// Pair with a browser-safe key strategy (signed short-lived
// tokens, demo-only keys, etc.).
<InferaGraph
  data={data}
  llm={openaiProvider({ apiKey: 'pk-...' })}
/>

Explicit

import { InferaGraph } from '@inferagraph/core/react';
import { inProcessTransport } from '@inferagraph/core/data';
import { openaiProvider } from '@inferagraph/openai-provider';

// Equivalent to the implicit form. Useful when you want to
// reuse one AIEngine across multiple consumers, or need to
// inspect/decorate the transport.
const transport = inProcessTransport({
  provider: openaiProvider({ apiKey: 'pk-...' }),
});

<InferaGraph data={data} transport={transport} />

HTTP transport (Next.js example)

The browser sends { message, emitToolCalls } as JSON. Your route runs the AIEngine and re-emits each ChatEvent as a data: line of an SSE stream. Blank lines delimit events; the transport ignores malformed lines so a single bad event doesn't break the stream.

0.10.1 — Transport.reconstructChatEvent now parses set_inferred_visibility events from the SSE wire format. Pre-0.10.1 the reconstructor silently dropped them despite the type existing in the union since Phase 5.

Browser

import { InferaGraph } from '@inferagraph/core/react';
import { httpTransport } from '@inferagraph/core/data';

// Browser POSTs to your route, the route runs AIEngine
// server-side, and SSE streams ChatEvents back.
<InferaGraph
  data={data}
  transport={httpTransport({
    url: '/api/chat',
    headers: { 'X-User': currentUserId },  // optional
  })}
/>

Server route

// app/api/chat/route.ts (Next.js App Router)
import { AIEngine, GraphStore, QueryEngine } from '@inferagraph/core/data';
import { openaiProvider } from '@inferagraph/openai-provider';
import { redisCacheProvider } from '@inferagraph/redis';

// Build the engine once at module scope. The store + cache
// survive between requests in a long-lived Node runtime.
const store = new GraphStore();
// ...load nodes/edges from your database into store...
const ai = new AIEngine(store, new QueryEngine(store));
ai.setProvider(openaiProvider({ apiKey: process.env.OPENAI_API_KEY }));
ai.setCache(redisCacheProvider({ url: process.env.REDIS_URL }));

export async function POST(req: Request) {
  const { message, emitToolCalls } = await req.json();
  const stream = new ReadableStream({
    async start(controller) {
      const encoder = new TextEncoder();
      try {
        for await (const event of ai.chat(message, { emitToolCalls })) {
          controller.enqueue(
            encoder.encode(`data: ${JSON.stringify(event)}\n\n`),
          );
        }
      } finally {
        controller.close();
      }
    },
  });
  return new Response(stream, {
    headers: {
      'Content-Type': 'text/event-stream',
      'Cache-Control': 'no-cache, no-transform',
    },
  });
}

The same pattern works in any framework that exposes a streaming Response body — Hono, Cloudflare Workers, Fastify with reply.raw, etc.

Custom transports

The contract is intentionally tiny — implement chat(message, opts) returning an AsyncIterable<ChatEvent> and you have a transport. Useful for queue-backed dispatch, retry, or multiplexing.

import type { Transport, ChatEvent } from '@inferagraph/core/data';

// A transport is any object whose chat(message, opts) returns
// an AsyncIterable<ChatEvent>. Implement once for queue-backed,
// retry-enabled, or fan-out routing.
const queuedTransport: Transport = {
  async *chat(message, opts) {
    const jobId = await enqueueChat(message);
    for await (const ev of subscribeToJob(jobId, opts?.signal)) {
      yield ev as ChatEvent;
    }
  },
};