Low-level Streaming—InferaGraph

When to use this

Most apps want Chat API — it streams text and dispatches tool calls in one pass. Reach for the lower-level streaming surface here when you need raw tokens without the chat tool layer: side panels, summarize-this buttons, off-graph generation, or tests.

AIEngine.queryStream()

High-level streaming with automatic context building from the current graph. Yields { type: 'text' | 'done' | 'error', content } chunks. No tool calls, no graph mutation — pure model output.

import { useState } from 'react';
import { GraphStore, QueryEngine, AIEngine } from '@inferagraph/core/data';
import { AnthropicProvider } from '@inferagraph/anthropic-provider';

const store = new GraphStore();
const provider = new AnthropicProvider({ apiKey: 'sk-ant-...' });
const ai = new AIEngine(store, new QueryEngine(store));
ai.setProvider(provider);

function StreamingChat() {
  const [response, setResponse] = useState('');
  const [streaming, setStreaming] = useState(false);

  const handleQuery = async () => {
    setResponse('');
    setStreaming(true);

    for await (const chunk of ai.queryStream('Tell me about Abraham's journey to Canaan')) {
      switch (chunk.type) {
        case 'text':
          setResponse(prev => prev + chunk.content);
          break;
        case 'done':
          setStreaming(false);
          break;
        case 'error':
          console.error('Error:', chunk.content);
          setStreaming(false);
          break;
      }
    }
  };

  return (
    <div>
      <button onClick={handleQuery} disabled={streaming}>Ask</button>
      <p>{response}</p>
    </div>
  );
}

Provider-level stream()

Lower still: every LLMProvider exposes a native stream() AsyncIterable. Bypass AIEngine entirely when you want to control the prompt yourself or stream from contexts that don't have a graph (e.g., a detail panel summarizing a single node's content).

Providers that don't override stream() automatically fall back to complete(), yielding a single text chunk followed by done.

import { useState } from 'react';
import { LLMProvider } from '@inferagraph/core/data';
import { AnthropicProvider } from '@inferagraph/anthropic-provider';

const provider = new AnthropicProvider({ apiKey: 'sk-ant-...' });

function ProviderStream() {
  const [result, setResult] = useState('');

  const handleStream = async () => {
    setResult('');

    for await (const chunk of provider.stream({
      messages: [{ role: 'user', content: 'Describe the Exodus' }],
    })) {
      if (chunk.type === 'text')
        setResult(prev => prev + chunk.content);
    }
  };

  return (
    <div>
      <button onClick={handleStream}>Stream</button>
      <p>{result}</p>
    </div>
  );
}

// Custom provider: override stream() or use default fallback
class MyProvider extends LLMProvider {
  async *stream(request) {
    yield { type: 'text', content: 'Jacob had twelve sons...' };
    yield { type: 'done', content: '' };
  }
}