Chat completions

Every chat method runs as the authenticated customer and bills the customer’s product.

Method	Returns
`client.chat.complete(request)`	A promise of the whole `ChatCompletion`.
`client.chat.stream(request, onText)`	A promise of the whole text. `onText` receives each piece as it arrives.
`client.chat.streamChunks(request)`	A `ReadableStream` of `ChatCompletionChunk` objects.
`client.chat.streamText(request)`	A `ReadableStream` of text pieces.
`client.chat.createProviderClient(scope)`	A promise of an OpenAI client scoped to the customer.

Request shape


import type { ChatCompletionMessageParam } from 'openai/resources/index.mjs'
import type { ServiceProvider } from '@preternatural-js/client-sdk'
 
type ChatRequest = {
  messages: ChatCompletionMessageParam[]
  scope: {
    modelIdentifier: string
    provider: ServiceProvider
    productId?: string
  }
  temperature?: number
  maxTokens?: number
  signal?: AbortSignal
}

Field	Description
`messages`	The OpenAI-compatible message array sent to the provider.
`scope.provider`	The provider that serves the model, as a `ServiceProvider` value.
`scope.modelIdentifier`	The provider’s model identifier.
`scope.productId`	The product to bill. Defaults to the project’s default product.
`temperature`	The sampling temperature passed to the provider. Defaults to `0.7`.
`maxTokens`	The maximum number of output tokens. Defaults to `1000`.
`signal`	An optional `AbortSignal` that cancels the request.

Streaming


import { ServiceProvider } from '@preternatural-js/client-sdk'
 
const result = await client.chat.stream(
  {
    messages: [
      {
        role: 'user',
        content: 'Explain why the sky appears blue.'
      }
    ],
    scope: {
      provider: ServiceProvider.openai,
      modelIdentifier: 'gpt-4o-mini',
      productId: 'your-product-id'
    },
    temperature: 0.7,
    maxTokens: 500
  },
  chunk => {
    appendToInterface(chunk)
  }
)

The callback gets each piece of text, and the promise resolves to the whole text.

See Streaming chat completions.

Non-streaming


const result = await client.chat.complete({
  messages: [
    {
      role: 'user',
      content: 'Explain why the sky appears blue.'
    }
  ],
  scope: {
    provider: ServiceProvider.openai,
    modelIdentifier: 'gpt-4o-mini',
    productId: 'your-product-id'
  },
  temperature: 0.7,
  maxTokens: 500
})

The promise resolves to an OpenAI-compatible ChatCompletion object.

See Non-streaming chat completions.

Provider support

The Client SDK sends OpenAI and Anthropic chat completions through the OpenAI-compatible API. The Anthropic SDK uses Anthropic’s Messages API instead. See Use Anthropic with Modem.

Jev uses the System One API, not chat completions. See Use Jev with Modem.

Errors

A refused request rejects with the OpenAI SDK’s APIError. See Gateway errors.

Advanced usage

client.chat.createProviderClient() returns the OpenAI client itself. Its requests and streams end when the customer’s session ends.


const providerClient = await client.chat.createProviderClient({
  provider: ServiceProvider.openai,
  modelIdentifier: 'gpt-4o-mini',
  productId: 'your-product-id'
})
 
const completion = await providerClient.chat.completions.create({
  model: 'gpt-4o-mini',
  messages: [
    {
      role: 'user',
      content: 'Hello'
    }
  ],
  stream: false
})