Chat completions

Use client.streamChat() to send a streaming or non-streaming chat completion for the authenticated customer.

Request shape


type ChatStreamRequest = {
  messages: unknown
  scope: {
    modelIdentifier: string
    provider: ServiceProvider
    productId: string | undefined
  }
  temperature: number
  maxTokens: number
  streamMode: boolean
  signal?: AbortSignal
}

Field	Description
`messages`	The OpenAI compatible message array sent to the provider.
`scope.provider`	provider name.
`scope.modelIdentifier`	The provider’s model identifier.
`scope.productId`	Preternatural Product ID being referenced.
`temperature`	The sampling temperature passed to the provider.
`maxTokens`	The maximum number of output tokens.
`streamMode`	Set to `true` to receive chunks or `false` to wait for one completion object.
`signal`	An optional `AbortSignal` to cancel a streaming request.

Streaming


const result = await client.streamChat(
  {
    messages: [
      {
        role: 'user',
        content: 'Explain why the sky appears blue.'
      }
    ],
    scope: {
      provider: ServiceProvider.openai,
      modelIdentifier: 'gpt-4o-mini',
      productId: 'your-product-id'
    },
    temperature: 0.7,
    maxTokens: 500,
    streamMode: true
  },
  chunk => {
    appendToInterface(chunk)
  }
)

When streamMode is true, the SDK provides callback for each text chunk.

See Streaming chat completions.

Non-streaming


const result = await client.streamChat({
  messages: [
    {
      role: 'user',
      content: 'Explain why the sky appears blue.'
    }
  ],
  scope: {
    provider: ServiceProvider.openai,
    modelIdentifier: 'gpt-4o-mini',
    productId: 'your-product-id'
  },
  temperature: 0.7,
  maxTokens: 500,
  streamMode: false
})

When streamMode is false, OpenAI-compatible ChatCompletion response is returned.

See Non-streaming chat completions.

Return type

The current method signature returns a union:


Promise<string | ChatCompletion>

Narrow the result based on response mode:


const result = await client.streamChat(request)
 
if (request.streamMode) {
  if (typeof result !== 'string') {
    throw new Error('Expected streamed text')
  }
 
  renderAnswer(result)
} else {
  if (typeof result === 'string') {
    throw new Error('Expected a chat completion')
  }
 
  renderAnswer(result.choices[0]?.message.content ?? '')
}

Provider support

With the Client SDK, OpenAI and Anthropic both use the OpenAI-compatible chat-completion API. The official Anthropic SDK uses Anthropic’s Messages API instead. See Use Anthropic with Preternatural.

Jev does not use chat completions. Jev uses TypeSafe AI’s native System One API. See Use Jev with Preternatural.

Advanced usage

Use createProviderChatClient() when the application needs direct access to the OpenAI client:


const providerClient = await client.createProviderChatClient(
  {
    streamMode: false,
    scope: {
      provider: ServiceProvider.openai,
      modelIdentifier: 'gpt-4o-mini',
      productId: 'your-product-id'
    }
  },
  ServiceProvider.openai
)
 
const completion = await providerClient.chat.completions.create({
  model: 'gpt-4o-mini',
  messages: [
    {
      role: 'user',
      content: 'Hello'
    }
  ],
  stream: false
})