Skip to Content
ProvidersUse OpenAI with Modem

OpenAI

Send OpenAI chat completions with the Client SDK’s chat methods, or with the OpenAI SDK pointed at the Modem gateway.

Using the Client SDK

Non-streaming completion

import { ServiceProvider } from '@preternatural-js/client-sdk'
 
const result = await client.chat.complete({
  scope: {
    provider: ServiceProvider.openai,
    modelIdentifier: 'gpt-4o-mini',
    productId: 'your-product-id',
  },
  messages: [
    {
      role: 'user',
      content: 'Explain quantum computing in one paragraph.',
    },
  ],
  maxTokens: 500,
  temperature: 0.7,
})
 
const message = result.choices[0]?.message.content

client.chat.complete() returns an OpenAI-compatible ChatCompletion. temperature and maxTokens default to 0.7 and 1000.

Streaming completion

client.chat.stream() passes each piece of text to the callback and resolves to the whole text.

const text = await client.chat.stream(
  {
    scope: {
      provider: ServiceProvider.openai,
      modelIdentifier: 'gpt-4o-mini',
      productId: 'your-product-id',
    },
    messages: [
      {
        role: 'user',
        content: 'Write a short story about a lunar gardener.',
      },
    ],
    maxTokens: 500,
    temperature: 0.7,
  },
  (chunk) => {
    console.log(chunk)
  },
)

Using the OpenAI SDK

Get the gateway configuration from the Client SDK and pass it to the OpenAI client:

import OpenAI from 'openai'
import { ServiceProvider } from '@preternatural-js/client-sdk'
 
const gateway = await client.gateway.providerConfiguration({
  provider: ServiceProvider.openai,
  modelIdentifier: 'gpt-4o-mini',
  productId: 'your-product-id',
})
 
const openai = new OpenAI({
  apiKey: 'PRETERNATURAL_GATEWAY_AUTHENTICATES_REQUESTS',
  baseURL: gateway.baseURL,
  defaultHeaders: gateway.headers,
  fetch: gateway.fetchImpl,
  maxRetries: 0,
})

Do not pass an OpenAI API key. The Client SDK’s headers authenticate the request, and apiKey only satisfies the OpenAI SDK. The gateway uses the key of the product’s provider account. maxRetries: 0 matches the Client SDK, because a retried completion can be billed twice.

Non-streaming completion

const completion = await openai.chat.completions.create({
  model: 'gpt-4o-mini',
  messages: [
    {
      role: 'user',
      content: 'Explain quantum computing in one paragraph.',
    },
  ],
  max_tokens: 500,
  temperature: 0.7,
})
 
const message = completion.choices[0]?.message.content

Streaming completion

const stream = await openai.chat.completions.create({
  model: 'gpt-4o-mini',
  messages: [
    {
      role: 'user',
      content: 'Write a short story about a lunar gardener.',
    },
  ],
  max_tokens: 500,
  temperature: 0.7,
  stream: true,
})
 
for await (const event of stream) {
  const text = event.choices[0]?.delta.content
 
  if (text) {
    console.log(text)
  }
}

More info

Use Jev with ModemSandbox mode