Responses, images and audio uploads
The gateway supports more than Chat Completions. This recipe uses an OpenAI API-key provider account, a customer session and an existing product. ChatGPT subscription relays have a separate protocol and do not provide these image/audio routes.
First connect the provider account, configure the product’s model rules and establish the customer session. Select a model that supports the operation and is available to that account. The public catalog is not a live entitlement check.
Configure a client for each operation
Install openai in your application. Call gateway.providerConfiguration for each operation. Here client is your authenticated Modem customer client and productId is its intended product:
import OpenAI from 'openai'
import { ExternalServiceProvider } from '@preternatural-js/client-sdk'
async function openAIFor(modelIdentifier: string) {
const gateway = await client.gateway.providerConfiguration({
provider: ExternalServiceProvider.openai,
productId,
modelIdentifier,
})
return new OpenAI({
apiKey: 'PRETERNATURAL_GATEWAY_AUTHENTICATES_REQUESTS',
baseURL: gateway.baseURL,
defaultHeaders: gateway.headers,
fetch: gateway.fetchImpl,
maxRetries: 0,
dangerouslyAllowBrowser: true, // Omit in server-only code.
})
}The placeholder satisfies the provider SDK; Modem’s customer headers authenticate the call. Never substitute your upstream OpenAI key in browser code. Refresh this configuration when the customer changes, and cancel external SDK requests/streams yourself on sign-out or account switching. The configuration is a snapshot; it does not install all of the high-level chat helper’s session listeners. See gateway configuration.
The functions below accept actual configured model IDs. Their input files come from your file picker or server upload, rather than placeholder file paths.
Responses
async function respond(model: string, input: string, signal: AbortSignal) {
const openai = await openAIFor(model)
const response = await openai.responses.create({
model,
input,
store: false,
}, { signal })
return { status: response.status, text: response.output_text, output: response.output }
}Inspect the status and output items; output_text is a convenient text projection, not every possible tool or reasoning item. The basic Modem client.chat helper uses Chat Completions and does not expose Responses-specific fields.
For streaming, consume typed Responses events rather than Chat Completions choices:
async function streamResponse(
model: string,
input: string,
signal: AbortSignal,
appendText: (text: string) => void,
) {
const openai = await openAIFor(model)
const events = await openai.responses.create({
model,
input,
store: false,
stream: true,
}, { signal })
for await (const event of events) {
if (event.type === 'response.output_text.delta') appendText(event.delta)
if (event.type === 'error') throw new Error(event.message)
if (event.type === 'response.failed' || event.type === 'response.incomplete') {
throw new Error(`Response ended with status ${event.response.status}`)
}
}
}Preserve partial text if iteration fails. This gateway mounts POST /v1/responses; it does not implement the provider SDK’s separate response retrieval, deletion or cancellation endpoints. A returned response ID is not a Modem reconnect handle. Keep the history your application needs rather than assuming a subsequent GET can recover an interrupted response. Provider-specific tools and stateful continuation still require their own compatibility checks.
Generate or edit an image
async function generateImage(model: string, prompt: string, signal: AbortSignal) {
const openai = await openAIFor(model)
return openai.images.generate({ model, prompt, n: 1 }, { signal })
}
async function editImage(
model: string,
image: File,
prompt: string,
signal: AbortSignal,
) {
const openai = await openAIFor(model)
return openai.images.edit({ model, image, prompt, n: 1 }, { signal })
}Both return JSON with a data array. Depending on the model and requested output options, an entry contains b64_json or a url; do not call arrayBuffer() on the JSON result as if it were the image itself. Decode a base64 entry into bytes or display the returned URL, using the selected image format when constructing a Blob/data URL. Handle an empty data array and missing image fields explicitly. The SDK’s response_format and image-format options are model-specific; a format accepted by one image model is not portable to all others.
Editing uploads a file with multipart form data. Let the SDK construct the multipart boundary. Use the file types, dimensions and optional mask supported by your selected model; a successful JSON image-generation request does not verify edit compatibility. This route does not offer chat-style SSE image streaming.
Transcribe or translate an audio file
async function transcribe(model: string, file: File, signal: AbortSignal) {
const openai = await openAIFor(model)
const result = await openai.audio.transcriptions.create({
model, file, response_format: 'json',
}, { signal })
return result.text
}
async function translate(model: string, file: File, signal: AbortSignal) {
const openai = await openAIFor(model)
const result = await openai.audio.translations.create({
model, file, response_format: 'json',
}, { signal })
return result.text
}These are file uploads, distinct from speech synthesis. Choose a model that supports the particular endpoint; transcription support does not imply translation support. The examples request JSON text results. Other response formats can be text/subtitles and require different handling; do not unconditionally parse every audio API response as JSON or treat it as playable audio.
Rules, size and accounting
Use the generated protocols and measures inventory when creating rules. A matching Chat Completions rule is insufficient if its measure does not apply to the requested API. Responses and embeddings can use token measures; image operations use image counts; audio uploads can use audio byte/duration measures. Requests are also a supported measure. costMicros is currently limited to regular Chat Completions and Completions; it is not a spending-cap measure for every route listed here.
Audio byte measurement comes from the uploaded file. Duration measurement depends on a supported readable audio format; do not assume every accepted codec produces a measured duration. Missing measurements are not verified zero usage. See limits and estimates for admission, settlement and unavailable pricing.
The current transport ceilings are 50 MiB for JSON/image-edit request bodies and 26 MiB for audio uploads, including multipart overhead. Upstream model/file restrictions can be smaller. A 413 requires reducing the request. Do not label a multipart body as JSON, override its boundary manually or treat a model’s context window as an upload-size allowance.
Pass an AbortSignal from your app’s cancel control and keep provider SDK automatic retries disabled as above. An abort or lost response does not prove that the provider performed no work; it is not permission to regenerate an image or upload again automatically. Keep returned output and the gateway request ID when diagnosing an uncertain result. Measured usage, a Modem allowance debit and an API-parity estimate, when available, are separate from an upstream charge. An estimate is not a receipt or a provider credit balance.
For provider request options, see the official Responses , images , transcription and translation references and the OpenAI TypeScript SDK . Modem-specific authentication and provider compatibility, errors and operational recovery still apply.