App server integration
Direct client-to-gateway calls work without an app server. Use @preternatural-js/self-hosted-app-server-sdk when your application needs its own chat route, server-side orchestration or a resumable-stream backend. It still calls Modem as a customer and uses the customer’s product rules; it is not a separate deployment of Modem.
Complete package access and registry setup before installing the server package.
bun add @preternatural-js/self-hosted-app-server-sdkRelay a customer request
import { createSelfHostedAppServer } from '@preternatural-js/self-hosted-app-server-sdk'
const server = createSelfHostedAppServer({ environment: 'production' })
export async function POST(request: Request): Promise<Response> {
const chat = server.createChatRequest(await request.json())
const text = server.streamChat(chat)
return new Response(text.pipeThrough(new TextEncoderStream()), {
headers: { 'Content-Type': 'text/plain; charset=utf-8' }
})
}The input is { messages, preternatural: { auth, scope } }. auth is the customer’s session, including customer/project IDs, access/refresh tokens and Sandbox mode. scope names provider, model and product. createChatRequest() validates this shape, then streamChat() uses a customer client. streamChatFromBody() combines both steps.
This example shows the streaming boundary, not a complete server authentication policy. Bind each request to your signed-in app user, validate its origin and scope, and translate validation/access errors into the response format your app expects. Do not accept an arbitrary third party’s posted customer session as authority to read a saved chat. Avoid logging request bodies containing refresh tokens.
Cancellation of the returned stream cancels its gateway source. Client disconnects, server execution limits and provider consumption remain distinct; cancellation does not refund generated work automatically.
Add resumability
Resumability is opt-in. The runtime needs an active-stream registry and a backend that can reconnect to an existing stream. The Redis adapter supplies both:
import { createIoredisResumableChatRuntime } from '@preternatural-js/self-hosted-app-server-sdk/ioredis'
const runtime = createIoredisResumableChatRuntime({
redisUrl: process.env.REDIS_URL,
activeStreamKeyPrefix: 'my-app:chat',
resumableStreamKeyPrefix: 'my-app:stream',
activeStreamTtlSeconds: 300
})Provide redisUrl or a Redis client. For local Redis, use the connection URL reported by your infrastructure rather than assuming a fixed port. Use separate key prefixes for isolated applications or checkouts.
Start with await runtime.create(chatId, sourceTextStream). Reconnect with await runtime.resume(chatId). A null result means no active resumable stream was found; it does not mean the original generation failed or should be replayed. The active marker defaults to 300 seconds and completion clears only the matching stream ID. A newer stream for the same chat is not cleared by an older stream finishing.
Authorize the chat owner on both start and resume routes. An opaque chatId alone is not access control. Arrange your host’s background-work lifetime through the adapter’s waitUntil option where required; storing a marker does not keep a serverless process alive.
Vercel AI SDK
The package exposes separate browser and server subpaths. Install a compatible ai peer dependency (the package declares version 6 or later).
import { createPreternaturalTextStreamChatTransport } from '@preternatural-js/self-hosted-app-server-sdk/vercel-ai-sdk/client'
const transport = createPreternaturalTextStreamChatTransport({
chatId: 'your-owned-chat-id',
getChatContext: () => ({
auth: currentCustomerSession,
scope: {
productId: 'your-product-id',
provider: selectedProvider,
modelIdentifier: selectedModel
}
}),
resumable: true
})The default non-resumable route is /api/chat. Resumable sends go to /api/chat/{chatId} and reconnects to /api/chat/{chatId}/stream. Configure routes to match your application. prepareBody, prepareRequest and prepareReconnectRequest customize application-specific fields/headers without replacing session context construction.
Wire the start and resume routes
The browser transport above needs both POST /api/chat/{chatId} and GET /api/chat/{chatId}/stream. The GET reconnects to existing work; it must never start another generation. Use the same configured server and runtime from the examples above:
import {
errorToJsonResponse,
ServerSdkHttpError,
type ServerChatRequest,
} from '@preternatural-js/self-hosted-app-server-sdk'
import {
createVercelAiTextStreamResponse,
createVercelAiResumableStreamResponse,
} from '@preternatural-js/self-hosted-app-server-sdk/vercel-ai-sdk/server'
type ChatAccess = {
session: ServerChatRequest['session']
scope: ServerChatRequest['request']['scope']
}
export function createChatRoutes(
authorizeChat: (request: Request, chatId: string) => Promise<ChatAccess>,
) {
return {
async start(request: Request, chatId: string): Promise<Response> {
try {
const access = await authorizeChat(request, chatId)
const chat = server.createChatRequest(await request.json())
const requested = chat.request.scope
if (
chat.session.customerId !== access.session.customerId ||
chat.projectId !== access.session.projectId ||
chat.session.sandboxMode !== access.session.sandboxMode ||
requested.productId !== access.scope.productId ||
requested.provider !== access.scope.provider ||
requested.modelIdentifier !== access.scope.modelIdentifier
) {
throw new ServerSdkHttpError('Chat scope does not match', 403)
}
// Adopt the server-authorized session, not tokens supplied in the body.
const source = server.streamChat({ ...chat, session: access.session })
const stream = await runtime.create(chatId, source)
return createVercelAiTextStreamResponse(stream)
} catch (error) {
return errorToJsonResponse(error)
}
},
async resume(request: Request, chatId: string): Promise<Response> {
try {
await authorizeChat(request, chatId)
const stream = await runtime.resume(chatId)
return stream
? createVercelAiResumableStreamResponse(stream)
: new Response(null, { status: 204 })
} catch (error) {
return errorToJsonResponse(error)
}
},
}
}authorizeChat is an application function you implement. Authenticate the app user on both routes, load the chat’s owner and allowed project/product/model from trusted storage, and return that customer’s current server-held Modem session. Reject unauthorized callers before touching the stream registry. Include mode and project in the chat’s ownership boundary; the same chat ID must not identify unrelated chats in different tenants. Validate origin/CSRF requirements for your app’s authentication. Parsing the posted session is not identity verification, and a resume GET does not contain a new customer session to verify through the gateway.
Mount start and resume at the POST and GET paths above, passing the route’s chat ID. These handlers expect the text-stream client transport; pair its start response with createVercelAiTextStreamResponse and its reconnect response with createVercelAiResumableStreamResponse. A 204 means no resumable stream is available. Keep the saved chat visible and offer an explicit new turn; do not replay the previous POST automatically.
The runtime is not a turn queue or an idempotency store. Serialize starts for each chat and reject a second start while its turn is running, using your application’s shared state. Calling create() again can replace the active marker; it does not deduplicate the generation. Preserve a durable turn identifier if your app retries an uncertain POST.
Disconnects, cancellation and errors
The resumable runtime tees the source: the browser reads one branch while the backend drains the other. Closing the browser connection therefore does not reliably cancel generation; that continuing work is what makes reconnect possible. Do not wire a transient HTTP disconnect to the source’s abort signal when continuation is intended. A deliberate Stop action needs application-owned cancellation of the source, for example a server-side customer chat request with an AbortSignal, plus cleanup of the application’s active-turn state. The runtime does not expose a public stop method. Cancellation does not refund accepted provider work.
The route’s catch handles failures before returning the response. A later provider error travels through the response stream; it cannot be replaced with a fresh JSON response after headers were sent. Preserve partial text and mark the answer interrupted. Configure the runtime’s onError for background-drain failures and your host’s waitUntil when needed. Verify normal completion, disconnect/reconnect, missing-stream 204, unauthorized resume, duplicate starts and explicit cancellation in your deployed host’s execution model.