Get customer usage

client.usage.retrieve() returns the usage limits and rate limits that apply to the signed-in customer.

Read usage allowances


const usage = await client.usage.retrieve()
 
for (const limit of usage.usageLimits) {
  console.log({
    metric: limit.allowanceMetric,
    used: limit.rawAmount,
    allowance: limit.allowanceValue,
    remaining: limit.remaining,
    usedPercentage: limit.usedPercentage,
    scopeKind: limit.scopeKind,
    scopeValue: limit.scopeValue
  })
}

Each usage-limit item contains:

Field	Type	Description
`id`	`string`	The usage record ID.
`rawAmount`	`number`	The amount used so far in the record’s billing period.
`allowanceValue`	`number`	The allowance set on the product.
`allowanceMetric`	`string`	The unit being measured. It is one of `numberOfRequest`, `numberOfTokens`, `numberOfCharacters`, `numberOfImages`, `audioFileSizeBytes`, or `audioDurationMilliseconds`.
`remaining`	`number`	The allowance minus the amount used. It can be below zero.
`usedPercentage`	`number`	The amount used as a percentage of the allowance, rounded to one decimal place. It is `0` when the allowance is `0`.
`scopeKind`	`string`	How the rule matches models. It is `modelIdentifier` or `requestRegexPatterns`.
`scopeValue`	`string`	The model identifier, or the regular expression that is matched against the model identifier.

Read rate-limit rules

The same response contains the customer’s rate-limit rules:


for (const limit of usage.rateLimits) {
  console.log({
    allowance: limit.allowanceValue,
    metric: limit.allowanceMetric,
    frequency: limit.rateLimitFrequency,
    scopeKind: limit.scopeKind,
    scopeValue: limit.scopeValue
  })
}

rateLimitFrequency is minute, hour, or day. A rate-limit entry describes the rule only, not the usage in the current window.

Get the aggregate percentage

Use client.usage.retrievePercentage() when the interface needs one percentage:


const { percentage } = await client.usage.retrievePercentage()
 
renderUsageBar(percentage)

The response is:


type CustomerUsagePercentage = {
  percentage: number
}

The percentage is the average of the usedPercentage values of the customer’s usage limits. The request fails when the customer has no usage limit records.

How limits are enforced

Limits on provider-reported usage, such as tokens, are soft caps. When recorded usage reaches the allowance, new requests are refused. Requests already running finish and can go over. No overage is billed. See Gateway errors for the refusal.

Tokens come from the provider’s usage report. When a response has no token count, the gateway estimates it from the text and labels it as an estimate. A provider count always replaces an estimate. Estimates count against allowances, not against provider usage or pricing.

The dashboard’s Customer requests view labels each request’s tokens as Reported provider usage, Estimated usage, or Mixed provider and estimated usage.