LlmInferenceGateway

1. The words of this server

2. The endpoints

MethodURLSummary
POST/LlmInferenceControlPlane/LlmAccessTokenInsertWithDisplayNameLlmAccessTokenInsertWithDisplayName
GET/LlmInferenceControlPlane/LlmAccessTokenListAllLlmAccessTokenListAll
GET/LlmInferenceControlPlane/LlmAccessTokenReadByTokenHashLlmAccessTokenReadByTokenHash
POST/LlmInferenceControlPlane/LlmAccessTokenRemoveByAccessTokenUidLlmAccessTokenRemoveByAccessTokenUid
POST/LlmInferenceControlPlane/LlmAccessTokenRenameByAccessTokenUidLlmAccessTokenRenameByAccessTokenUid
POST/LlmInferenceControlPlane/LlmAccessTokenRevokeByAccessTokenUidLlmAccessTokenRevokeByAccessTokenUid
POST/LlmInferenceControlPlane/LlmModelConfigurationInsertWithModelConfigurationUidLlmModelConfigurationInsertWithModelConfigurationUid
POST/LlmInferenceControlPlane/LlmModelConfigurationInsertWithModelNameLlmModelConfigurationInsertWithModelName
GET/LlmInferenceControlPlane/LlmModelConfigurationListAllLlmModelConfigurationListAll
GET/LlmInferenceControlPlane/LlmModelConfigurationReadByModelConfigurationUidLlmModelConfigurationReadByModelConfigurationUid
GET/LlmInferenceControlPlane/LlmModelConfigurationReadByModelNameLlmModelConfigurationReadByModelName
POST/LlmInferenceControlPlane/LlmModelConfigurationRemoveByModelConfigurationUidLlmModelConfigurationRemoveByModelConfigurationUid
POST/LlmInferenceControlPlane/LlmModelConfigurationUpdateByModelConfigurationUidLlmModelConfigurationUpdateByModelConfigurationUid
GET/LlmInferenceGateway/LlmAccessTokenValidateCurrentValidate Access Key
POST/LlmInferenceGateway/LlmAudioTranscriptionCrossCheckJobEnqueueWithFileContentHashSha256Enqueue the cross-checked transcription of a stored file
POST/LlmInferenceGateway/LlmAudioTranscriptionCrossCheckJobForceRemoveByAudioTranscriptionCrossCheckJobUidRemove an audio transcription cross-check job of any token
GET/LlmInferenceGateway/LlmAudioTranscriptionCrossCheckJobListAllList the audio transcription cross-check jobs of every token
GET/LlmInferenceGateway/LlmAudioTranscriptionCrossCheckJobListByAccessTokenList the audio transcription cross-check jobs of the token
GET/LlmInferenceGateway/LlmAudioTranscriptionCrossCheckJobReadByAudioTranscriptionCrossCheckJobUidRead an audio transcription cross-check job
POST/LlmInferenceGateway/LlmAudioTranscriptionCrossCheckJobRemoveByAudioTranscriptionCrossCheckJobUidRemove an audio transcription cross-check job
POST/LlmInferenceGateway/LlmAudioTranscriptionJobEnqueueWithFileContentHashSha256Enqueue the transcription of a stored file
POST/LlmInferenceGateway/LlmAudioTranscriptionJobForceRemoveByAudioTranscriptionJobUidRemove an audio transcription job of any token
GET/LlmInferenceGateway/LlmAudioTranscriptionJobListAllList the audio transcription jobs of every token
GET/LlmInferenceGateway/LlmAudioTranscriptionJobListByAccessTokenList the audio transcription jobs of the token
GET/LlmInferenceGateway/LlmAudioTranscriptionJobReadByAudioTranscriptionJobUidRead an audio transcription job
POST/LlmInferenceGateway/LlmAudioTranscriptionJobRemoveByAudioTranscriptionJobUidRemove an audio transcription job
POST/LlmInferenceGateway/LlmChatCompletionGenerateWithModelNameGenerate 1 chat completion with every sampling setting stated
POST/LlmInferenceGateway/LlmChatCompletionGenerateWithStreamLeaseUidRun chat on a stream lease
GET/LlmInferenceGateway/LlmInferenceLogDownloadAllDownload inference log
GET/LlmInferenceGateway/LlmInferenceLogLineListAllInference logs (raw paginated)
GET/LlmInferenceGateway/LlmInferenceLogListAllInference logs
POST/LlmInferenceGateway/LlmInferenceLogRemoveAllClear inference logs
POST/LlmInferenceGateway/LlmInferenceLogRemoveByDateToPrune inference logs
POST/LlmInferenceGateway/LlmInferenceServerInsertWithInferenceServerUidInsert an inference server with the uid of the sender
GET/LlmInferenceGateway/LlmInferenceServerListAllList every inference server
POST/LlmInferenceGateway/LlmInferenceServerRemoveByInferenceServerUidRemove a stale inference server row
POST/LlmInferenceGateway/LlmInferenceServerUpdateByInferenceServerUidUpdate the heartbeat of 1 inference server
GET/LlmInferenceGateway/LlmLoadableModelListAllList every model that 1 server can load and no server serves
POST/LlmInferenceGateway/LlmModelLoadByInferenceServerUidLoad a model on 1 inference server
GET/LlmInferenceGateway/LlmModelStatusListAllList every model with its status and its workers
POST/LlmInferenceGateway/LlmModelUnloadByInferenceServerUidUnload the model of 1 inference server
POST/LlmInferenceGateway/LlmServerLeaseAcquireWithInferenceServerUidAcquire a server lease
POST/LlmInferenceGateway/LlmServerLeaseForceReleaseByServerLeaseUidForce release a server lease
GET/LlmInferenceGateway/LlmServerLeaseListAllList the server leases of every token
GET/LlmInferenceGateway/LlmServerLeaseListByAccessTokenList server leases
POST/LlmInferenceGateway/LlmServerLeaseReleaseByServerLeaseUidRelease a server lease
GET/LlmInferenceGateway/LlmStoredFileBlockDownloadByBlockIndexDownload 1 block of a file
POST/LlmInferenceGateway/LlmStoredFileBlockInsertWithBlockIndexUpload 1 block of a file
GET/LlmInferenceGateway/LlmStoredFileBlockListByFileContentHashSha256List the blocks of a file
GET/LlmInferenceGateway/LlmStoredFileDownloadByFileContentHashSha256Download a file
POST/LlmInferenceGateway/LlmStoredFileForceRemoveByFileContentHashSha256Delete a stored file of every token
POST/LlmInferenceGateway/LlmStoredFileInsertWithFileContentHashSha256Start the upload of a file
GET/LlmInferenceGateway/LlmStoredFileListAllList the stored files of every token
GET/LlmInferenceGateway/LlmStoredFileListByAccessTokenList the stored files of the token
POST/LlmInferenceGateway/LlmStoredFileRemoveByFileContentHashSha256Delete a stored file
POST/LlmInferenceGateway/LlmStreamLeaseAcquireWithServerLeaseUidAcquire a stream lease
GET/LlmInferenceGateway/LlmStreamLeaseListAllList the stream leases of every token
GET/LlmInferenceGateway/LlmStreamLeaseListByAccessTokenList stream leases
POST/LlmInferenceGateway/LlmStreamLeaseReleaseByStreamLeaseUidRelease a stream lease
POST/LlmInferenceGateway/LlmStreamLeaseResetByStreamLeaseUidReset the context of a stream lease
GET/LlmInferenceGateway/LlmUsageReportGenerateWithDateFromAndDateToModel-usage report
GET/LlmInferenceGateway/LlmUsageTotalGenerateWithDateFromAndDateToToken usage
GET/LlmInferenceGateway/ServiceHealthReadCurrentGateway health (readiness)
GET/LlmInferenceGateway/ServiceInfoReadCurrentRead the server name and its listen addresses
POST/v1/audio/transcriptionsOpenAI audio transcriptions
POST/v1/chat/completionsOpenAI chat completions
POST/v1/messagesAnthropic messages
GET/v1/modelsList models (OpenAI-compatible)

3. The request fields and the answers of each endpoint

POST /LlmInferenceControlPlane/LlmAccessTokenInsertWithDisplayName

The gateway forwards this path to the control plane. Send the administrator key as the X-Admin-Key header, or an access token as Authorization: Bearer. An administrator endpoint answers a valid access token with 403 and a missing or invalid credential with 401.

Answers:

StatusDescriptionType
201Created
401Unauthorized
403Forbidden
409Conflict
422Unprocessable Content
500Internal Server Error
502Bad Gateway
503Service Unavailable

GET /LlmInferenceControlPlane/LlmAccessTokenListAll

The gateway forwards this path to the control plane. Send the administrator key as the X-Admin-Key header, or an access token as Authorization: Bearer. An administrator endpoint answers a valid access token with 403 and a missing or invalid credential with 401.

Answers:

StatusDescriptionType
200OK
401Unauthorized
403Forbidden
422Unprocessable Content
500Internal Server Error
502Bad Gateway
503Service Unavailable

GET /LlmInferenceControlPlane/LlmAccessTokenReadByTokenHash

The gateway forwards this path to the control plane. Send the administrator key as the X-Admin-Key header, or an access token as Authorization: Bearer. An administrator endpoint answers a valid access token with 403 and a missing or invalid credential with 401.

Answers:

StatusDescriptionType
200OK
401Unauthorized
403Forbidden
404Not Found
422Unprocessable Content
500Internal Server Error
502Bad Gateway
503Service Unavailable

POST /LlmInferenceControlPlane/LlmAccessTokenRemoveByAccessTokenUid

The gateway forwards this path to the control plane. Send the administrator key as the X-Admin-Key header, or an access token as Authorization: Bearer. An administrator endpoint answers a valid access token with 403 and a missing or invalid credential with 401.

Answers:

StatusDescriptionType
200OK
401Unauthorized
403Forbidden
404Not Found
422Unprocessable Content
500Internal Server Error
502Bad Gateway
503Service Unavailable

POST /LlmInferenceControlPlane/LlmAccessTokenRenameByAccessTokenUid

The gateway forwards this path to the control plane. Send the administrator key as the X-Admin-Key header, or an access token as Authorization: Bearer. An administrator endpoint answers a valid access token with 403 and a missing or invalid credential with 401.

Answers:

StatusDescriptionType
200OK
401Unauthorized
403Forbidden
404Not Found
422Unprocessable Content
500Internal Server Error
502Bad Gateway
503Service Unavailable

POST /LlmInferenceControlPlane/LlmAccessTokenRevokeByAccessTokenUid

The gateway forwards this path to the control plane. Send the administrator key as the X-Admin-Key header, or an access token as Authorization: Bearer. An administrator endpoint answers a valid access token with 403 and a missing or invalid credential with 401.

Answers:

StatusDescriptionType
200OK
401Unauthorized
403Forbidden
404Not Found
422Unprocessable Content
500Internal Server Error
502Bad Gateway
503Service Unavailable

POST /LlmInferenceControlPlane/LlmModelConfigurationInsertWithModelConfigurationUid

The gateway forwards this path to the control plane. Send the administrator key as the X-Admin-Key header, or an access token as Authorization: Bearer. An administrator endpoint answers a valid access token with 403 and a missing or invalid credential with 401.

Answers:

StatusDescriptionType
201Created
401Unauthorized
403Forbidden
409Conflict
422Unprocessable Content
500Internal Server Error
502Bad Gateway
503Service Unavailable

POST /LlmInferenceControlPlane/LlmModelConfigurationInsertWithModelName

The gateway forwards this path to the control plane. Send the administrator key as the X-Admin-Key header, or an access token as Authorization: Bearer. An administrator endpoint answers a valid access token with 403 and a missing or invalid credential with 401.

Answers:

StatusDescriptionType
201Created
401Unauthorized
403Forbidden
409Conflict
422Unprocessable Content
500Internal Server Error
502Bad Gateway
503Service Unavailable

GET /LlmInferenceControlPlane/LlmModelConfigurationListAll

The gateway forwards this path to the control plane. Send the administrator key as the X-Admin-Key header, or an access token as Authorization: Bearer. An administrator endpoint answers a valid access token with 403 and a missing or invalid credential with 401.

Answers:

StatusDescriptionType
200OK
401Unauthorized
422Unprocessable Content
500Internal Server Error
502Bad Gateway
503Service Unavailable

GET /LlmInferenceControlPlane/LlmModelConfigurationReadByModelConfigurationUid

The gateway forwards this path to the control plane. Send the administrator key as the X-Admin-Key header, or an access token as Authorization: Bearer. An administrator endpoint answers a valid access token with 403 and a missing or invalid credential with 401.

Answers:

StatusDescriptionType
200OK
401Unauthorized
404Not Found
422Unprocessable Content
500Internal Server Error
502Bad Gateway
503Service Unavailable

GET /LlmInferenceControlPlane/LlmModelConfigurationReadByModelName

The gateway forwards this path to the control plane. Send the administrator key as the X-Admin-Key header, or an access token as Authorization: Bearer. An administrator endpoint answers a valid access token with 403 and a missing or invalid credential with 401.

Answers:

StatusDescriptionType
200OK
401Unauthorized
404Not Found
422Unprocessable Content
500Internal Server Error
502Bad Gateway
503Service Unavailable

POST /LlmInferenceControlPlane/LlmModelConfigurationRemoveByModelConfigurationUid

The gateway forwards this path to the control plane. Send the administrator key as the X-Admin-Key header, or an access token as Authorization: Bearer. An administrator endpoint answers a valid access token with 403 and a missing or invalid credential with 401.

Answers:

StatusDescriptionType
200OK
401Unauthorized
403Forbidden
404Not Found
422Unprocessable Content
500Internal Server Error
502Bad Gateway
503Service Unavailable

POST /LlmInferenceControlPlane/LlmModelConfigurationUpdateByModelConfigurationUid

The gateway forwards this path to the control plane. Send the administrator key as the X-Admin-Key header, or an access token as Authorization: Bearer. An administrator endpoint answers a valid access token with 403 and a missing or invalid credential with 401.

Answers:

StatusDescriptionType
200OK
401Unauthorized
403Forbidden
404Not Found
422Unprocessable Content
500Internal Server Error
502Bad Gateway
503Service Unavailable

GET /LlmInferenceGateway/LlmAccessTokenValidateCurrent

Checks the access token supplied in the Authorization: Bearer header and reports whether it is valid, unrecognized, or expired, plus the key's display name and expiry when valid. Always returns 200 with a verdict body; never returns the secret. A token check the control plane cannot answer gives 503 with the text "The access token could not be checked right now. Retry shortly.", and the call is retried.

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
503Service Unavailableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

POST /LlmInferenceGateway/LlmAudioTranscriptionCrossCheckJobEnqueueWithFileContentHashSha256

Saves 1 job that transcribes 1 stored file of the calling token with 3 Whisper models and finds who speaks when, and answers at once. Body, each field required: FileContentHashSha256 (a stored file of the token with a complete upload), TranscriptionModelNames (exactly 3 different Whisper models of GET /v1/models), Language (a language code of Whisper as en, or "" for the detection by Whisper; the 3 models get the same value), Prompt (words and names that guide the spelling, or ""), Temperature ({"enabled": bool, "value": 0.0 to 1.0}; enabled false reads as 0.0). Answers AudioTranscriptionCrossCheckJobUid and AudioTranscriptionCrossCheckJobState Queued. The gateway runs the job when 1 worker of each model and 1 speaker worker are free, votes each word 2 of 3, and LlmAudioTranscriptionCrossCheckJobReadByAudioTranscriptionCrossCheckJobUid answers the speaker turns. A file the token does not hold answers 404, a file whose upload is not complete answers 409, a file that is not WAV, MP3 or FLAC answers 422, a chat model or a speaker identification model answers 422, a name that is no model answers 404, and a Whisper model that no worker holds or no speaker worker answers 503. Requires a valid access token that may use each model; the X-Admin-Key header alone answers 403.

Request body, application/json: object

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
404Not Foundapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
409Conflictapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
429Too Many Requestsapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
500Internal Server Errorapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
503Service Unavailableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

POST /LlmInferenceGateway/LlmAudioTranscriptionCrossCheckJobForceRemoveByAudioTranscriptionCrossCheckJobUid

Removes 1 audio transcription cross-check job of any token for the administrator. A Queued job never runs. Body: AudioTranscriptionCrossCheckJobUid (required). Answers AccessTokenUid (the owner of the job), AudioTranscriptionCrossCheckJobUid and IsAudioTranscriptionCrossCheckJobDeleted true. No job with the uid answers 404, a Running job answers 409, and a malformed uid answers 422. Requires the X-Admin-Key header; an access token alone answers 403.

Request body, application/json: object

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
404Not Foundapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
409Conflictapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
503Service Unavailableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

GET /LlmInferenceGateway/LlmAudioTranscriptionCrossCheckJobListAll

Lists the audio transcription cross-check jobs of every token for the administrator, newest first, with the fields LlmAudioTranscriptionCrossCheckJobListByAccessToken answers. Query, each field required: AudioTranscriptionCrossCheckJobState (Queued, Running, Completed, Failed or All), DateFrom and DateTo (ISO 8601 UTC, DateFrom inclusive, DateTo exclusive, on CreatedAt), Offset (0 or more) and Limit (1 to 1000). Requires the X-Admin-Key header; an access token alone answers 403.

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
503Service Unavailableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

GET /LlmInferenceGateway/LlmAudioTranscriptionCrossCheckJobListByAccessToken

Lists the audio transcription cross-check jobs of the calling token, newest first. Query, each field required: AudioTranscriptionCrossCheckJobState (Queued, Running, Completed, Failed or All), DateFrom and DateTo (ISO 8601 UTC as 2026-10-02T00:00:00Z, DateFrom inclusive, DateTo exclusive, on CreatedAt), Offset (0 or more) and Limit (1 to 1000). Answers Offset, Limit, TotalCount (the jobs the filters keep) and AudioTranscriptionCrossCheckJobs, each job with the fields LlmAudioTranscriptionCrossCheckJobReadByAudioTranscriptionCrossCheckJobUid answers. Requires a valid access token; the X-Admin-Key header alone answers 403.

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
503Service Unavailableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

GET /LlmInferenceGateway/LlmAudioTranscriptionCrossCheckJobReadByAudioTranscriptionCrossCheckJobUid

Answers 1 job of the calling token. Query: AudioTranscriptionCrossCheckJobUid (required). Answers AudioTranscriptionCrossCheckJobUid, FileContentHashSha256, TranscriptionModelNames, AudioTranscriptionCrossCheckJobState (Queued, Running, Completed or Failed), CreatedAt, StartedAt and CompletedAt (ISO 8601 UTC, empty before the event), DetectedLanguage, AudioDurationSeconds, SpeakerCount, FinalTranscriptText, SpeakerTurns and ErrorMessage (the reason of a Failed job). SpeakerTurns: 1 entry per speaker turn with SpeakerLabel, StartSeconds, EndSeconds, FinalText, ModelTranscriptTexts (ModelName, Text) and Words. Words: 1 entry per voted word with Text, StartSeconds, EndSeconds, AgreeingModelCount (1 to 3; below 3 marks a word the models disagree on) and ModelWordTexts (ModelName, Text; "" when the model dropped the word). No job with the uid answers 404, a job of another token answers 403, and a malformed uid answers 422. Requires a valid access token; the X-Admin-Key header alone answers 403.

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
404Not Foundapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
503Service Unavailableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

POST /LlmInferenceGateway/LlmAudioTranscriptionCrossCheckJobRemoveByAudioTranscriptionCrossCheckJobUid

Removes 1 audio transcription cross-check job of the calling token. A Queued job never runs. Body: AudioTranscriptionCrossCheckJobUid (required). Answers AccessTokenUid, AudioTranscriptionCrossCheckJobUid and IsAudioTranscriptionCrossCheckJobDeleted true. No job with the uid answers 404, a job of another token answers 403, a Running job answers 409, and a malformed uid answers 422. Requires a valid access token; the X-Admin-Key header alone answers 403, and the administrator removes with LlmAudioTranscriptionCrossCheckJobForceRemoveByAudioTranscriptionCrossCheckJobUid.

Request body, application/json: object

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
404Not Foundapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
409Conflictapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
503Service Unavailableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

POST /LlmInferenceGateway/LlmAudioTranscriptionJobEnqueueWithFileContentHashSha256

Saves 1 job that transcribes 1 stored file of the calling token, and answers at once. Body, each field required: FileContentHashSha256 (a stored file of the token with a complete upload), ModelName (a Whisper model of GET /v1/models), Language (a language code of Whisper as en or zh, or "" for the detection by Whisper), Prompt (words and names that guide the spelling, or ""), Temperature ({"enabled": bool, "value": 0.0 to 1.0}; enabled false reads as 0.0). Answers AudioTranscriptionJobUid and AudioTranscriptionJobState Queued. The gateway runs the job when a worker of the model is free, and LlmAudioTranscriptionJobReadByAudioTranscriptionJobUid answers the state and the transcript. A file the token does not hold answers 404, a file whose upload is not complete answers 409, a file that is not WAV, MP3 or FLAC answers 422, a chat model or a speaker identification model answers 422, a name that is no model answers 404, and a Whisper model that no worker holds answers 503. Requires a valid access token that may use the model; the X-Admin-Key header alone answers 403.

Request body, application/json: object

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
404Not Foundapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
409Conflictapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
429Too Many Requestsapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
500Internal Server Errorapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
503Service Unavailableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

POST /LlmInferenceGateway/LlmAudioTranscriptionJobForceRemoveByAudioTranscriptionJobUid

Removes 1 audio transcription job of any token for the administrator. A Queued job never runs. Body: AudioTranscriptionJobUid (required). Answers AccessTokenUid (the owner of the job), AudioTranscriptionJobUid and IsAudioTranscriptionJobDeleted true. No job with the uid answers 404, a Running job answers 409, and a malformed uid answers 422. Requires the X-Admin-Key header; an access token alone answers 403.

Request body, application/json: object

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
404Not Foundapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
409Conflictapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
503Service Unavailableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

GET /LlmInferenceGateway/LlmAudioTranscriptionJobListAll

Lists the audio transcription jobs of every token for the administrator, newest first, with the fields LlmAudioTranscriptionJobListByAccessToken answers. Query, each field required: AudioTranscriptionJobState (Queued, Running, Completed, Failed or All), DateFrom and DateTo (ISO 8601 UTC, DateFrom inclusive, DateTo exclusive, on CreatedAt), Offset (0 or more) and Limit (1 to 1000). Requires the X-Admin-Key header; an access token alone answers 403.

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
503Service Unavailableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

GET /LlmInferenceGateway/LlmAudioTranscriptionJobListByAccessToken

Lists the audio transcription jobs of the calling token, newest first. Query, each field required: AudioTranscriptionJobState (Queued, Running, Completed, Failed or All), DateFrom and DateTo (ISO 8601 UTC as 2026-10-02T00:00:00Z, DateFrom inclusive, DateTo exclusive, on CreatedAt), Offset (0 or more) and Limit (1 to 1000). Answers Offset, Limit, TotalCount (the jobs the filters keep) and AudioTranscriptionJobs, each job with the fields LlmAudioTranscriptionJobReadByAudioTranscriptionJobUid answers. Requires a valid access token; the X-Admin-Key header alone answers 403.

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
503Service Unavailableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

GET /LlmInferenceGateway/LlmAudioTranscriptionJobReadByAudioTranscriptionJobUid

Answers 1 job of the calling token. Query: AudioTranscriptionJobUid (required). Answers AudioTranscriptionJobUid, FileContentHashSha256, ModelName, AudioTranscriptionJobState (Queued, Running, Completed or Failed), CreatedAt, StartedAt and CompletedAt (ISO 8601 UTC, empty before the event), InferenceServerUid (the worker of the last run), DetectedLanguage, AudioDurationSeconds, TranscriptText, TranscriptSegments (StartSeconds, EndSeconds and Text of each timed line) and ErrorMessage (the reason of a Failed job). No job with the uid answers 404, a job of another token answers 403, and a malformed uid answers 422. Requires a valid access token; the X-Admin-Key header alone answers 403.

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
404Not Foundapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
503Service Unavailableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

POST /LlmInferenceGateway/LlmAudioTranscriptionJobRemoveByAudioTranscriptionJobUid

Removes 1 audio transcription job of the calling token. A Queued job never runs. Body: AudioTranscriptionJobUid (required). Answers AccessTokenUid, AudioTranscriptionJobUid and IsAudioTranscriptionJobDeleted true. No job with the uid answers 404, a job of another token answers 403, a Running job answers 409, and a malformed uid answers 422. Requires a valid access token; the X-Admin-Key header alone answers 403, and the administrator removes with LlmAudioTranscriptionJobForceRemoveByAudioTranscriptionJobUid.

Request body, application/json: object

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
404Not Foundapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
409Conflictapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
503Service Unavailableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

POST /LlmInferenceGateway/LlmChatCompletionGenerateWithModelName

Runs a chat completion on the least-loaded server serving the model. Each of the 22 fields is required, and 15 of the 22 are a pair of 'enabled' and 'value': a body that leaves one out answers 422 and names it. ResponseFormat, ParallelToolCalls and StreamOptions reach llama-server as response_format, parallel_tool_calls and stream_options, as on POST /v1/chat/completions; a streamed call always asks llama-server for the usage. The answer names its top-level fields in CamelCase, and each choice keeps the OpenAI shape. Logprobs true adds Choices[0].logprobs.content, 1 entry per generated token, TopLogprobs (0 to 20) above 0 with Logprobs false answers 422, and a Grammar (a GBNF grammar, empty for none) limits the reply to the text it allows. A setting sent with 'enabled' false reaches llama-server as no key, so the server keeps its own value for it, and its 'value' is not range-checked. Requires the same access token POST /v1/chat/completions requires, sent as Authorization: Bearer. A token check the control plane cannot answer gives 503 with the text "The access token could not be checked right now. Retry shortly.", and the call is retried. A 503 that the same call passes a few seconds later, because every worker of the model decodes at its full slot count or no healthy worker serves the model, sends the header Retry-After with the seconds of the gateway flag --retry-after-seconds, 5 by default. A 503 of a server lease, of an unloaded model or of a stream lease server sends no Retry-After header. A prompt over the context of 1 request answers 413, and the message names the tokens of the request and the tokens 1 request of the model holds: llama-server counts the tokens, and GET /v1/models names that context in context_size. A request body over the gateway flag --max-request-bytes, 8388608 by default, answers 413 before it reaches a server, and 0 turns that check off.

Request body, application/json: object

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
404Not Foundapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
413Payload Too Largeapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
429Too Many Requestsapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
503Service Unavailableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
500Internal Server Errorapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

POST /LlmInferenceGateway/LlmChatCompletionGenerateWithStreamLeaseUid

Runs 1 chat call on the server of StreamLeaseUid, on the decode slot of that stream lease. The body is the body of POST /LlmInferenceGateway/LlmChatCompletionGenerateWithModelName with StreamLeaseUid, a required string; stream true streams the reply. Answers 409 at once while a call, a reset or a release runs on the stream lease, 403 when another token holds it, 404 when no stream lease has the uid, 400 when model is not the model of the lease or the call does not fit the context of the slot (the message names POST /LlmInferenceGateway/LlmStreamLeaseResetByStreamLeaseUid), 413 when the request body is over the gateway flag --max-request-bytes, 8388608 by default (0 turns the check off), 422 for a malformed body, and 503 when the server of the lease does not serve. Requires the access token of the server lease, sent as Authorization: Bearer.

Request body, application/json: object

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
400Bad Requestapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
404Not Foundapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
409Conflictapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
413Payload Too Largeapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
503Service Unavailableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
500Internal Server Errorapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

GET /LlmInferenceGateway/LlmInferenceLogDownloadAll

Answers the rows of the inference log as a JSONL file download, oldest day first: Content-Type application/octet-stream, Content-Length of the answer, Content-Disposition attachment with the filename inference-log.jsonl, and Accept-Ranges bytes. Query parameters: DateFrom and DateTo (ISO 8601 UTC as 2026-09-16T00:00:00Z, DateFrom inclusive, DateTo exclusive), both optional; a window answers the rows of the window. The filename names the 2 ends of the window as yymmdd-hhmmss UTC, as inference-log.260923-000000.260923-045612.jsonl: a window with no DateFrom starts at the oldest day with rows, and a window with no DateTo ends at the time of the answer. Send the header "Range: bytes=<first>-<last>" to read one part of the answer: that answers 206 with the Content-Range header. A first byte past the end of the answer answers 416. An empty log answers 200 with 0 bytes. A wrong value or an unknown field answers 422 naming the field. Requires the X-Admin-Key header when the gateway is started with a non-empty administrator key (ADMIN_API_KEY).

Answers:

StatusDescriptionType
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
416Range Not Satisfiableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
500Internal Server Errorapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
200OKapplication/octet-stream string (binary)
206OKapplication/octet-stream string (binary)

GET /LlmInferenceGateway/LlmInferenceLogLineListAll

Returns the inference-log records of the window as their JSON text, oldest first, from every day file of the log. Query parameters: Offset (0 or more, default 0), Limit (1 to 1000, default 100), DateFrom and DateTo (ISO 8601 UTC as 2026-09-16T00:00:00Z, DateFrom inclusive, DateTo exclusive). The answer names lines, Offset, Limit, count and TotalCount, the rows of the window. A wrong value or an unknown field answers 422 naming the field. Requires the X-Admin-Key header when the gateway is started with a non-empty administrator key (ADMIN_API_KEY).

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

GET /LlmInferenceGateway/LlmInferenceLogListAll

Inference records (model, worker, tokens, latency, status) of the whole log, newest first. Query parameters: Offset (0 or more, default 0), Limit (1 to 1000, default 100), DateFrom and DateTo (ISO 8601 UTC as 2026-09-16T00:00:00Z, DateFrom inclusive, DateTo exclusive). The answer names entries, Offset, Limit, count and TotalCount, the records of the window. A wrong value or an unknown field answers 422 naming the field.

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

POST /LlmInferenceGateway/LlmInferenceLogRemoveAll

Clears the gateway inference log: every day file, every day summary and the usage totals, so a restart does not bring cleared entries back. Answers ClearedCount, the rows the day files held. Destructive. Requires the X-Admin-Key header when the gateway is started with a non-empty administrator key (ADMIN_API_KEY).

Request body, application/json: object

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
500Internal Server Errorapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

POST /LlmInferenceGateway/LlmInferenceLogRemoveByDateTo

Removes inference-log entries. By caller: KeepAccessTokenUids keeps only those callers, or RemoveAccessTokenUids plus RemoveEmptyToken drop selected callers and no-token entries; the day files are rewritten and the usage totals drop the removed entries. By date: DateTo, a UTC midnight such as 2026-09-13T00:00:00Z at or before the start of the current UTC day, removes the day files before it and keeps their day summaries, so GET /LlmInferenceGateway/LlmUsageTotalGenerateWithDateFromAndDateTo, the weekly report and the access-token quotas keep their figures. With a caller rule, DateTo limits that rule to the days before it. Answers RemovedCount. Each entry of KeepAccessTokenUids and RemoveAccessTokenUids must be a uid of the form xxxx-xxxx-xxxx-xxxx; an upper-case uid matches the same caller, and a malformed entry answers 422 naming the field and the sent text and removes nothing. A wrong value or an unknown field answers 422 naming the field. Destructive. Requires the X-Admin-Key header when the gateway is started with a non-empty administrator key.

Request body, application/json: object

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
500Internal Server Errorapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

POST /LlmInferenceGateway/LlmInferenceServerInsertWithInferenceServerUid

Registers an inference server (one worker-host process on one node) directly with the gateway. The gateway adds the row to its in-memory registry and the OpenAiWorkerRouter then considers the server a routing candidate.

Request body, application/json: object

Answers:

StatusDescriptionType
201Createdapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
500Internal Server Errorapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

GET /LlmInferenceGateway/LlmInferenceServerListAll

Returns one row per registered inference server with the registry fields plus the live in-flight request count and the staleness verdict the routing pass uses. Requires a valid access token, or the X-Admin-Key header. A token check the control plane cannot answer gives 503 with the text "The access token could not be checked right now. Retry shortly.", and the call is retried.

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
503Service Unavailableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

POST /LlmInferenceGateway/LlmInferenceServerRemoveByInferenceServerUid

Removes the row of InferenceServerUid from the registry and from its snapshot file, so a gateway restart does not restore it. A row whose last heartbeat is not older than the stale age answers 409 NotStale, and a row under a server lease answers 409 HeldByServerLease. Requires the X-Admin-Key header; an access token alone answers 403.

Request body, application/json: object

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
404Not Foundapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
409Conflictapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
503Service Unavailableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

POST /LlmInferenceGateway/LlmInferenceServerUpdateByInferenceServerUid

Refreshes the LastHeartbeat field on the gateway's in-memory registry row for the given InferenceServerUid. Returns ok=true on match, ok=false when the uid is unknown (the caller may then re-register).

Request body, application/json: object

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

GET /LlmInferenceGateway/LlmLoadableModelListAll

Lists every loadable model: a servable model (real gguf in the MinIO bucket) that no non-stale worker is serving now, with its MinIO file count. Complements GET /v1/models, which lists loaded models only. An operator loads one with POST /LlmInferenceGateway/LlmModelLoadByInferenceServerUid, after which it moves onto GET /v1/models. Requires the X-Admin-Key header when the gateway is started with a non-empty administrator key.

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

POST /LlmInferenceGateway/LlmModelLoadByInferenceServerUid

Records an explicit command for the inference server InferenceServerUid to load ModelName; the worker applies it on its next heartbeat. Manual model placement; the gateway never auto-loads. Requires a valid access token, or the X-Admin-Key header. Answers 409 while a server lease holds the server, force included. force=true additionally requires the X-Admin-Key header. A token check the control plane cannot answer gives 503 with the text "The access token could not be checked right now. Retry shortly.", and the call is retried.

Request body, application/json: object

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
404Not Foundapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
409Conflictapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
429Too Many Requestsapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
503Service Unavailableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

GET /LlmInferenceGateway/LlmModelStatusListAll

Lists every servable model with its status (loaded when a worker serves it now, loadable otherwise), its MinIO file count, and the workers serving it (InferenceServerUid, WorkerName, HostAddress, InFlight, GpuType, stale). A non-OpenAI endpoint for operators and scripts. Requires a valid access token when the gateway runs with access-token auth. A token check the control plane cannot answer gives 503 with the text "The access token could not be checked right now. Retry shortly.", and the call is retried.

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
503Service Unavailableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

POST /LlmInferenceGateway/LlmModelUnloadByInferenceServerUid

Records an explicit command for the inference server InferenceServerUid to unload its current model and free the GPU; the worker applies it on its next heartbeat. Manual model placement. Requires a valid access token, or the X-Admin-Key header. Answers 409 while a server lease holds the server, force included. force=true additionally requires the X-Admin-Key header. A token check the control plane cannot answer gives 503 with the text "The access token could not be checked right now. Retry shortly.", and the call is retried.

Request body, application/json: object

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
404Not Foundapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
409Conflictapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
429Too Many Requestsapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
503Service Unavailableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

POST /LlmInferenceGateway/LlmServerLeaseAcquireWithInferenceServerUid

Claims InferenceServerUid for the calling token. Only calls on its stream leases, sent to POST /LlmInferenceGateway/LlmChatCompletionGenerateWithStreamLeaseUid, reach a leased server. StreamContextSize is the context of each stream and KvCacheQuantization the KV cache type. The same token acquiring its own server again re-tunes the lease and releases its stream leases. Answers 409 when another token holds the server. Requires a valid access token; the X-Admin-Key header alone answers 403. A token check the control plane cannot answer gives 503 with the text "The access token could not be checked right now. Retry shortly.", and the call is retried.

Request body, application/json: object

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
404Not Foundapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
409Conflictapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
503Service Unavailableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

POST /LlmInferenceGateway/LlmServerLeaseForceReleaseByServerLeaseUid

Releases ServerLeaseUid and every stream lease under it for any token. The holder's next call on the lease answers 404. Requires the X-Admin-Key header; an access token alone answers 403.

Request body, application/json: object

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
404Not Foundapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
503Service Unavailableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

GET /LlmInferenceGateway/LlmServerLeaseListAll

Lists the server leases of every token, 1 row each with the fields LlmServerLeaseListByAccessToken answers, AccessTokenUid included. Query, each field required: Offset (0 or more) and Limit (1 to 1000), and answers TotalCount. Requires the X-Admin-Key header; an access token alone answers 403.

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
503Service Unavailableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

GET /LlmInferenceGateway/LlmServerLeaseListByAccessToken

Lists the server leases of the calling token only, 1 row each with 20 fields: ServerLease (ServerLeaseUid, InferenceServerUid, AccessTokenUid, ModelName, StreamContextSize in tokens, KvCacheQuantization, StreamCapacity in slots, KvCacheBytesPerToken, LaunchGeneration, CreatedAt), ServerName, ActiveModel, LoadState, LoadPhase, LoadError, InFlight (calls running now), LastHeartbeatAgeSeconds, Stale, StreamLeaseCount, Gpus (per card: GpuCudaId, GpuName, GpuFanPercentage, VramUsed, VramMax, PowerCurrent, PowerMax), Ram (RamUtilization, RamUsage, RamMax), VramUsedMib, VramMaxMib and VramFreeMib (all cards added up, MiB), MemoryPressure (VramUsedMib over VramMaxMib, 0 to 1), KvCacheBudgetBytes (StreamCapacity x StreamContextSize x KvCacheBytesPerToken), KvCacheUsedBytes (the sum of MemoryBytesUsed of the stream leases), KvCacheFreeBytes, and KvCachePressure (used over budget, 0 to 1). Requires a valid access token; the X-Admin-Key header alone answers 403. A token check the control plane cannot answer gives 503 with the text "The access token could not be checked right now. Retry shortly.", and the call is retried.

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
503Service Unavailableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

POST /LlmInferenceGateway/LlmServerLeaseReleaseByServerLeaseUid

Releases ServerLeaseUid and every stream lease under it. Answers 403 when another token holds the lease. Requires a valid access token; the X-Admin-Key header alone answers 403. A token check the control plane cannot answer gives 503 with the text "The access token could not be checked right now. Retry shortly.", and the call is retried.

Request body, application/json: object

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
503Service Unavailableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
404Not Foundapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

GET /LlmInferenceGateway/LlmStoredFileBlockDownloadByBlockIndex

Answers the bytes of 1 block that arrived, of 1 file of the calling token: Content-Type application/octet-stream and Content-Length. Query, each field required: FileContentHashSha256 and BlockIndex (0 to TotalBlockCount minus 1). A block that did not arrive, and a file the token does not hold, answer 404. Requires the access token that started the file; the X-Admin-Key header alone answers 403.

Answers:

StatusDescriptionType
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
404Not Foundapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
500Internal Server Errorapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
503Service Unavailableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
200OKapplication/octet-stream string (binary)

POST /LlmInferenceGateway/LlmStoredFileBlockInsertWithBlockIndex

Sends the data of 1 block of a file LlmStoredFileInsertWithFileContentHashSha256 described. The body is multipart/form-data, each field required: FileContentHashSha256, BlockIndex (0 to TotalBlockCount minus 1) and BlockData (the bytes of the block: BlockSizeInBytes, or the rest of the file for the last block). The gateway checks the size and the SHA-256 of the block against the description, and a block that differs answers 422 and stays missing. A block that arrived before answers 200 and changes nothing. After the last block the gateway joins the blocks and checks the SHA-256 of the whole file: FileUploadState reads Complete, or FileHashMismatch when the joined file differs from FileContentHashSha256. A file in FileHashMismatch answers 409. A file the token does not hold answers 404. Requires the access token that started the file; the X-Admin-Key header alone answers 403.

Request body, multipart/form-data: object

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
404Not Foundapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
409Conflictapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
413Payload Too Largeapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
500Internal Server Errorapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
503Service Unavailableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

GET /LlmInferenceGateway/LlmStoredFileBlockListByFileContentHashSha256

Lists the blocks of 1 file of the calling token. Query, each field required: FileContentHashSha256, BlockReceiveState (Received, Missing or All), Offset (0 or more) and Limit (1 to 1000). Answers FileContentHashSha256, FileUploadState, Offset, Limit, TotalCount (the blocks the filter keeps) and FileBlocks: BlockIndex, BlockSizeInBytes, BlockContentHashSha256 and IsBlockReceived. A file the token does not hold answers 404. Requires the access token that started the file; the X-Admin-Key header alone answers 403.

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
404Not Foundapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
503Service Unavailableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

GET /LlmInferenceGateway/LlmStoredFileDownloadByFileContentHashSha256

Answers the bytes of 1 complete file of the calling token: Content-Type application/octet-stream, Content-Length, Content-Disposition attachment with OriginalFileName, and Accept-Ranges bytes. Query: FileContentHashSha256 (required). Send the header "Range: bytes=<first>-<last>" to read one part: that answers 206 with the Content-Range header, and a first byte past the end answers 416. A file that is not Complete answers 409. A file the token does not hold answers 404. Requires the access token that started the file; the X-Admin-Key header alone answers 403.

Answers:

StatusDescriptionType
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
404Not Foundapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
409Conflictapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
416Range Not Satisfiableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
503Service Unavailableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
200OKapplication/octet-stream string (binary)
206OKapplication/octet-stream string (binary)

POST /LlmInferenceGateway/LlmStoredFileForceRemoveByFileContentHashSha256

Removes the file of FileContentHashSha256 of every access token that holds it, for the administrator, with its blocks and its data. Body: FileContentHashSha256 (required). Answers FileContentHashSha256 and IsFileDeleted true. A hash that no access token holds answers 404. Requires the X-Admin-Key header; an access token alone answers 403.

Request body, application/json: object

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
404Not Foundapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
503Service Unavailableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

POST /LlmInferenceGateway/LlmStoredFileInsertWithFileContentHashSha256

Describes 1 file before its data, and sends no data. Body, each field required: FileContentHashSha256 (the SHA-256 of the whole file, 64 lowercase hexadecimal characters), FileSizeInBytes (1 to the gateway flag --max-file-bytes, 314572800 by default), OriginalFileName (1 to 255 characters), BlockSizeInBytes (524288, or 8388608; a request of 8388608 bytes does not pass the reverse proxy of the public address), BlockContentHashSha256List (the SHA-256 of each block in order, FileSizeInBytes divided by BlockSizeInBytes rounded up). Answers 201 for a new file and 200 for a file the token already holds: the description, TotalBlockCount, ReceivedBlockCount, MissingBlockIndexList (the blocks to send with LlmStoredFileBlockInsertWithBlockIndex, from 0) and FileUploadState (WaitingForBlocks, Complete or FileHashMismatch). Send the call again with the same body to read the missing blocks after a lost connection. A known hash with another description answers 409. A file belongs to the access token that starts it; the X-Admin-Key header alone answers 403.

Request body, application/json: object

Answers:

StatusDescriptionType
201Createdapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
409Conflictapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
503Service Unavailableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

GET /LlmInferenceGateway/LlmStoredFileListAll

Lists the stored files of every token for the administrator, newest first, with the fields LlmStoredFileListByAccessToken answers. Query, each field required: FileUploadState (WaitingForBlocks, Complete, FileHashMismatch or All), DateFrom and DateTo (ISO 8601 UTC, DateFrom inclusive, DateTo exclusive, on CreatedAtIso), Offset (0 or more) and Limit (1 to 1000). Requires the X-Admin-Key header; an access token alone answers 403.

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
503Service Unavailableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

GET /LlmInferenceGateway/LlmStoredFileListByAccessToken

Lists the stored files of the calling token, newest first. Query, each field required: FileUploadState (WaitingForBlocks, Complete, FileHashMismatch or All), DateFrom and DateTo (ISO 8601 UTC as 2026-09-16T00:00:00Z, DateFrom inclusive, DateTo exclusive, on CreatedAtIso), Offset (0 or more) and Limit (1 to 1000). Answers Offset, Limit, TotalCount (the files the filters keep) and Files: AccessTokenUid, FileContentHashSha256, OriginalFileName, FileSizeInBytes, BlockSizeInBytes, TotalBlockCount, ReceivedBlockCount, FileUploadState, CreatedAtIso and CompletedAtIso. Requires a valid access token; the X-Admin-Key header alone answers 403.

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
503Service Unavailableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

POST /LlmInferenceGateway/LlmStoredFileRemoveByFileContentHashSha256

Removes 1 file of the calling token with its blocks and its data. Body: FileContentHashSha256 (required). Answers AccessTokenUid, FileContentHashSha256 and IsFileDeleted true. A file the token does not hold answers 404. Requires a valid access token; the X-Admin-Key header alone answers 403, and the administrator deletes with LlmStoredFileForceRemoveByFileContentHashSha256.

Request body, application/json: object

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
404Not Foundapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
503Service Unavailableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

POST /LlmInferenceGateway/LlmStreamLeaseAcquireWithServerLeaseUid

Takes 1 decode slot of the server of ServerLeaseUid. Acquire 1 stream lease per concurrent stream. Answers 409 when every slot of the server is held, and 403 when another token holds the server lease. Requires a valid access token; the X-Admin-Key header alone answers 403. A token check the control plane cannot answer gives 503 with the text "The access token could not be checked right now. Retry shortly.", and the call is retried.

Request body, application/json: object

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
404Not Foundapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
409Conflictapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
503Service Unavailableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

GET /LlmInferenceGateway/LlmStreamLeaseListAll

Lists the stream leases of every token, 1 row each with the 8 fields LlmStreamLeaseListByAccessToken answers: StreamLeaseUid, ServerLeaseUid, SlotId, state, ContextTokensUsed, MemoryBytesUsed, ContextPressure and CreatedAt. Query, each field required: Offset (0 or more) and Limit (1 to 1000), and answers TotalCount. Requires the X-Admin-Key header; an access token alone answers 403.

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
503Service Unavailableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

GET /LlmInferenceGateway/LlmStreamLeaseListByAccessToken

Lists the stream leases of the calling token only, 1 row each with 8 fields: StreamLeaseUid, ServerLeaseUid, SlotId (the llama-server slot, sent as id_slot), State (Idle, Prefill, Decode or Turn, read from GET /slots of the server at list time), ContextTokensUsed (tokens the slot holds), MemoryBytesUsed (ContextTokensUsed x KvCacheBytesPerToken), ContextPressure (ContextTokensUsed over StreamContextSize, 0 to 1) and CreatedAt. Requires a valid access token; the X-Admin-Key header alone answers 403. A token check the control plane cannot answer gives 503 with the text "The access token could not be checked right now. Retry shortly.", and the call is retried.

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
503Service Unavailableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

POST /LlmInferenceGateway/LlmStreamLeaseReleaseByStreamLeaseUid

Returns the decode slot of StreamLeaseUid to its server lease. Erases the slot first, so the next stream lease on it starts with no context. Answers 403 when another token holds the stream, 409 while a call runs on it or llama-server does not answer the erase in time, 404 when its server lease was released meanwhile, and 502 when llama-server refuses the erase. Requires a valid access token; the X-Admin-Key header alone answers 403. A token check the control plane cannot answer gives 503 with the text "The access token could not be checked right now. Retry shortly.", and the call is retried.

Request body, application/json: object

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
503Service Unavailableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
404Not Foundapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
409Conflictapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
502Bad Gatewayapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

POST /LlmInferenceGateway/LlmStreamLeaseResetByStreamLeaseUid

Wipes the KV cache and the context of the slot of StreamLeaseUid and keeps the stream lease, so the next call on it starts a fresh conversation. Answers 409 while a call runs on the stream or llama-server does not answer in time, 403 when another token holds it, 404 when its server lease was released meanwhile, and 502 when llama-server refuses the erase. Requires a valid access token; the X-Admin-Key header alone answers 403. A token check the control plane cannot answer gives 503 with the text "The access token could not be checked right now. Retry shortly.", and the call is retried.

Request body, application/json: object

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
503Service Unavailableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
404Not Foundapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
409Conflictapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
502Bad Gatewayapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

GET /LlmInferenceGateway/LlmUsageReportGenerateWithDateFromAndDateTo

Per-model request counts, token totals, idle percentage, callers by token name, and never-used models over 1 window, rendered as plain text. The window reads DateFrom and DateTo, ISO 8601 UTC as 2026-09-16T00:00:00Z, from inclusive to exclusive. A call with no field reads the 7 days that end now. A value the field cannot take answers 422 with the field, the value and the values the field takes.

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

GET /LlmInferenceGateway/LlmUsageTotalGenerateWithDateFromAndDateTo

Per-model, per-access-token and overall token totals, request counts and errors recorded by the gateway: overall, PerModel keyed by the model in model, and PerToken keyed by the AccessTokenUid in model. With no window the totals count every request of the log. DateFrom and DateTo (ISO 8601 UTC as 2026-09-16T00:00:00Z, DateFrom inclusive, DateTo exclusive) count the requests of the window. The totals of a day whose rows POST /LlmInferenceGateway/LlmInferenceLogRemoveByDateTo removed stay in both. A wrong value or an unknown field answers 422 naming the field.

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

GET /LlmInferenceGateway/ServiceHealthReadCurrent

Returns 200 when at least one registered inference server is not stale, otherwise 503. No authentication; intended for load balancers and Kubernetes liveness/readiness probes.

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
503Service Unavailableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

GET /LlmInferenceGateway/ServiceInfoReadCurrent

Answers the name, the description, and the listen addresses of the gateway. Requires a valid access token when the gateway runs with access-token auth; the administrator key alone answers 403. A token check the control plane cannot answer gives 503.

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
503Service Unavailableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

POST /v1/audio/transcriptions

OpenAI-compatible speech-to-text. The body is multipart/form-data: file (required, 1 WAV, MP3 or FLAC file), model (required, a Whisper model of GET /v1/models), language, prompt, response_format (json, verbose_json, text, srt or vtt) and temperature (0.0 to 1.0). Requires the access token POST /v1/chat/completions requires, sent as Authorization: Bearer. A field the route does not declare answers 422 and names the field. A body over the gateway flag --max-audio-bytes, 26214400 by default, answers 413. A chat model or a speaker identification model answers 422, a name that is no model answers 404, and a Whisper model that no worker holds answers 503. Every worker of the model transcribes 1 file at a time, and a call that finds every worker busy answers 503 at once with the header Retry-After.

Request body, multipart/form-data: object

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
404Not Foundapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
413Payload Too Largeapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
429Too Many Requestsapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
500Internal Server Errorapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
503Service Unavailableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

POST /v1/chat/completions

OpenAI-compatible chat completion. Routes by the model field to the least-loaded worker holding it (or loads locally as a fallback), and forwards to its llama-server. A token check the control plane cannot answer gives 503 with the text "The access token could not be checked right now. Retry shortly.", and the call is retried. A 503 that the same call passes a few seconds later, because every worker of the model decodes at its full slot count or no healthy worker serves the model, sends the header Retry-After with the seconds of the gateway flag --retry-after-seconds, 5 by default. A 503 of a server lease, of an unloaded model or of a stream lease server sends no Retry-After header. A prompt over the context of 1 request answers 413, and the message names the tokens of the request and the tokens 1 request of the model holds: llama-server counts the tokens, and GET /v1/models names that context in context_size. A request body over the gateway flag --max-request-bytes, 8388608 by default, answers 413 before it reaches a server, and 0 turns that check off.

Request body, application/json: object

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
404Not Foundapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
503Service Unavailableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
413Payload Too Largeapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
429Too Many Requestsapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
500Internal Server Errorapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

POST /v1/messages

Anthropic Messages-format chat. Routes by the model field to the least-loaded worker holding it (or loads locally as a fallback) and translates the request and response between the Anthropic Messages contract and the worker's OpenAI chat contract. A token check the control plane cannot answer gives 503 with the text "The access token could not be checked right now. Retry shortly.", and the call is retried. A 503 that the same call passes a few seconds later, because every worker of the model decodes at its full slot count or no healthy worker serves the model, sends the header Retry-After with the seconds of the gateway flag --retry-after-seconds, 5 by default. A 503 of a server lease, of an unloaded model or of a stream lease server sends no Retry-After header. A prompt over the context of 1 request answers 413, and the message names the tokens of the request and the tokens 1 request of the model holds: llama-server counts the tokens, and GET /v1/models names that context in context_size. A request body over the gateway flag --max-request-bytes, 8388608 by default, answers 413 before it reaches a server, and 0 turns that check off.

Request body, application/json: object

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
404Not Foundapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
413Payload Too Largeapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
429Too Many Requestsapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
503Service Unavailableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
500Internal Server Errorapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

GET /v1/models

Lists LOADED models by default -- those a worker is serving now, usable immediately in the OpenAI list shape, each with a non-standard status field (loaded or loadable). A loaded model carries the non-standard field context_size: the tokens 1 request of the model holds, the smallest context_size divided by the decode slots among the live servers of the model. A server gives each decode slot an equal part of its context_size, rounded up to a multiple of 256 tokens, so context_size equals the tokens that the 413 of POST /v1/chat/completions names. A loadable model, or a loaded model with no live server that reports a context size, omits context_size. A normal caller cannot load a model, so loadable-but-unloaded models are not listed by default; use ?available=loadable (or GET /LlmInferenceGateway/LlmLoadableModelListAll) for those, or ?available=all for loaded + loadable. MinIO folder artifacts and stub entries are never listed. Requires a valid access token when the gateway runs with access-token auth; the token's allowed-model list and rate limits are not charged by a listing. A token check the control plane cannot answer gives 503 with the text "The access token could not be checked right now. Retry shortly.", and the call is retried.

Answers:

StatusDescriptionType
200OKapplication/json t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest
403Forbiddenapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
401Unauthorizedapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
503Service Unavailableapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest
422Unprocessable Entityapplication/json t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

4. The types of the request bodies and of the answers

t1.kcgXlib.httpModManagerRequestLog.ErrorHttpRequest

object

t1.kcgXlib.httpModManagerRequestLog.SuccessfulHttpRequest

object