Operate
Check model execution and worker recovery
Separate model memory state, processor availability and recovery after a failed worker.
Check the selected server
Open AI → Models. Check the server, supplied runtime, model deployment and memory state. Read live processor activity and the original request failure. In memory means the model is loaded; it does not prove that a generation or embedding request completed successfully.
Understand worker recovery
The 1.0.0 source retires failed native workers and delays reuse of the affected model and backend. The first cooldown is 30 seconds. Repeated failures increase it, up to five minutes. A fresh worker is admitted cautiously after cooldown; successful work gradually restores concurrency within the configured limits.
Recovery is different from missing compatible hardware. It can contain a failed request, but it cannot repair a driver or guarantee that an incompatible model will succeed. CPU allocation failures can also require worker recovery.
Check processing policy
Generation and embedding use separate policies. A GPU-only request must not silently change to CPU. Where policy permits, available compatible alternatives or CPU may be used. Confirm the actual backend for a successful request and check the package inventory before expecting a GPU worker.
Investigate and retest
- Keep the original model, runtime, node, time and request or job references.
- Check memory, model-file access and service readiness. Low memory can prevent a new worker from starting.
- Read the document’s retry or failure state. Eligible document retries retain the configured retry limit; paused or cancelled work is not automatically resumed.
- After the underlying problem and cooldown are resolved, test one short generation request or one small document embedding on the intended server.
- Inspect the final result and observed processor before increasing work.
A chat or other uncertain business request must not be submitted repeatedly just because a worker can restart. Check its saved run first.
If failures continue
Use the diagnostic guide and the authenticated Support case. Preserve the package identity and first failure. Do not erase protected installation state, repeatedly restart the service or apply an older release’s repair bundle.
Runtime in the 1.0.0 packages
The published Linux packages supply CPU inference and embedding. They contain no GPU worker runtime. GPU policy controls describe what the source can do with a suitable runtime; they do not add one to the installed package. The published Windows MSI includes a CPU inference runtime. Confirm any GPU runtime against the exact installed package. Check the package runtime scope before choosing a backend.
Was this page helpful?
Your answer helps us improve the documentation.
Do not include personal information, customer information, passwords, or keys.