debug-inference

Diagnosing slow latency in inference service deployments involves analyzing pod logs, GPU health, and observability insights using tools like query_tempo_tool and execute_promql to trace and resolve performance bottlenecks.

The best way to diagnose GPU scheduling issues for model deployments is to analyze GPU health and system events specifically within your OpenShift AI environment to identify resource constraints or scheduling failures.

Yes, this troubleshooting process requires access to OpenShift AI cluster resources, utilizing multiple tools like pods_list, events_list, and get_gpu_info to perform in-depth diagnosis of your InferenceService deployments.

To analyze vLLM errors in your inference service, use the analyze_vllm and chat_vllm tools to inspect model endpoints and deployment info, ensuring effective troubleshooting of runtime issues.

Your inference service may be failing to start due to deployment delays, GPU scheduling issues, or configuration errors, which can be identified by analyzing system status, events, and pod logs.