auditor

Automate LLM vulnerability scanning by configuring audit jobs that systematically test inference endpoints against adversarial probes. This skill manages audit configurations, registers targets like NIM or OpenAI models, and executes jobs to evaluate model robustness for production AI safety.

Testing LLM robustness against jailbreaks involves using probe specifications like AutoDANCached within an audit configuration. Security engineers can run these automated vulnerability scanning jobs locally or on a cluster to identify jailbreak vulnerabilities before moving models to production.

Yes, you can audit OpenAI and NIM models using the same workflow. The target deployment feature allows you to register and manage various inference endpoints, enabling systematic vulnerability scanning across different model architectures using standardized audit configurations.

To run automated security audits on a cluster, use the job execution feature to submit audit jobs remotely. This facilitates production-grade AI safety testing by allowing you to execute vulnerability scanning workflows across distributed computing environments rather than just local execution.

Adversarial probes in LLM security auditing are standardized test specifications, such as AutoDANCached, used to evaluate model robustness. They systematically query inference endpoints to identify potential vulnerabilities and ensure compliance with security benchmarks before production deployment.

Yes, the auditor skill supports compliance testing by providing a standardized interface for managing audit configurations and jobs. It systematically tests AI agents against security benchmarks to identify vulnerabilities, ensuring production deployments meet required safety and compliance standards.