tao-generate-referring-expressions

You can generate referring expressions by running the auto_label generate command with a spec YAML file, specifying the image directory, KITTI label directory, and a VLM API key to automate the annotation pipeline.

Referring expressions in autonomous driving scene understanding are natural language phrases that uniquely identify specific objects or regions within an image, generated from visual data and KITTI bounding box coordinates for training datasets.

Yes, a VLM endpoint is required to generate scene captions from KITTI labels. The pipeline relies on a vision language model like Gemini or an OpenAI-compatible server to analyze images and produce text descriptions.

Yes, you can use OpenAI-compatible servers instead of Gemini for image annotation. The pipeline supports OpenAI-compatible VLM endpoints by configuring the base_url, model_name, and api_key parameters in the specification file.

The best way to automate region description generation for autonomous driving datasets is using a pipeline that integrates KITTI bounding-box labels with a VLM endpoint to process images and output structured scene captions and referring expressions.