stable-diffusion-image-generation

Load a DiffusionPipeline from a pretrained model like stable-diffusion-v1-5 with torch.float16, move it to CUDA, then call it with a prompt, num_inference_steps, and guidance_scale. The returned images list contains PIL images you can save directly.

SD 1.5 generates 512x512 images with lower memory requirements, SDXL produces higher quality 1024x1024 outputs, and Flux is a newer transformer-based model. Each uses a dedicated pipeline class in Diffusers such as StableDiffusionXLPipeline or FluxPipeline.

Enable model CPU offloading with pipe.enable_model_cpu_offload(), add attention slicing, and use VAE slicing for large images. You can also load models in float16 precision or reduce image dimensions and batch size.

Yes, Diffusers supports loading LoRA weights with pipe.load_lora_weights() and adjusting strength via fuse_lora with a lora_scale parameter. Multiple LoRAs can be combined using set_adapters with individual adapter weights.

Black images often result from VAE dtype mismatches or safety checker interference, while noise indicates too few inference steps or scheduler misconfiguration. Ensure consistent float16 dtypes, increase num_inference_steps, and verify the scheduler config.

Load a ControlNetModel such as control_v11p_sd15_canny, pass it to StableDiffusionControlNetPipeline, then provide a preprocessed control image like a Canny edge map alongside your text prompt. Adjust controlnet_conditioning_scale between 0.5 and 1.5 for conditioning strength.