stable-diffusion-image-generation

Load a DiffusionPipeline from a pretrained model like stable-diffusion-v1-5 with torch.float16, move it to CUDA, then call it with a prompt, num_inference_steps, and guidance_scale. The returned images list contains PIL images you can save directly.

SD 1.5 generates 512x512 images with lower memory needs, SDXL produces higher-quality 1024x1024 images, and Flux is a newer transformer-based model. Diffusers provides dedicated pipeline classes for each, all sharing the same calling pattern.

Enable model CPU offloading with pipe.enable_model_cpu_offload(), add attention slicing, and use float16 precision. For large images, also enable VAE slicing and tiling, or reduce batch size and image dimensions.

Yes, load a ControlNetModel such as the openpose or canny variant and pass it to StableDiffusionControlNetPipeline. Provide a preprocessed control image alongside your prompt to condition the generation on structure, pose, or depth.

Call pipe.load_lora_weights() with the LoRA path and weight file name, then optionally fuse with pipe.fuse_lora(lora_scale=0.8). Multiple LoRAs can be combined using adapter names and pipe.set_adapters() with per-adapter weights.

Black images often come from the safety checker or VAE dtype mismatches; try disabling the safety checker and keeping the VAE in float32. Noisy outputs usually mean too few inference steps or a misconfigured scheduler.