tensorboard

Create a SummaryWriter with a log directory, then call writer.add_scalar with a tag, value, and step inside your training loop. Launch the dashboard with tensorboard --logdir=runs and open http://localhost:6006.

Write each experiment to its own subdirectory under a common logdir, such as runs/exp1 and runs/exp2. Launch TensorBoard on the parent directory and all runs appear together, where you can toggle, filter, and overlay their metric charts.

Yes, TensorFlow includes TensorBoard via tf.keras.callbacks.TensorBoard, which logs metrics, histograms, graphs, and profiling data during model.fit. You can also use tf.summary directly in custom training loops.

Yes, set report_to='tensorboard' and logging_dir in TrainingArguments, and the Trainer automatically logs training and evaluation metrics. Custom compute_metrics functions also flow into the TensorBoard dashboard.

Logging every batch instead of every epoch creates excessive event files. Log batch metrics only every 100 steps, log histograms every few epochs, and always close the writer when training finishes.

Use torch.profiler with ProfilerActivity.CUDA and a tensorboard_trace_handler, or set profile_batch in the Keras TensorBoard callback. The Profile tab then shows GPU utilization, kernel stats, memory usage, and bottlenecks.