ml-framework-implementation

Set seeds for random, numpy, and torch, enable cudnn deterministic mode, and pin the device. Use AdamW, pass raw logits to CrossEntropyLoss, clip gradients, and separate train and eval phases with model.train() and torch.no_grad().

Load a tokenizer and model with AutoTokenizer and AutoModelForSequenceClassification, tokenize with truncation and padding, then train with the Trainer API and TrainingArguments. Pin a model revision in production and pass a compute_metrics function for evaluation.

PyTorch uses nn.Module with manual backward passes, JAX/Flax uses functional modules with jax.value_and_grad and optax optimizers, and scikit-learn fits estimators via Pipeline with .fit(X, y). Choose based on task: deep learning favors PyTorch or JAX, tabular baselines favor scikit-learn.

NaN losses usually come from a learning rate that is too high, log of zero, poor initialization, or missing gradient clipping. Lower the learning rate, add eps or clamping, clip gradient norms, and inspect the input data for invalid values.

CrossEntropyLoss expects raw logits of shape (B, C) and target class indices of shape (B,) with long dtype. Passing probabilities, one-hot targets, or mismatched batch dimensions causes the mismatch.

Reduce the batch size, use gradient accumulation to simulate larger batches, enable automatic mixed precision with torch.amp, or apply gradient checkpointing. These trade compute or memory to fit the model on available GPU memory.