model-inversion-membership-inference

To audit for membership inference risks, you assess API-enabled models to detect if an attacker can determine whether specific data was used in training. This involves evaluating confidence scores, logits exposure, and memorization levels to identify privacy vulnerabilities in the system.

Model inversion is a privacy attack where an attacker reconstructs sensitive training data by querying a model's API. It exposes training data by exploiting the model's confidence outputs to reverse-engineer and memorize verbatim information from the original dataset.

To prevent model inversion attacks on deployed ML APIs, you must implement API hardening by limiting exposure of confidences and logits. Additionally, applying data de-identification, rate limiting, and differential privacy training helps restrict attackers from reconstructing sensitive training data.

Yes, differential privacy training protects against membership inference attacks by adding mathematical noise during the training process. This ensures that the model's outputs do not significantly change whether or not a specific individual's data is included, preventing attackers from determining membership.

You should run a memorization audit on your ML data pipelines when training data contains sensitive information and is exposed through API-enabled models. This audit detects verbatim reconstruction risks and verifies that functional privacy controls like de-identification are properly implemented.

The limitation of using API rate limiting for ML model privacy is that it only restricts the volume of queries rather than preventing the extraction of sensitive training data. Without differential privacy or data de-identification, attackers can still perform model inversion within the allowed rate limits.