data-autocleaning

Automate data cleaning for BigQuery tables by leveraging Dataplex profiling to assess data quality and apply predefined rules for normalization, type conversions, and garbage value handling. This reduces manual data preparation time.

Yes, you can clean data sourced from Google Cloud Storage using Dataplex profiling. The process handles data quality checks, JSON parsing, array processing, and STRUCT extraction to prepare raw files for analysis.

Yes, you need the gcloud command-line tool installed and configured to execute Dataplex scans. The gcloud tool is a required dependency for running the automated data profiling and cleaning processes.

The best way to handle garbage values and normalize formats in BigQuery is to apply automated data cleaning rules after running Dataplex profiling scans. This standardizes data, resolves anomalies, and normalizes formats efficiently.

Yes, the automated data cleaning process parses complex nested JSON, processes arrays, and extracts STRUCT or Record fields. It transforms these complex data types into standardized formats during the profiling stage.

Avoid automated data profiling when your dataset requires highly customized transformation logic outside standard normalization, type conversions, or garbage value handling, as the process relies on predefined data cleaning rules.