expanso-content-splitting

Splitting a JSON array into individual messages for Kafka routing involves parsing the batch payload and validating each record. Valid items are routed to Kafka, while invalid items are sent to a file-based dead-letter queue for review.

Routing invalid records to a dead-letter queue during batch processing requires content-based validation. Invalid items are diverted to a file-based DLQ, while valid items are routed to a processing queue like Kafka.

Processing raw arrays and CSV records for content-based routing is supported. The pipeline splits various input formats into individual records, enriches each with batch context, and routes them based on validation status.

This message splitting pipeline enriches individual items with batch context during the splitting process. Each record from the JSON array or raw array is processed and tagged before routing to either Kafka or a file-based DLQ.

Separating malformed orders from a daily batch involves splitting the payload into individual messages and validating each one. Malformed orders are routed to a file-based dead-letter queue for logging and review.

File-based dead-letter queues for invalid message routing are limited to local or file system storage. Invalid items are logged to files rather than being sent to a distributed queue like Kafka, which may impact downstream retry automation.