Kafka to S3 with Kafka Connect: S3 Sink config, partitioning and a working demo
Why not write your own Kafka-to-S3 consumer, what Kafka Connect gives you, and the S3 Sink settings that matter: flush.size, rotation, Parquet and partitioners.
Why not write your own Kafka-to-S3 consumer, what Kafka Connect gives you, and the S3 Sink settings that matter: flush.size, rotation, Parquet and partitioners.