FAQ Why do creates, updates and deletes get sent to different Kafka topics with the Aerospike Outbound Kafka Connector
When using the Aerospike Outbound Kafka Connector to ship changes from an Aerospike database to Kafka and routing by set, deletes are sent to a different Kafka topic than creates and updates. Why is this?
If sets are being used to route between different Kafka topics then deletes will not be sent to that topic as deletes are shipped without a set by XDR. As they cannot be routed, the deletes are sent to the default Kafka topic.
In the example below, creates and updates belonging to the set
user_profile_test in namespace
customer_user_profile_test will be sent to Kafka topic
user_profile_test whereas the deletes will be sent to
service: port: 8080 producer-props: bootstrap.servers: - kafka-development.mycloud.com:9092 logging: file: /var/log/aerospike-kafka-outbound/aerospike-kafka-outbound.log format: mode: json routing: mode: static destination: default # <<< this is a name of a topic >>> namespaces: customer_universal_user_profile: sets: user_profile_test: routing: mode: static destination: user_profile_test format: mode: flat-json metadata-key: metadata
KAFKA OUTBOUND TOPIC DELETE SET