Invalid Proto Version when setting up outbound Kafka connector in Aerospike versions < 5.0

Knowledge Base Connectors
1

The Aerospike Knowledge Base has moved to https://support.aerospike.com. Content on https://discuss.aerospike.com is being migrated to either https://support.aerospike.com or https://docs.aerospike.com. Maintenance on articles stored in this repository ceased on December 31st 2022 and this article may be stale. If you have any questions, please do not hesitate to raise a case via https://support.aerospike.com.

Invalid Proto Version when setting up outbound Kafka connector in Aerospike versions < 5.0

Problem Description

When setting up an outbound Kafka connector from Aerospike the following error is observed in the logs:

May 31 2020 09:20:04 GMT: INFO (xdr): (xdr_ship.c:713) Connecting to 'kafka_connector'
May 31 2020 09:20:04 GMT: INFO (xdr): (xdr_dlog.c:106) dlog: free-pct 100 reclaimed 0 glst 1590912631088 (2020-05-31 08:10:31 GMT)
May 31 2020 09:20:04 GMT: INFO (xdr): (xdr.c:544) [kafka_connector]: dc-state CLUSTER_INACTIVE timelag-sec 0 lst 1590912636851 mlst 1590916804566 (2020-05-31 09:20:04 GMT) fnlst 0 (-) wslst 0 (-) shlat-ms 0 errcl 0 errsrv 0 rsas-ms 0.000 rsas-pct 0.0
May 31 2020 09:20:04 GMT: INFO (xdr): (xdr.c:586) summary: throughput 0 inflight 0 dlog-outstanding 0 dlog-delta-per-sec 0.0
May 31 2020 09:20:04 GMT: INFO (xdr): (xdr.c:596) detail: sh 0 ul 0 lg 14 rlg 0 rlgi 4 rlgo 3 lproc 14 rproc 1 lkdproc 0 errcl 0 errsrv 0 hkskip 8 hkf 1 flat 0
May 31 2020 09:20:04 GMT: WARNING (xdr-client): (as_cluster.c:236) Failed to connect to seed kafka-connector-customer-staging.domain.com 3000. AEROSPIKE_ERR_CLIENT Received invalid proto version: 72 Expected: 2, 10.0.0.12:3000
May 31 2020 09:20:04 GMT: WARNING (xdr-client): (as_cluster.c:236) Failed to connect to seed kafka-connector-customer-staging.domain.com 3000.

Explanation

This error will be displayed when the destination cluster specification the source has not been done properly. In this instance the most likely cuplrit is that the Kafka destination has been specified in the same way as an Aerospike cluster, which is incorrect. The destination must be configured as a Change Notification DC.

Solution

To resolve the issue the destination cluster should be specified correctly for Change Notification. The error message above shows the connection is being made on port 3000 which is a smoking gun as this is normally the port an Aerospike cluster listens on. A Kafka connector would more likely listen on 8080. An example DC specification for Aerospike < 5.0 is shown below.

xdr {
        enable-xdr true
        enable-change-notification true
        xdr-digestlog-path /etc/aerospike/digestlog.log1 100G
        xdr-compression-threshold 1000
        xdr-info-port 3004



        datacenter kafka_connector {
                dc-type http
                http-version v1
                http-url http://10.0.0.1:8080
                http-url http://10.0.0.2:8080
        }
}

Notes

  • Full instructions for configuring Change Notification destinations can be found here.
  • Change notification destinations are used by several Aerospike outbound connectors such as Kafka and JMS.
  • Aerospike versions > 5.0 are simplified in their Change Notification configuration.

Keywords

KAFKA INVALID PROTO VERSION OUTBOUND CONNECTOR HTTP

Timestamp

June 2020