Kafka-aerospike connector issue

arulkumar · July 14, 2017, 11:45am

Hi I am tried to start kafka to aerospike connector but there is error

when i ran this command

$ connect-standalone config/connect-standalone.properties config/aerospike-connector.properties

[root@rrm3 kafka_2.11-0.9.0.1]# connect-standalone config/connect-standalone.properties config/aerospike-connector.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/local/confluent/confluent-3.2.0/share/java/kafka-serde-tools/slf4j-log4j12-1.7.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/confluent/confluent-3.2.0/share/java/kafka-connect-elasticsearch/slf4j-simple-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/confluent/confluent-3.2.0/share/java/kafka-connect-hdfs/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/confluent/confluent-3.2.0/share/java/kafka-connect-replicator/slf4j-log4j12-1.7.21.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/confluent/confluent-3.2.0/share/java/kafka-connect-s3/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/confluent/confluent-3.2.0/share/java/kafka-connect-storage-common/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/confluent/confluent-3.2.0/share/java/kafka/slf4j-log4j12-1.7.21.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
[2017-07-15 05:14:55,291] INFO StandaloneConfig values:
        access.control.allow.methods =
        access.control.allow.origin =
        bootstrap.servers = [localhost:9092]
        internal.key.converter = class org.apache.kafka.connect.json.JsonConverter
        internal.value.converter = class org.apache.kafka.connect.json.JsonConverter
        key.converter = class org.apache.kafka.connect.json.JsonConverter
        offset.flush.interval.ms = 10000
        offset.flush.timeout.ms = 5000
        offset.storage.file.filename = /tmp/connect.offsets
        rest.advertised.host.name = null
        rest.advertised.port = null
        rest.host.name = null
        rest.port = 8083
        task.shutdown.graceful.timeout.ms = 5000
        value.converter = class org.apache.kafka.connect.json.JsonConverter
 (org.apache.kafka.connect.runtime.standalone.StandaloneConfig:180)
[2017-07-15 05:14:55,464] INFO Logging initialized @1052ms (org.eclipse.jetty.util.log:186)
[2017-07-15 05:14:55,820] INFO Kafka Connect starting (org.apache.kafka.connect.runtime.Connect:50)
[2017-07-15 05:14:55,821] INFO Herder starting (org.apache.kafka.connect.runtime.standalone.StandaloneHerder:72)
[2017-07-15 05:14:55,821] INFO Worker starting (org.apache.kafka.connect.runtime.Worker:119)
[2017-07-15 05:14:55,821] INFO Starting FileOffsetBackingStore with file /tmp/connect.offsets (org.apache.kafka.connect.storage.FileOffsetBackingStore:60)
[2017-07-15 05:14:55,824] INFO Worker started (org.apache.kafka.connect.runtime.Worker:124)
[2017-07-15 05:14:55,825] INFO Herder started (org.apache.kafka.connect.runtime.standalone.StandaloneHerder:74)
[2017-07-15 05:14:55,825] INFO Starting REST server (org.apache.kafka.connect.runtime.rest.RestServer:98)
[2017-07-15 05:14:55,989] INFO jetty-9.2.15.v20160210 (org.eclipse.jetty.server.Server:327)
Jul 15, 2017 5:14:57 AM org.glassfish.jersey.internal.Errors logErrors
WARNING: The following warnings have been detected: WARNING: The (sub)resource method listConnectors in org.apache.kafka.connect.runtime.rest.resources.ConnectorsResource contains empty path annotation.
WARNING: The (sub)resource method createConnector in org.apache.kafka.connect.runtime.rest.resources.ConnectorsResource contains empty path annotation.
WARNING: The (sub)resource method listConnectorPlugins in org.apache.kafka.connect.runtime.rest.resources.ConnectorPluginsResource contains empty path annotation.
WARNING: The (sub)resource method serverInfo in org.apache.kafka.connect.runtime.rest.resources.RootResource contains empty path annotation.

[2017-07-15 05:14:57,062] INFO Started o.e.j.s.ServletContextHandler@1d9bec4d{/,null,AVAILABLE} (org.eclipse.jetty.server.handler.ContextHandler:744)
[2017-07-15 05:14:57,078] INFO Started ServerConnector@9d157ff{HTTP/1.1}{0.0.0.0:8083} (org.eclipse.jetty.server.ServerConnector:266)
[2017-07-15 05:14:57,080] INFO Started @2668ms (org.eclipse.jetty.server.Server:379)
[2017-07-15 05:14:57,081] INFO REST server listening at http://172.31.1.42:8083/, advertising URL http://172.31.1.42:8083/ (org.apache.kafka.connect.runtime.rest.RestServer:150)
[2017-07-15 05:14:57,082] INFO Kafka Connect started (org.apache.kafka.connect.runtime.Connect:56)
[2017-07-15 05:14:57,082] ERROR Stopping after connector error (org.apache.kafka.connect.cli.ConnectStandalone:99)
java.io.FileNotFoundException: config/aerospike-connector.properties (No such file or directory)
        at java.io.FileInputStream.open0(Native Method)
        at java.io.FileInputStream.open(FileInputStream.java:195)
        at java.io.FileInputStream.<init>(FileInputStream.java:138)
        at java.io.FileInputStream.<init>(FileInputStream.java:93)
        at org.apache.kafka.common.utils.Utils.loadProps(Utils.java:488)
        at org.apache.kafka.connect.cli.ConnectStandalone.main(ConnectStandalone.java:83)
[2017-07-15 05:14:57,089] INFO Kafka Connect stopping (org.apache.kafka.connect.runtime.Connect:66)
[2017-07-15 05:14:57,089] INFO Stopping REST server (org.apache.kafka.connect.runtime.rest.RestServer:154)
[2017-07-15 05:14:57,112] INFO Stopped ServerConnector@9d157ff{HTTP/1.1}{0.0.0.0:8083} (org.eclipse.jetty.server.ServerConnector:306)
[2017-07-15 05:14:57,134] INFO Stopped o.e.j.s.ServletContextHandler@1d9bec4d{/,null,UNAVAILABLE} (org.eclipse.jetty.server.handler.ContextHandler:865)
[2017-07-15 05:14:57,136] INFO REST server stopped (org.apache.kafka.connect.runtime.rest.RestServer:165)
[2017-07-15 05:14:57,136] INFO Herder stopping (org.apache.kafka.connect.runtime.standalone.StandaloneHerder:78)
[2017-07-15 05:14:57,136] INFO Worker stopping (org.apache.kafka.connect.runtime.Worker:131)
[2017-07-15 05:14:57,137] INFO Stopped FileOffsetBackingStore (org.apache.kafka.connect.storage.FileOffsetBackingStore:68)
[2017-07-15 05:14:57,137] INFO Worker stopped (org.apache.kafka.connect.runtime.Worker:151)
[2017-07-15 05:15:14,972] INFO Reflections took 19055 ms to scan 579 urls, producing 13713 keys and 89742 values  (org.reflections.Reflections:229)
[2017-07-15 05:15:15,006] INFO Herder stopped (org.apache.kafka.connect.runtime.standalone.StandaloneHerder:88)
[2017-07-15 05:15:15,007] INFO Kafka Connect stopped (org.apache.kafka.connect.runtime.Connect:71)

Jan · July 15, 2017, 2:14am

@arulkumar,

This seems to be a mistake in the documentation for the connector. The name of the config file is actually connect-aerospike-sink.properties. So you should start the connector in stand-alone mode like this:

connect-standalone config/connect-standalone.properties config/config/connect-aerospike-sink.properties

I will fix the documentation as well.

Cheers, Jan

arulkumar · July 17, 2017, 1:56pm

Hi ,Kafka-aerospike connectivity issue

I have follwed same property as you given for this issue

$connect-standalone config/connect-standalone.properties config/connect-aerospike-sink.properties

i got following error

[root@rrm3 kafka_2.11-0.9.0.1]# connect-standalone config/connect-standalone.properties config/connect-aerospike-sink.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/local/confluent/confluent-3.2.0/share/java/kafka-serde-tools/slf4j-log4j12-1.7.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/confluent/confluent-3.2.0/share/java/kafka-connect-elasticsearch/slf4j-simple-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/confluent/confluent-3.2.0/share/java/kafka-connect-hdfs/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/confluent/confluent-3.2.0/share/java/kafka-connect-replicator/slf4j-log4j12-1.7.21.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/confluent/confluent-3.2.0/share/java/kafka-connect-s3/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/confluent/confluent-3.2.0/share/java/kafka-connect-storage-common/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/confluent/confluent-3.2.0/share/java/kafka/slf4j-log4j12-1.7.21.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
[2017-07-18 07:39:59,519] INFO StandaloneConfig values:
        access.control.allow.methods =
        access.control.allow.origin =
        bootstrap.servers = [localhost:9092]
        internal.key.converter = class org.apache.kafka.connect.json.JsonConverter
        internal.value.converter = class org.apache.kafka.connect.json.JsonConverter
        key.converter = class org.apache.kafka.connect.json.JsonConverter
        offset.flush.interval.ms = 10000
        offset.flush.timeout.ms = 5000
        offset.storage.file.filename = /tmp/connect.offsets
        rest.advertised.host.name = null
        rest.advertised.port = null
        rest.host.name = null
        rest.port = 8083
        task.shutdown.graceful.timeout.ms = 5000
        value.converter = class org.apache.kafka.connect.json.JsonConverter
 (org.apache.kafka.connect.runtime.standalone.StandaloneConfig:180)
[2017-07-18 07:39:59,689] INFO Logging initialized @871ms (org.eclipse.jetty.util.log:186)
[2017-07-18 07:40:00,031] INFO Kafka Connect starting (org.apache.kafka.connect.runtime.Connect:50)
[2017-07-18 07:40:00,032] INFO Herder starting (org.apache.kafka.connect.runtime.standalone.StandaloneHerder:72)
[2017-07-18 07:40:00,032] INFO Worker starting (org.apache.kafka.connect.runtime.Worker:119)
[2017-07-18 07:40:00,032] INFO Starting FileOffsetBackingStore with file /tmp/connect.offsets (org.apache.kafka.connect.storage.FileOffsetBackingStore:60)
[2017-07-18 07:40:00,035] INFO Worker started (org.apache.kafka.connect.runtime.Worker:124)
[2017-07-18 07:40:00,036] INFO Herder started (org.apache.kafka.connect.runtime.standalone.StandaloneHerder:74)
[2017-07-18 07:40:00,036] INFO Starting REST server (org.apache.kafka.connect.runtime.rest.RestServer:98)
[2017-07-18 07:40:00,196] INFO jetty-9.2.15.v20160210 (org.eclipse.jetty.server.Server:327)
Jul 18, 2017 7:40:01 AM org.glassfish.jersey.internal.Errors logErrors
WARNING: The following warnings have been detected: WARNING: The (sub)resource method listConnectors in org.apache.kafka.connect.runtime.rest.resources.ConnectorsResource contains empty path annotation.
WARNING: The (sub)resource method createConnector in org.apache.kafka.connect.runtime.rest.resources.ConnectorsResource contains empty path annotation.
WARNING: The (sub)resource method listConnectorPlugins in org.apache.kafka.connect.runtime.rest.resources.ConnectorPluginsResource contains empty path annotation.
WARNING: The (sub)resource method serverInfo in org.apache.kafka.connect.runtime.rest.resources.RootResource contains empty path annotation.

[2017-07-18 07:40:01,287] INFO Started o.e.j.s.ServletContextHandler@1d9bec4d{/,null,AVAILABLE} (org.eclipse.jetty.server.handler.ContextHandler:744)
[2017-07-18 07:40:01,303] INFO Started ServerConnector@9d157ff{HTTP/1.1}{0.0.0.0:8083} (org.eclipse.jetty.server.ServerConnector:266)
[2017-07-18 07:40:01,305] INFO Started @2489ms (org.eclipse.jetty.server.Server:379)
[2017-07-18 07:40:01,307] INFO REST server listening at http://172.31.1.42:8083/, advertising URL http://172.31.1.42:8083/ (org.apache.kafka.connect.runtime.rest.RestServer:150)
[2017-07-18 07:40:01,307] INFO Kafka Connect started (org.apache.kafka.connect.runtime.Connect:56)
[2017-07-18 07:40:19,924] INFO Reflections took 19802 ms to scan 579 urls, producing 13713 keys and 89742 values  (org.reflections.Reflections:229)
[2017-07-18 07:40:20,548] INFO Reflections took 19207 ms to scan 579 urls, producing 13713 keys and 89742 values  (org.reflections.Reflections:229)
[2017-07-18 07:40:20,551] ERROR Failed to create job for config/connect-aerospike-sink.properties (org.apache.kafka.connect.cli.ConnectStandalone:88)
[2017-07-18 07:40:20,552] ERROR Stopping after connector error (org.apache.kafka.connect.cli.ConnectStandalone:99)
java.util.concurrent.ExecutionException: org.apache.kafka.connect.errors.ConnectException: Failed to find any class that implements Connector and which name matches com.aerospike.kafka.connect.sink.AerospikeSinkConnector, available connectors are: org.apache.kafka.connect.sink.SinkConnector, org.apache.kafka.connect.tools.VerifiableSourceConnector, org.apache.kafka.connect.tools.MockSourceConnector, org.apache.kafka.connect.tools.SchemaSourceConnector, io.confluent.connect.elasticsearch.ElasticsearchSinkConnector, io.confluent.connect.storage.tools.SchemaSourceConnector, io.confluent.connect.jdbc.JdbcSinkConnector, io.confluent.connect.hdfs.tools.SchemaSourceConnector, org.apache.kafka.connect.tools.MockConnector, io.confluent.connect.s3.S3SinkConnector, org.apache.kafka.connect.tools.VerifiableSinkConnector, io.confluent.connect.jdbc.JdbcSourceConnector, org.apache.kafka.connect.tools.MockSinkConnector, org.apache.kafka.connect.file.FileStreamSourceConnector, io.confluent.connect.hdfs.HdfsSinkConnector, org.apache.kafka.connect.file.FileStreamSinkConnector, org.apache.kafka.connect.source.SourceConnector, io.confluent.connect.replicator.ReplicatorSourceConnector
        at org.apache.kafka.connect.util.ConvertingFutureCallback.result(ConvertingFutureCallback.java:80)
        at org.apache.kafka.connect.util.ConvertingFutureCallback.get(ConvertingFutureCallback.java:67)
        at org.apache.kafka.connect.cli.ConnectStandalone.main(ConnectStandalone.java:96)
Caused by: org.apache.kafka.connect.errors.ConnectException: Failed to find any class that implements Connector and which name matches com.aerospike.kafka.connect.sink.AerospikeSinkConnector, available connectors are: org.apache.kafka.connect.sink.SinkConnector, org.apache.kafka.connect.tools.VerifiableSourceConnector, org.apache.kafka.connect.tools.MockSourceConnector, org.apache.kafka.connect.tools.SchemaSourceConnector, io.confluent.connect.elasticsearch.ElasticsearchSinkConnector, io.confluent.connect.storage.tools.SchemaSourceConnector, io.confluent.connect.jdbc.JdbcSinkConnector, io.confluent.connect.hdfs.tools.SchemaSourceConnector, org.apache.kafka.connect.tools.MockConnector, io.confluent.connect.s3.S3SinkConnector, org.apache.kafka.connect.tools.VerifiableSinkConnector, io.confluent.connect.jdbc.JdbcSourceConnector, org.apache.kafka.connect.tools.MockSinkConnector, org.apache.kafka.connect.file.FileStreamSourceConnector, io.confluent.connect.hdfs.HdfsSinkConnector, org.apache.kafka.connect.file.FileStreamSinkConnector, org.apache.kafka.connect.source.SourceConnector, io.confluent.connect.replicator.ReplicatorSourceConnector
        at org.apache.kafka.connect.runtime.ConnectorFactory.getConnectorClass(ConnectorFactory.java:84)
        at org.apache.kafka.connect.runtime.ConnectorFactory.newConnector(ConnectorFactory.java:38)
        at org.apache.kafka.connect.runtime.AbstractHerder.getConnector(AbstractHerder.java:334)
        at org.apache.kafka.connect.runtime.AbstractHerder.validateConnectorConfig(AbstractHerder.java:233)
        at org.apache.kafka.connect.runtime.standalone.StandaloneHerder.putConnectorConfig(StandaloneHerder.java:159)
        at org.apache.kafka.connect.cli.ConnectStandalone.main(ConnectStandalone.java:93)
[2017-07-18 07:40:20,554] INFO Kafka Connect stopping (org.apache.kafka.connect.runtime.Connect:66)
[2017-07-18 07:40:20,554] INFO Stopping REST server (org.apache.kafka.connect.runtime.rest.RestServer:154)
[2017-07-18 07:40:20,558] INFO Stopped ServerConnector@9d157ff{HTTP/1.1}{0.0.0.0:8083} (org.eclipse.jetty.server.ServerConnector:306)
[2017-07-18 07:40:20,565] INFO Stopped o.e.j.s.ServletContextHandler@1d9bec4d{/,null,UNAVAILABLE} (org.eclipse.jetty.server.handler.ContextHandler:865)
[2017-07-18 07:40:20,567] INFO REST server stopped (org.apache.kafka.connect.runtime.rest.RestServer:165)
[2017-07-18 07:40:20,567] INFO Herder stopping (org.apache.kafka.connect.runtime.standalone.StandaloneHerder:78)
[2017-07-18 07:40:20,567] INFO Worker stopping (org.apache.kafka.connect.runtime.Worker:131)
[2017-07-18 07:40:20,568] INFO Stopped FileOffsetBackingStore (org.apache.kafka.connect.storage.FileOffsetBackingStore:68)
[2017-07-18 07:40:20,568] INFO Worker stopped (org.apache.kafka.connect.runtime.Worker:151)
[2017-07-18 07:40:20,568] INFO Herder stopped (org.apache.kafka.connect.runtime.standalone.StandaloneHerder:88)
[2017-07-18 07:40:20,568] INFO Kafka Connect stopped (org.apache.kafka.connect.runtime.Connect:71)
[root@rrm3 kafka_2.11-0.9.0.1]#  ERROR Failed to create job for config/connect-aerospike-sink.properties (org.apache.kafka.connect.cli.ConnectStandalone:88)

rbotzer · July 17, 2017, 3:47pm

When it seems to be a bug that kind of information is useful on the GitHub repo, in this case aerospike/aerospike-kafka-connector.

Please open a new issue at the repo, with this information. It would be good if you added information about your environment (OS, etc).

Topic		Replies	Views
Aerospike Kafka connector throwing error max connections 25 would be exceeded Kafka	3	1835	July 10, 2019
Aerospike outbound source connector 2.2.0 XDR (Cross Data Center Replication) query	2	150	December 18, 2023
Aerospike connector sink errors.tollerance=all is not working Kafka	0	834	February 24, 2020
Aerospike docker connection refused using docker servicename	1	1104	November 30, 2019
Aerospike JDBC connector is not working	5	2590	June 11, 2016

Kafka-aerospike connector issue

Related Topics