Hi, I am using aerospike cluster of size 7 and I am getting SocketTimeoutException from aerospike for some of the requests. I am having 2 billion records in my namespace and I am also using secondary indexes. When I am using command “stat system <namespace_name>”, I m getting sindex_gc_objects_validated as 8388370110. I looked at the logs of aerospike and secondary index gc was running too frequently in it.
Jun 20 2018 07:39:26 GMT: INFO (nsup): (thr_sindex.c:492) {kratos} sindex-gc start
Jun 20 2018 07:39:27 GMT: INFO (nsup): (thr_sindex.c:523) {kratos} sindex-gc: Processed: 8238, found:0, deleted: 0: Total time: 954 ms
-----------------
Jun 20 2018 07:39:36 GMT: INFO (nsup): (thr_sindex.c:492) {kratos} sindex-gc start
Jun 20 2018 07:39:37 GMT: INFO (nsup): (thr_sindex.c:523) {kratos} sindex-gc: Processed: 8238, found:0, deleted: 0: Total time: 1094 ms
--------------------------
Secondary index gc is running too frequently in all the machines of my aerospike cluster. Is this a normal behaviour? And why is this taking too much time. For every secondary index gc its taking nearly 1 seconds. Is this somehow related to the SocketTimeoutException I am getting intermittently.
Thanks for the help in advance