Go client brings partial data when got disconnected from a node

Hi, I’m running an AS cluster with 2 nodes and replication factor of 2 on GCE. The cluster is configured to use mesh over private network for internode communication. When I connect to the cluster from outside, supplying one seed using it’s public IP and then run a scan, it brings only master copies stored on that node. When I shut the other node down and run the scan again, it brings the whole data (master + replica) as expected.

Please advise if it’s a bug or there is some workaround.

Thanks, Dmitri.

If that’s the case, it’s should be a bug. Let me investigate and get back to you.

Could you provide your cluster configuration? (or is it possible to use your cluster for debugging?)

Thank you for getting back. Actually my configuration is pretty standard (please see below):

# AS configuration file

service {
        user root
        group root
        paxos-single-replica-limit 1
        pidfile /var/run/aerospike/asd.pid
        service-threads 4
        transaction-queues 4
        transaction-threads-per-queue 4
        proto-fd-max 15000
}

logging {
        # Log file must be an absolute path.
        file /var/log/aerospike/aerospike.log {
                context any info
        }
}

network {
        service {
                address any
                port 3000
        }
        heartbeat {
                mode mesh
                address ${MY_INTERNAL_IP}  # IP of the NIC on which this node is listening
                port 3002

                # IP address for seed node in the cluster
                mesh-seed-address-port ${MY_INTERNAL_IP} 3002
                mesh-seed-address-port ${OTHER_NODE_INTERNAL_IP} 3002

                interval 150
                timeout 20
        }
        fabric {
                port 3001
        }
        info {
                port 3003
        }
}

namespace ${MY_NS} {
        replication-factor 2
        memory-size YG
        default-ttl 0
        ldt-enabled true
        storage-engine device {
               file /opt/aerospike/data/${MY_NS}.dat
               filesize XG
               disable-odirect true
               data-in-memory true
       }

Unfortunately, it’s impossible to open the cluster for debugging. But I may provide you with steps how to reproduce the issue. As for the versions, I use Debian 7.9 and AS 3.7.4 community edition.

Thank you, Dmitri.