Wrong node list from cluster

Operations Tuning
1

by ikochergin » Tue Aug 12, 2014 8:39 am

We had a problem with the client connection:

Configuration:

CODE: SELECT ALL
--------------------------
|        node_1          |
|  eth0    |    bond0    |
| ext_ip1  | 192.168.1.1 |
--------------------------
    |          | | | 
 INTERNET     INT_LAN
    |          | | |
--------------------------
| ext_ip2  | 192.168.1.2 |
|  ext0    |    bond0    |
|        node_2          |
--------------------------

node_1

CODE: SELECT ALL
network {
    service {
        address                       ext_ip1
        port                          3000
        reuse-address
    }
    fabric {
        address                       192.168.1.1
        port                          3001
    }
    heartbeat {
        mode                          mesh
        address                       192.168.1.1
        port                          3002
        mesh-address                  192.168.1.2
        mesh-port                     3002
        interval                      500
        timeout                       30
    }
    info {
        address                       any
        port                          3003
    }
}


node_2
CODE: SELECT ALL
network {
    service {
        address                       ext_ip2
        port                          3000
        reuse-address
    }
    fabric {
        address                       192.168.1.2
        port                          3001
    }
    heartbeat {
        mode                          mesh
        address                       192.168.1.2
        port                          3002
        mesh-address                  192.168.1.1
        mesh-port                     3002
        interval                      500
        timeout                       30
    }
    info {
        address                       any
        port                          3003
    }
}

To client returns all addreses internal and external

CODE: SELECT ALL
15:31:44.949 [main] DEBUG r.m.e.aerospike.AerospikeService - INFO: Add node BB970D95F3ACAB8 ext_ip1:3000
15:31:44.951 [main] DEBUG r.m.e.aerospike.AerospikeService - INFO: Add node BB9188A6D3ACAB8 ext_ip2:3000
15:31:45.071 [main] DEBUG r.m.e.aerospike.AerospikeService - DEBUG: Node BB970D95F3ACAB8 ext_ip1:3000 partition generation 27 changed.
15:31:45.258 [main] DEBUG r.m.e.aerospike.AerospikeService - DEBUG: Node BB9188A6D3ACAB8 ext_ip2:3000 partition generation 27 changed.
15:31:46.326 [main] DEBUG r.m.e.aerospike.AerospikeService - DEBUG: Alias 192.168.1.2:3000 failed: Error Code 11: java.net.SocketTimeoutException: connect timed out
15:31:46.327 [main] DEBUG r.m.e.aerospike.AerospikeService - WARN: Add node 192.168.1.2:3000 failed: Error Code 11: Failed to connect to host aliases: [192.168.1.2:3000]
15:31:47.328 [main] DEBUG r.m.e.aerospike.AerospikeService - DEBUG: Alias 192.168.1.1:3000 failed: Error Code 11: java.net.SocketTimeoutException: connect timed out
15:31:47.328 [main] DEBUG r.m.e.aerospike.AerospikeService - WARN: Add node 192.168.1.1:3000 failed: Error Code 11: Failed to connect to host aliases: [192.168.1.1:3000]
15:31:47.330 [main] DEBUG r.m.e.aerospike.AerospikeService - DEBUG: Add seed aerospike.madnet.ru:3000
15:31:47.330 [main] DEBUG r.m.e.aerospike.AerospikeService - DEBUG: Add seed ext_ip2:3000
15:31:47.330 [main] DEBUG r.m.e.aerospike.AerospikeService - DEBUG: Add seed ext_ip1:3000


CODE: SELECT ALL
asinfo -h ext_ip2 -v "services"
requested value  services
value is  ext_ip1:3000;192.168.1.2:3000

asinfo -h ext_ip2 -v "services"
requested value  services
value is  ext_ip1:3000;192.168.1.1:3000

But 192.168.1.0/24 is internal lan and client do not have access to it.

How to disable the sending of internal addresses to the client?

2

by devops01 » Tue Aug 12, 2014 10:24 am

Adding the access-address should resolve your issue:

CODE: SELECT ALL
network {
  service {
    address ext_ip1                  # ip of the NIC on which the service is
                                 # listening.
    port 3000                    # port on which the service is listening.
    access-address ext_ip1 # ip address exported to clients that access
                                 # the service.
  }

http://www.aerospike.com/docs/operations/configure/network/general/

best,

Lucien

3

by ikochergin » Tue Aug 12, 2014 11:16 pm

Is it possible to listen on all addresses but give to clients the node address from the subnet from which they came? We have a part of services connecting internal network and part of external network.

Thanks

4

by devops01 » Thu Aug 14, 2014 1:12 pm

Would it be possible to know which client you are using? There may may options at the client level.

5

by ikochergin » Fri Aug 15, 2014 2:08 am

All our software is built on C / C++ 3.0.70 and Java 3.0.27 clients.