Reply with service IPs on interface where client asks

Operations Configuration
1

Hi,

I set up internal network for intra-cluster communication and expects if client asks cluster nodes via internal IP address then service-addresses to be returned have to be also internal. But when client asks via public IP then external IPs have to be returned.

network {
service {
address                       any
port                          3000
access-address                public_ip
reuse-address
}
fabric {
address                       private_ip
port                          3001
}
heartbeat {
mode                          multicast
address                       239.X.X.X
port                          9091
interface-address             private_ip
interval                      500
timeout                       30
}
info {
address                       any
port                          3003
}
}

Regards, Alex

3

The access-address parameter determines what a given host will advertise to the client applications. There isn’t any configuration option available for what you are requesting.

4

Hi kporter,

Well, I had thought so too. But … Right now client applications came to decision to use private_ip to connect to servers. Although, public_ip is specified in access-address on all AE servers. I’m confused how it works.

Regards, Alex

5

Hi kporter,

Here’s the dump of the answer from from cluster with configuration listed above:

root@ae1:~# java -jar ae-nodes-list-0.1.jar ae-internal.domain.com 3000
14:32:01.000 [main] INFO  r.m.enrichment.tools.ae.nodes.Main -

local:
private_ip_1
public_ip_1
127.0.0.1

14:32:01.011 [main] INFO  r.m.enrichment.tools.ae.nodes.Main - connecting to ae-internal.domain.com:3000...
14:32:01.080 [main] INFO  r.m.enrichment.tools.ae.nodes.Main - connected
14:32:01.080 [main] INFO  r.m.enrichment.tools.ae.nodes.Main - nodes=3
14:32:01.080 [main] INFO  r.m.enrichment.tools.ae.nodes.Main -

node 0:
node.name=BB9B498XXXXXXXX
node.host=private_ip_1:3000
node.address=/private_ip_1:3000
node.aliases=[private_ip_1:3000, public_ip_1:3000]

14:32:01.080 [main] INFO  r.m.enrichment.tools.ae.nodes.Main -

node 1:
node.name=BB95AE2XXXXXXXX
node.host=private_ip_3:3000
node.address=/private_ip_3:3000
node.aliases=[private_ip_3:3000, public_ip_3:3000]

14:32:01.081 [main] INFO  r.m.enrichment.tools.ae.nodes.Main -

node 2:
node.name=BB91CA8XXXXXXXX
node.host=private_ip_2:3000
node.address=/private_ip_2:3000
node.aliases=[private_ip_2:3000, public_ip_2:3000]

14:32:01.081 [main] INFO  r.m.enrichment.tools.ae.nodes.Main - --- done ---

Regards, Alex

6

Looks like there’s an internal to external IP mapping defined in the client policy: http://www.aerospike.com/apidocs/java/com/aerospike/client/policy/ClientPolicy.html#ipMap

7

Hi kporter,

No policy at all (at least no traces in client’s config). I have 2 A records in DNS: one has all 3 public IPs, second - all 3 private IPs. When client is pointed to public A record - query returns only public IPs. When pointed to private A record - Aerospike servers return what I wrote above.

But what I can’t understand - how it does work. The initial question had emerged when I installed Aerospike cluster in second DC (we already had working Aerospike cluster with scheme described in this topic) and clients, pointed to internal A record began connecting to public IPs. After some time had passed they happened to connect to private IPs. Without any changes in client config.

Regards, Alex

8

Best to concentrate on the predictable method:

  • Have access-address be public-ips, for all nodes in the cluster.
  • Have one DNS entry also with only public ips as entries.
  • If the client wants to access cluster via public ips, then nothing special needs to be done.
  • If a different client wants to access the same cluster via private ips, then create on client side an ipmap, so public ip will be translated before accessing.
9

Hi wchu,

I think it’s better to reply with public ips when asked on public interface. And with private ips when asked on private interface. Client might not have idea how to map public->private ip.

Regards, Alex

10

I was just trying to figure out the exact same thing - my cluster only uses private ip-s so when I connect to it from outside of aws using public IP how does it know which node to go to for the data?!

11

The client is probably connecting to only the seed nodes you provided, if there were any nodes outside that list the client would probably have to choose a random node to send the request to which would then proxy to the appropriate node. You can see you proxy rate by running asadm -e "show latency like proxy".