How to know where the metrics streams are coming from?


#1

I’m using aerospike enterprise 3.10 and telegraf to ship the metrics in a cluster of aerospike nodes. I found the metrics being generated for tx and rx part of replication from each of the nodes, but lacking any field for conveying the hostname or ip address of the origin nodes.

I’m wondering if there is any way to configure the aerospike including its hostname or ip address in the metrics set, or how can I distinguish between them?


#2

Looks like there is a PR for this issue in the telegraf plugin for Aerospike:

https://github.com/influxdata/telegraf/pull/2918

It was merged 22 days ago.


#3

I saw it. But the node_name itself is from the aerospike which is generated based on interface mac address.


#4

That is how that plugin was implemented. Typically monitoring tools would have a small daemon on each server that ships data from its server only to an aggregator. This plugin opted to query all servers. In the thread I linked, the author acknowledged this issue but didn’t know of a go client api to query local node. I responded to that thread with an api I believe should work.

The author also mentioned a workaround in the thread I linked.


#5

Btw, Aerospike maintains our own collectd plugin and there is a collectd to Prometheus exporter:

May consider this as an alternative.


#6

Quick look, the telegraf plugin only reports service and namespace stats. There are also network, latency, and xdr contexts that are reported by the collectd plugin and missing from telegraf’s.


#7

Thanks for your insights on the monitoring plugin and taking a look at the telegraf. We switched from collectd due to the license issues for some of the major plugins (e.g., network). Our laywers don’t like GPL. As far as I tested, the telegraf sends two contexts for aerospike: namespace of the cluster and node-level stats (but still missing the origin host id).

Yes, the guys working on the PR seemed not knowing how to get the info from the aerospike nodes, and thanks for your input in there. Will keep an eye on the thread.