Spark jobs fail with
partition map empty
When a Spark job runs it fails very quickly with an error in the following form:
com.aerospike.client.AerospikeException$InvalidNamespace: Error 20,1: Partition map empty
This error indicates that the node on which the Spark client resides has a connectivity issue to the Aerospike cluster. The Spark Connector uses an instance of the Aerospike Java Smart Client to connect to the Aerospike database. When the Client instance is created the Client will connect to each cluster node to determine what partitions that node owns. This process is called
tending. Ultimately, once the tending process has completed, the Client has a partition map which cross references partitions with the nodes upon which they reside. This error indicates that there is no partition map which means that the Client has not been able to connect to any of the cluster nodes.
The error above will happen when there is a lack of connectivity between the Client used by the Spark connector and the Aerospike cluster. To resolve the error, resolve the connectivity problem. Connectivity can be checked using normal network tools such as
- The tending process is discussed in greater detail in the following article.
- The tending process repeats every 1 second by default. The error discussed here indicates a continuing inability to connect to the cluster.
- If the Client can connect to some of the cluster nodes but not all there may be gaps in the partition map which would lead to
Node Not Found for Partitionerrors.
SPARK ERROR PARTITION JOB FAIL EMPTY