Spark jobs fail with partition map empty

Knowledge Base Connectors
1

The Aerospike Knowledge Base has moved to https://support.aerospike.com. Content on https://discuss.aerospike.com is being migrated to either https://support.aerospike.com or https://docs.aerospike.com. Maintenance on articles stored in this repository ceased on December 31st 2022 and this article may be stale. If you have any questions, please do not hesitate to raise a case via https://support.aerospike.com.

Spark jobs fail with partition map empty

Problem Description

When a Spark job runs it fails very quickly with an error in the following form:

com.aerospike.client.AerospikeException$InvalidNamespace: Error 20,1: Partition map empty

Explanation

This error indicates that the node on which the Spark client resides has a connectivity issue to the Aerospike cluster. The Spark Connector uses an instance of the Aerospike Java Smart Client to connect to the Aerospike database. When the Client instance is created the Client will connect to each cluster node to determine what partitions that node owns. This process is called tending. Ultimately, once the tending process has completed, the Client has a partition map which cross references partitions with the nodes upon which they reside. This error indicates that there is no partition map which means that the Client has not been able to connect to any of the cluster nodes.

Solution

The error above will happen when there is a lack of connectivity between the Client used by the Spark connector and the Aerospike cluster. To resolve the error, resolve the connectivity problem. Connectivity can be checked using normal network tools such as ping and telnet.

Notes

  • The tending process is discussed in greater detail in the following article.
  • The tending process repeats every 1 second by default. The error discussed here indicates a continuing inability to connect to the cluster.
  • If the Client can connect to some of the cluster nodes but not all there may be gaps in the partition map which would lead to Node Not Found for Partition errors.

Keywords

SPARK ERROR PARTITION JOB FAIL EMPTY

Timestamp

June 2020