Partition map logic & partitioning algorithm

index

#1

Hey!!

A Key is hashed to map to partitions that is random. But how partitions are mapped to nodes?

RIPEMD-160 is used to Map keys with partitions but there is nothing mentioned about how the partition map is created. How it is decided that which is the replica and which one is the master node for a partition.

Moreover how come do you decided the number 4096 y not 1024 or anything else?


#2

Hi nOOb,

partition map logic is mentioned at http://www.aerospike.com/docs/architecture/data-distribution.html . Every node as well as Aerospike client maintains latest copy of the partition map for the cluster and can look up which partition belong to which nodes (master as well as replica).

We love binary numbers, hence 4096 is 2^12 which is big enough to handle data distribution across large cluster and also small enough to keep partition map in limited size footprint.

-samir


#3

Hi Samir

I went through the docs. Here it was mentioned that each partition is mapped to a master and a replica node. I didn’t understand how RIPEMD is used to map key with partition not partitions with nodes!


#4

The actual partitioning algorithm is generally considered to be below the abstraction layer, but it is documented very well in the source code. Also a while back I wanted to understand the partitioning algorithm better so I wrote a python script to approximate how the partitioning works.