Hybrid Memory Architecture

1

does SSD in aerospike contains only data? Or Does it contain keys and indexes along with data?

2

Primary Index to locate a record’s data, along with its metadata is always stored in RAM. When records are stored on SSDs, enough metadata is stored along with the record on SSD that the Primary Index can be re-created on an Aerospike process re-start.

1 Like
3

hi, but the documentation of Hybrid Memory Architecture says that the index is purely in-memory (not persisted), and data is stored only on a persistent storage (SSD). Does it means that SSD only contains data (emphasis added) and related metadata, indxes, storage pointer etc is stored in DRAM (no data stored in DRAM).

4

It would be good if you quoted actual sentences from documentation - maybe it’s somehow wrong, and we can fix it. However, your own sentence has no contradictions, but is only partially correct. Let’s break it down:

  1. “the index is purely in-memory”. This is correct - 64B of metadata per-object is the DRAM cost of the primary index, and it’s one of the reasons Aerospike is fast.
  2. “and data is stored only on a persistent storage (SSD)” is correct for a namespace that is configured to store data on SSD. You can also have a namespace that stores fully in-memory, or in-memory with persistence. See Namespace Storage Configuration.

Your assertion that these two bits of information above mean that the SSD only contains data is inaccurate. It says all the data is stored on the SSD. It doesn’t say only the data. You made that claim, probably due to a misunderstanding of the docs.

That aside, because the primary index is fully in memory consider the common case when a node (the actual machine) restarts, rather than just the Aerospike service. At this point there is no primary index. How would the Aerospike daemon on that machine rebuild the primary index? Well, it has to read the data from the persistent storage (SSD) and recreate the metadata entries. This process goes much faster when all the metadata ingredients for this 64B entry in the primary index are stored right next to the data on the SSD.

I hope that clarifies Hybrid Memory Architecture.

One more thing, you also asked if the keys are stored on SSD. That depends on the key write policy you use from the client. By default it’s set to sendKey=false, so the key is not stored with the data. If you set it to true the key will also be stored along with the data. Your key might be a large string, so by default the server doesn’t want to impose that extra storage, relying on the digest.

1 Like
5

Thanku rbotzer, your post clarifies concepts of hybrid memory architecture.

6

This topic was automatically closed 6 days after the last reply. New replies are no longer allowed.