Disk Based Storage is taking too much physical memory

1

Hi,

I am using disk-based storage for our namespace with the below configuration.

namespace lgp {
        replication-factor 2
        memory-size 10G
        default-ttl 0 # 5 days, use 0 to never expire/evict.
        high-water-memory-pct 90 # How full may the memory become before the server begins eviction (expiring records early)
        high-water-disk-pct 80 # How full may the disk become before the server begins eviction (expiring records early)
        stop-writes-pct 90  # How full may the memory  become before  we disallow new writes
#       storage-engine memory

        # To use file storage backing, comment out the line above and use the
        # following lines instead.
        storage-engine device {
                file /myntra/aerospike/data/lgp/object.dat
                filesize 600G
               data-in-memory false
        }
}

yesterday we show that our HWM memory has breached and one node had stopped writing on the namespace.

I saw below logs in that particular node.

Jul 15 2018 23:56:43 GMT+0530: WARNING (namespace): (namespace.c:440) {lgp} hwm_breached true (memory), stop_writes true (memory), memory sz:9668694592 (9668694592 + 0) hwm:9663676416 sw:9663676416, disk sz:180203699968 hwm:515396075520

Jul 15 2018 23:55:00 GMT+0530: WARNING (nsup): (thr_nsup.c:1038) {lgp} no records eligible for eviction

I am not able to understand why disk based namespace is taking so much physical memory.

2

The primary index is stored in memory, 64 bytes per record. Future versions would allow storing index on disk, but this is not available yet.

So, 9668694592/64=151,073,353 which would be the number of records you have in your system at this point (master and its replicas).

3

Thanks for the clarification.