Development Environment with Small Ram


#1

Hi,

I’ve been fighting with aerospike since late last night.

I have a maximum memory allocation for a virtual machine of 4GB.

I need 3 bins to simulate my production environment.

I keep on getting Server Full issues.

I’ve done:

sysctl vm.overcommit_memory=1

But to no avail.

Below is my config.

I’m at the point, where I just want this to work, test it and put it into production.

Don’t care of I have to rig something to make it work, just want it to work!

Remember, this is a dev env. Don’t need to be running gigs and gigs of data.

# Aerospike database configuration file.

service {
    user root
    group root
    paxos-single-replica-limit 1 # Number of nodes where the replica count is automatically reduced to 1.
    pidfile /var/run/aerospike/asd.pid
    service-threads 4
    transaction-queues 4
    transaction-threads-per-queue 4
    proto-fd-max 15000
}

logging {
    # Log file must be an absolute path.
    file /var/log/aerospike/aerospike.log {
        context any info
    }
}

network {
    service {
        address any              # IP of the NIC on which the service is
                                 # listening.
        port 3000                    # port on which the service is listening.
    }

    heartbeat {
        mode multicast                  # Send heartbeats using Multicast
        address 239.1.99.2              # multicast address
        port 9918                       # multicast port
        interface-address 172.16.153.176  # IP of the NIC to use to send out heartbeat
                                        # and bind fabric ports
        interval 150                    # Number of milliseconds between heartbeats
        timeout 10                      # Number of heartbeat intervals to wait
                                        # before timing out a node
    }

    fabric {
        port 3001 # Intra-cluster communication port (migrates, replication, etc).
    }

    info {
        port 3003 # Plain text telnet management port.
    }
}


namespace cache {
    memory-size 1G
    #replication-factor 2
    default-ttl 0

    storage-engine device {
        file /opt/aerospike/data/cache.dat
        filesize 1G
        data-in-memory false # Store data in memory in addition to file.
        write-block-size 128K
    }
}

namespace sessions {
    memory-size 1G
    #replication-factor 2
    default-ttl 30

    storage-engine device {
        file /opt/aerospike/data/sessions.dat
        filesize 1G
        data-in-memory false # Store data in memory in addition to file.
    }
}

namespace clicks {
    memory-size 1G
    default-ttl 30

    storage-engine device {
        file /opt/aerospike/data/clicks.dat
        filesize 1G
        data-in-memory false
    }
}

#2

Hi no1youknowz-

How many records do you need for the testing? Are you using secondary indexes? I can see in the .conf file that you have several namespaces on this node. Do you need all of the namespaces? Do all of the namespaces contain data? More to the point, do all namespaces need to hold data?

What is happening at the failure? Is the node running out of memory? Is it running out of drive space? What else can you tell me about the crash?

I’m sure you have already looked at this, but the following document describes sizing and memory allocation:

http://www.aerospike.com/docs/operations/plan/capacity/

Between knowing the content of the records, and the quantity of records, you should be able to determine how much memory the node needs.

Would you let me know the content of the records? Include the number of bins and the datatypes for each bin. We can make sure that the namespaces are sized properly.

If you are comfortable doing so, would you send the output of asmonitor -e info?

Thank you for your time, and I hope this helps,

-DM


#3

Hi Dave,

Thanks for the in-depth reply. Appreciate it.

I only need double digit records for testing. I’m not using any secondary indexes. Yes, I need all these namespaces for separating the data. I don’t understand what you mean by, do they need to hold data. Yes they do.

The node was running out of memory. But I figured out if I have memory-size 0G, it fails quickly. Now that I have memory-size 1G for all namespces and having the server overcommit memory…

Everything things good now.

Thanks

Paul