System giving input/output error for a configuration

Operations Configuration
1

I tried to put load-at-start true as per this doc: (see slide 51)

I changed the storage-engine to device.

When i restarted the server, it have the below error and after that it crashed.

What could be the reason here.

Deprecated parameter load-at-startup in cnf.c at line 73.

The system has become unresponsive

2

Thanks for posting on our forum. Which version of Aerospike are you using? As logged, this parameter is deprecated but it shouldn’t cause the server to crash. Could you provide server logs (the relevant part right during the startup sequence).

I just tried this on my local 3.4.1 community edition instance and had the warning but server came up fine:

[vagrant@localhost ~]$ sudo service aerospike restart
Stopping aerospike:                                        [  OK  ]
Starting and checking aerospike: Mar 02 2015 22:47:48 GMT: WARNING (config): (cfg.c::1167) line 74 :: load-at-startup is deprecated - please remove
                                                           [  OK  ]

Sincerely, –meher

3

Hello,

I am using 3.3.9. Server is hosted in AWS (centos). I am not able to login to the server now so am not able to access the logs.

Tried re-starting the server, shutdown etc…didn’t work.

4

Hi Meher,

We tried this on another instance with 20GB space.

PFA the logs and the aerospike.conf’s namespace screenshot.

PFA the error log.

https://gist.githubusercontent.com/pushpesh4u/97c0ed4601cfb1e119bb/raw/9bba024d53a2c453d202811185b6f212dd90ec01/Crash-dump

The aerospike log is below.

https://gist.githubusercontent.com/pushpesh4u/47e82bfad330c2d18770/raw/aerspike.log

Please let me know what could be the possible issue.

The /dev/xvde node also holds the operating system.

I would appreciate you giving me a sample .conf file for using storage-engine device. We will not use either test/bar namespace. Let me know if that would also cause any issue.

Thanks,

5

That is the root of both your problems - asd crashing and not being able to login to the machine.

When given a device, aerospike will use the entire disk. So essentially, your root OS gets corrupted when aerospike overwrites the OS files.

Please create either a partition, or use a separate disk.

6

Thanks @anshu for replying.

I will try this with another disk which does not host the OS and let you know if we come across any issues.

But, still why did it try to overwrite more than the space on the disk which caused corruption.

Can you suggest the size of disk which we should use. Also, please give a default conf setting. I don’t want to use the test/bar namespaces.

7

It is not the attempt to write more than space on the disk, but the write on the disk itself. Your configuration points to root device and Aerospike goes ahead and overwrites the data there, causing the OS layout to be corrupted …

Hope this clarifies things!

– R

8

This topic was automatically closed 84 days after the last reply. New replies are no longer allowed.