# SSD storage-engine for Aerospike in Docker on a VPS not working **URL:** \ **Category:** Containers (Docker, etc.)\ **Created:** [September 24, 2019, 4:29pm UTC](https://discuss.aerospike.com/t/ssd-storage-engine-for-aerospike-in-docker-on-a-vps-not-working/6663 "2019-09-24T16:29:10Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![Saif\_Ali\_Khan](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.aerospike.com/saif_ali_khan/32/1268_2.png) [@Saif\_Ali\_Khan](https://discuss.aerospike.com/u/Saif_Ali_Khan)\ **Post date:** [September 24, 2019, 4:29pm UTC](https://discuss.aerospike.com/t/ssd-storage-engine-for-aerospike-in-docker-on-a-vps-not-working/6663/1 "2019-09-24T16:29:10Z")
I am trying to run Aerospike in Docker with SSD as `storage-engine` device on a VPS. Following is my `aerospike.conf` file- ```auto service { user root group root paxos-single-replica-limit 1 pidfile /var/run/aerospike/asd.pid service-threads 4 transaction-queues 4 transaction-threads-per-queue 4 proto-fd-max 15000 } logging { file /var/log/aerospike/aerospike.log { context any warning } console { context any warning } } network { service { address eth0 port 3000 } heartbeat { address eth0 mode mesh port 3002 interval 150 timeout 10 } fabric { address eth0 port 3001 } info { port 3003 } } namespace test { replication-factor 1 memory-size 2G default-ttl 5d storage-engine device { device /dev/vda1 write-block-size 128K data-in-memory true } } ``` The VPS has SSD and is mounted on `/dev/vda1` as evident by the following- ```auto root@test:~# df -h Filesystem Size Used Avail Use% Mounted on udev 3.9G 0 3.9G 0% /dev tmpfs 798M 684K 797M 1% /run /dev/vda1 49G 11G 38G 22% / tmpfs 3.9G 0 3.9G 0% /dev/shm tmpfs 5.0M 0 5.0M 0% /run/lock tmpfs 3.9G 0 3.9G 0% /sys/fs/cgroup /dev/vda15 105M 3.6M 101M 4% /boot/efi tmpfs 798M 0 798M 0% /run/user/0 ``` However on deploying I get the following error- ```auto link eth0 state up link eth0 state up in 0 FAILED ASSERTION (drv_ssd): (drv_ssd.c:3194) unable to open device /dev/vda1: No such file or directory WARNING (as): (signal.c:213) SIGUSR1 received, aborting Aerospike Community Edition build 4.6.0.2 os debian9 WARNING (as): (signal.c:215) stacktrace: registers: rax 0000000000000000 rbx 000000000000000a rcx 00007f193590bfbf rdx 0000000000000000 rsi 00007ffd023a55f0 rdi 0000000000000002 rbp 00007ffd023a5c20 rsp 00007ffd023a5668 r8 0000000000000000 r9 00007ffd023a55f0 r10 0000000000000008 r11 0000000000000246 r12 0000000000000011 r13 0000000000000000 r14 0000000000000080 r15 000055c2d04d3100 rip 00007f193590bfbf WARNING (as): (signal.c:215) stacktrace: found 10 frames: 0x55c2ce7ceee7 0x7f193590c0e0 0x7f193590bfbf 0x55c2ce87546e 0x55c2ce85032c 0x55c2ce850cbb 0x55c2ce8535fd 0x55c2ce762d69 0x7f19346b92e1 0x55c2ce76374a offset 0x55c2ce713000 WARNING (as): (signal.c:215) stacktrace: frame 0: asd(as_sig_handle_usr1+0x11f) [0x55c2ce7ceee7] WARNING (as): (signal.c:215) stacktrace: frame 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x110e0) [0x7f193590c0e0] WARNING (as): (signal.c:215) stacktrace: frame 2: /lib/x86_64-linux-gnu/libpthread.so.0(raise+0xcf) [0x7f193590bfbf] WARNING (as): (signal.c:215) stacktrace: frame 3: asd(cf_fault_event+0x231) [0x55c2ce87546e] WARNING (as): (signal.c:215) stacktrace: frame 4: asd(ssd_init_devices+0x410) [0x55c2ce85032c] WARNING (as): (signal.c:215) stacktrace: frame 5: asd(as_storage_namespace_init_ssd+0x2c) [0x55c2ce850cbb] WARNING (as): (signal.c:215) stacktrace: frame 6: asd(as_storage_init+0x34) [0x55c2ce8535fd] WARNING (as): (signal.c:215) stacktrace: frame 7: asd(main+0x309) [0x55c2ce762d69] WARNING (as): (signal.c:215) stacktrace: frame 8: /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf1) [0x7f19346b92e1] WARNING (as): (signal.c:215) stacktrace: frame 9: asd(_start+0x2a) [0x55c2ce76374a] FAILED ASSERTION (drv_ssd): (drv_ssd.c:3140) usable device size 0 must be greater than header size 8388608 ``` I noticed that there is no `/dev/vda1` inside the Aerospike container. So to sum it up, how to use SSD storage-engine device for Aerospike in Docker on a VPS? Reference- - [Recipe for an SSD Storage Engine](https://www.aerospike.com/docs/operations/configure/namespace/storage/#recipe-for-an-ssd-storage-engine) ---
**Author:** ![kporter](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.aerospike.com/kporter/32/515_2.png) [@kporter](https://discuss.aerospike.com/u/kporter)\ **Post date:** [September 24, 2019, 5:09pm UTC](https://discuss.aerospike.com/t/ssd-storage-engine-for-aerospike-in-docker-on-a-vps-not-working/6663/2 "2019-09-24T17:09:50Z")
> [@Saif\_Ali\_Khan](#): > > /dev/vda1 49G 11G 38G 22% / You probably don’t want to use vda1 since it is mounted as root (‘/’). Using vda1 would cause Aerospike to overwrite your root device. When using `device` the underlying device shouldn’t be mounted and Aerospike should be the only process writing to it. To attach a raw device to a docker container, you need to use one of the [`--device`](https://docs.docker.com/engine/reference/commandline/run/) options. ---
**Author:** ![Saif\_Ali\_Khan](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.aerospike.com/saif_ali_khan/32/1268_2.png) [@Saif\_Ali\_Khan](https://discuss.aerospike.com/u/Saif_Ali_Khan)\ **Post date:** [September 24, 2019, 5:17pm UTC](https://discuss.aerospike.com/t/ssd-storage-engine-for-aerospike-in-docker-on-a-vps-not-working/6663/3 "2019-09-24T17:17:25Z")
I cannot have a separate raw SSD device to attach to Aerospike. Instead I want to use the SSD that comes with the VPS, but as you mentioned that it is not possible unless I use a SSD dedicated to Aerospike. The only option I see is to use filesystem in `storage-engine device` and mount it as a docker volume. Am I correct? ---
**Author:** ![kporter](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.aerospike.com/kporter/32/515_2.png) [@kporter](https://discuss.aerospike.com/u/kporter)\ **Post date:** [September 24, 2019, 7:35pm UTC](https://discuss.aerospike.com/t/ssd-storage-engine-for-aerospike-in-docker-on-a-vps-not-working/6663/4 "2019-09-24T19:35:44Z")
Yes, volumes are the typical choice. Just in case it isn’t clear, you will need to use the [`file`](https://www.aerospike.com/docs/reference/configuration/index.html#file) configuration rather than the [`device`](https://www.aerospike.com/docs/reference/configuration/index.html#device) configuration. ---
**Author:** ![Saif\_Ali\_Khan](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.aerospike.com/saif_ali_khan/32/1268_2.png) [@Saif\_Ali\_Khan](https://discuss.aerospike.com/u/Saif_Ali_Khan)\ **Post date:** [September 25, 2019, 9:44am UTC](https://discuss.aerospike.com/t/ssd-storage-engine-for-aerospike-in-docker-on-a-vps-not-working/6663/5 "2019-09-25T09:44:53Z")
Thank you for clarification- ```auto storage-engine device { file /opt/aerospike/data/test.dat filesize 4G write-block-size 128K data-in-memory true } ``` One last question, documentation says that for better performance on SSD `write-block-size` should be `128K`, considering that I’m on a VPS with SSD storage and using `file` configuration should I use `write-block-size 128K`? I want to avoid OOM problem. Also with the above file config I am getting the following log warning every 5 secs. ```auto WARNING (hardware): (hardware.c:2262) failed to resolve mounted device /dev/vda1: 2 (No such file or directory) ``` I read this ([High memory usage on Kubernetes GKE using helm chart](https://discuss.aerospike.com/t/high-memory-usage-on-kubernetes-gke-using-helm-chart/6052/10)) discussion but couldn’t find a solution. ---
**Author:** ![kporter](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.aerospike.com/kporter/32/515_2.png) [@kporter](https://discuss.aerospike.com/u/kporter)\ **Post date:** [September 25, 2019, 6:53pm UTC](https://discuss.aerospike.com/t/ssd-storage-engine-for-aerospike-in-docker-on-a-vps-not-working/6663/6 "2019-09-25T18:53:11Z")
> [@Saif\_Ali\_Khan](#): > > One last question, documentation says that for better performance on SSD `write-block-size` should be `128K` , considering that I’m on a VPS with SSD storage and using `file` configuration should I use `write-block-size 128K` ? I want to avoid OOM problem. The [`post-write-queue`](https://www.aerospike.com/docs/reference/configuration/#post-write-queue) is a cache that holds recently written write-blocks. If you increase/decrease the size of the `write-block-size` you will increase/decrease the amount of RAM used by the `post-write-queue`. So it is possible to accidentally allocate all of your RAM to the post-write-queue if you aren’t careful. ---
**Author:** ![kporter](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.aerospike.com/kporter/32/515_2.png) [@kporter](https://discuss.aerospike.com/u/kporter)\ **Post date:** [September 25, 2019, 7:04pm UTC](https://discuss.aerospike.com/t/ssd-storage-engine-for-aerospike-in-docker-on-a-vps-not-working/6663/7 "2019-09-25T19:04:32Z")
> [@Saif\_Ali\_Khan](#): > > WARNING (hardware): (hardware.c:2262) failed to resolve mounted device /dev/vda1: 2 (No such file or directory) Seems this is appearing several times today, please read this response: [Issues with restarting an instance? - #6 by kporter](https://discuss.aerospike.com/t/issues-with-restarting-an-instance/6665/6) ---
**Author:** ![Saif\_Ali\_Khan](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.aerospike.com/saif_ali_khan/32/1268_2.png) [@Saif\_Ali\_Khan](https://discuss.aerospike.com/u/Saif_Ali_Khan)\ **Post date:** [September 25, 2019, 7:17pm UTC](https://discuss.aerospike.com/t/ssd-storage-engine-for-aerospike-in-docker-on-a-vps-not-working/6663/8 "2019-09-25T19:17:43Z")
Thank you for your help so far. After this issue getting solved I’ve started running into OOM crash. [Aerospike RAM consumption keeps increasing and crashes (OOM)](https://discuss.aerospike.com/t/aerospike-ram-consumption-keeps-increasing-and-crashes-oom/6676) ---
**Author:** ![tlo](https://avatars.discourse-cdn.com/v4/letter/t/4da419/32.png) [@tlo](https://discuss.aerospike.com/u/tlo)\ **Post date:** [September 26, 2019, 2:47pm UTC](https://discuss.aerospike.com/t/ssd-storage-engine-for-aerospike-in-docker-on-a-vps-not-working/6663/9 "2019-09-26T14:47:14Z")
Saif, could you run the following two commands inside the container and share their output? - `cat /proc/mounts` - `ls -l /dev` It does seem like `/proc/mounts` properly indicates that `/opt/aerospike/data` resides on `/dev/vda1` . But it also seems like `/dev` does not contain a device node for `vda1` . Hence I’d like to double-check the contents of `/proc/mounts` as well as what’s in `/dev` inside the container. ---
**Author:** ![Saif\_Ali\_Khan](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.aerospike.com/saif_ali_khan/32/1268_2.png) [@Saif\_Ali\_Khan](https://discuss.aerospike.com/u/Saif_Ali_Khan)\ **Post date:** [September 26, 2019, 3:29pm UTC](https://discuss.aerospike.com/t/ssd-storage-engine-for-aerospike-in-docker-on-a-vps-not-working/6663/10 "2019-09-26T15:29:17Z")
```auto $ cat /proc/mounts overlay / overlay rw,relatime,lowerdir=/var/lib/docker/overlay2/l/5GTHODYWF2QVOXZK4KIOM6BHAY:/var/lib/docker/overlay2/l/AB5AX524OTOUJZTTQ7PNFB6S4D:/var/lib/docker/overlay2/l/HK42LK4Y33U76QC47I7XJP56OT:/var/lib/docker/overlay2/l/SZXT6L4XGJ5PYEVGBLX7JPQ2UF:/var/lib/docker/overlay2/l/O2QKKT5J2PUQBKBHCGFVKNJ6JH,upperdir=/var/lib/docker/overlay2/1b509e29f9a2a204466179b34059627361cd49d1e744900eb7f4b86177e12fac/diff,workdir=/var/lib/docker/overlay2/1b509e29f9a2a204466179b34059627361cd49d1e744900eb7f4b86177e12fac/work 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev tmpfs rw,nosuid,size=65536k,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=666 0 0 sysfs /sys sysfs ro,nosuid,nodev,noexec,relatime 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,relatime,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup ro,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 cgroup /sys/fs/cgroup/cpuset cgroup ro,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup ro,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup ro,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup ro,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup ro,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup ro,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup ro,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup ro,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup ro,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup ro,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/devices cgroup ro,nosuid,nodev,noexec,relatime,devices 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 shm /dev/shm tmpfs rw,nosuid,nodev,noexec,relatime,size=65536k 0 0 /dev/vda1 /etc/resolv.conf ext4 rw,relatime,data=ordered 0 0 /dev/vda1 /etc/hostname ext4 rw,relatime,data=ordered 0 0 /dev/vda1 /etc/hosts ext4 rw,relatime,data=ordered 0 0 /dev/vda1 /opt/aerospike/data ext4 rw,relatime,data=ordered 0 0 tmpfs /opt/aerospike/etc/aerospike.conf tmpfs ro,relatime 0 0 proc /proc/bus proc ro,relatime 0 0 proc /proc/fs proc ro,relatime 0 0 proc /proc/irq proc ro,relatime 0 0 proc /proc/sys proc ro,relatime 0 0 proc /proc/sysrq-trigger proc ro,relatime 0 0 tmpfs /proc/acpi tmpfs ro,relatime 0 0 tmpfs /proc/kcore tmpfs rw,nosuid,size=65536k,mode=755 0 0 tmpfs /proc/keys tmpfs rw,nosuid,size=65536k,mode=755 0 0 tmpfs /proc/timer_list tmpfs rw,nosuid,size=65536k,mode=755 0 0 tmpfs /proc/sched_debug tmpfs rw,nosuid,size=65536k,mode=755 0 0 tmpfs /proc/scsi tmpfs ro,relatime 0 0 tmpfs /sys/firmware tmpfs ro,relatime 0 0 ``` ```auto $ ls -l /dev total 0 lrwxrwxrwx 1 root root 11 Sep 26 12:08 core -> /proc/kcore lrwxrwxrwx 1 root root 13 Sep 26 12:08 fd -> /proc/self/fd crw-rw-rw- 1 root root 1, 7 Sep 26 12:08 full drwxrwxrwt 2 root root 40 Sep 26 12:08 mqueue crw-rw-rw- 1 root root 1, 3 Sep 26 12:08 null lrwxrwxrwx 1 root root 8 Sep 26 12:08 ptmx -> pts/ptmx drwxr-xr-x 2 root root 0 Sep 26 12:08 pts crw-rw-rw- 1 root root 1, 8 Sep 26 12:08 random drwxrwxrwt 2 root root 40 Sep 26 12:08 shm lrwxrwxrwx 1 root root 15 Sep 26 12:08 stderr -> /proc/self/fd/2 lrwxrwxrwx 1 root root 15 Sep 26 12:08 stdin -> /proc/self/fd/0 lrwxrwxrwx 1 root root 15 Sep 26 12:08 stdout -> /proc/self/fd/1 crw-rw-rw- 1 root root 5, 0 Sep 26 12:08 tty crw-rw-rw- 1 root root 1, 9 Sep 26 12:08 urandom crw-rw-rw- 1 root root 1, 5 Sep 26 12:08 zero ``` ---
**Author:** ![tlo](https://avatars.discourse-cdn.com/v4/letter/t/4da419/32.png) [@tlo](https://discuss.aerospike.com/u/tlo)\ **Post date:** [September 26, 2019, 4:56pm UTC](https://discuss.aerospike.com/t/ssd-storage-engine-for-aerospike-in-docker-on-a-vps-not-working/6663/11 "2019-09-26T16:56:50Z")
Thanks for your quick response, Saif! Yes, as suspected, the Docker container says that `/dev/vda1` is mounted at `/opt/aerospike/data`. (And also, strangely, at `resolv.conf`, `hostname`, and `hosts`.) See the following excerpt from `/proc/mounts`. ```auto /dev/vda1 /etc/resolv.conf ext4 rw,relatime,data=ordered 0 0 /dev/vda1 /etc/hostname ext4 rw,relatime,data=ordered 0 0 /dev/vda1 /etc/hosts ext4 rw,relatime,data=ordered 0 0 /dev/vda1 /opt/aerospike/data ext4 rw,relatime,data=ordered 0 0 ``` Let’s leave `resolv.conf`, `hostname`, and `hosts` aside for now - I’ll have to research this a little more. It seems unusual to see a single device (a) mounted multiple times and (b) as a file instead of a directory. So, let’s go back to the warning you’re seeing. As `/proc/mounts` says that it’s `/dev/vda1` that’s mounted at `/opt/aerospike/data`, the Aerospike server tries to take a look at `/dev/vda1`. But `vda1` does not exist in your `/dev` directory. So, Aerospike cannot find `/dev/vda1` - hence the warning. The warning is likely triggered periodically, because something periodically queries the cluster node’s SSD health information, which is part of the cluster node’s namespace stats. So, if you ran `asinfo -v namespace/foobar` against this node, then this would trigger the warning. It’s safe to just ignore this warning for now. You won’t get any SSD health information, but this only works for physical NVMe drives anyway. So, you aren’t losing any functionality in your configuration. My understanding of Docker’s `/dev` management is very limited, so I’ll have to figure out how to best avoid this warning. For now, however, my recommendation would be to ignore it. ---
**Author:** ![system](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.aerospike.com/system/32/2274_2.png) [@system](https://discuss.aerospike.com/u/system)\ **Post date:** [October 2, 2019, 4:56pm UTC](https://discuss.aerospike.com/t/ssd-storage-engine-for-aerospike-in-docker-on-a-vps-not-working/6663/12 "2019-10-02T16:56:58Z")
Closed on October 2, 2019, 4:56pm UTC This topic was automatically closed 6 days after the last reply. New replies are no longer allowed.