We’ve been using using both enterprise and community versions of aerospike (right now 3.7.4.1, this happened with previous versions)
Our cluster size has been 32 and 8 (different instance class) in AWS
Suddenly one nodes disk free will start free-falling until it gets to 0. Currently I erase the data and start it up again
Only thing I see in the logs is:
Mar 22 2016 04:22:11 GMT: WARNING (drv_ssd): (drv_ssd.c::4163) {user_profiles} write fail: queue too deep: q 68, max 64
Mar 22 2016 04:22:11 GMT: WARNING (drv_ssd): (drv_ssd.c::4163) {user_profiles} write fail: queue too deep: q 68, max 64
Mar 22 2016 04:22:11 GMT: WARNING (drv_ssd): (drv_ssd.c::4163) {user_profiles} write fail: queue too deep: q 67, max 64
Mar 22 2016 04:22:11 GMT: WARNING (drv_ssd): (drv_ssd.c::4163) {user_profiles} write fail: queue too deep: q 67, max 64
Any ideas on this?
Thanks