In my JRuby application, I am getting following error periodically,
Aerospike::Exceptions::Connection: Resource temporarily unavailable - errno backtraces disabled; run with -Xerrno.backtrace=true to enable
with following configuration (3 nodes),
network {
service {
address any
port 3000
}
heartbeat {
mode mesh
address 10.240.0.53
port 3002
# To use unicast-mesh heartbeats, remove the 3 lines above, and see
# aerospike_mesh.conf for alternative.
mesh-seed-address-port 10.240.0.53 3002
mesh-seed-address-port 10.240.0.34 3002
mesh-seed-address-port 10.240.0.48 3002
interval 150
timeout 10
}
fabric {
port 3001
}
info {
port 3003
}
}
In addition to this, I have also added timeouts for read and write operation(250ms). Is there any particular reason behind this error?
Based on the error message (“Resource temporarily unavailable”) this is probably a form of EAGAIN error where the client is trying to read from or write to a TCP socket asynchronously but fails to do so. Can you please enable backtraces (i.e. jruby -Xerrno.backtrace=true ...) so we can see exactly where this exception is happening?
What version of the client are you using?
What version of JRuby are you using?
/GEM_ROOT/gems/aerospike-2.7.0/lib/aerospike/socket/base.rb:44 in read_from_socket
/GEM_ROOT/gems/aerospike-2.7.0/lib/aerospike/socket/base.rb:31 in read
/GEM_ROOT/gems/aerospike-2.7.0/lib/aerospike/command/write_command.rb:47 in parse_result
/GEM_ROOT/gems/aerospike-2.7.0/lib/aerospike/command/command.rb:403 in execute
/GEM_ROOT/gems/aerospike-2.7.0/lib/aerospike/client.rb:857 in execute_command
/GEM_ROOT/gems/aerospike-2.7.0/lib/aerospike/client.rb:117 in put```
Does this happen all the time? How much load is the server under? Have you checked your bottlenecks (IO/Network/CPU)? Anything in the aerospike log? What does your topology look like - server/client specs, hops between nodes/server<->client? Any latency in the histograms?