Help to configure Aerospike for lowest CPU consumption (for develop environment)

benbor · April 2, 2021, 10:57am

Hi guys. I have ~3-5% CPU usage by Aerospike (AS), while there are no interaction with AS. For example, MySql consumes < 0.1% in the same situation.

That CPU consumption warms my laptop, so it is uncomfortable to work:) Can you help me? I would be pleasure for some configuration tips or links (if such exists)

The config for my develop env (vagrant +VBox + Ubuntu based) and log sample in “detailed” mode for 10 sec are attached.

Thanks!

Configuration

service {
        user aerospike
        group aerospike
        pidfile /var/run/aerospike/asd.pid
        paxos-single-replica-limit 1 
        service-threads 2
        transaction-queues 2
        transaction-threads-per-queue 2
        proto-fd-max 15000
        proto-fd-idle-ms 60000
}

logging {
        # Log file must be an absolute path.
        file /var/log/aerospike/aerospike.log {
                context any info
        }
}

network {
        service {
                address any
                port 3000
        }

        heartbeat {
                mode mesh
                port 3002 # Heartbeat port for this node.
                 # List one or more other nodes, one ip-address & port per line:
                address any
                mesh-seed-address-port 127.0.0.1 3002
                interval 10000
                timeout 10
        }

        fabric {
                port 3001
        }

        info {
                port 3003
        }
}

namespace COMMON {
        replication-factor 2
        memory-size 128M
        high-water-memory-pct 60
        default-ttl 4d # use 0 to never expire/evict.
        storage-engine memory
}
namespace CACHE {
        replication-factor 2
        memory-size 128M
        high-water-memory-pct 60
        default-ttl 4d # use 0 to never expire/evict.
        storage-engine memory
}service {
        user aerospike
        group aerospike
        pidfile /var/run/aerospike/asd.pid
        paxos-single-replica-limit 1 # Number of nodes where the replica count is automatically reduced to 1.
        service-threads 2
        transaction-queues 2
        transaction-threads-per-queue 2
        proto-fd-max 15000
        proto-fd-idle-ms 60000
}

logging {
        # Log file must be an absolute path.
        file /var/log/aerospike/aerospike.log {
                context any info
        }
}

network {
        service {
                address any
                port 3000
        }

        heartbeat {
                mode mesh
                port 3002 # Heartbeat port for this node.
                 # List one or more other nodes, one ip-address & port per line:
                address any
                mesh-seed-address-port 127.0.0.1 3002
                interval 10000
                timeout 10
        }

        fabric {
                port 3001
        }

        info {
                port 3003
        }
}

namespace COMMON {
        replication-factor 2
        memory-size 128M
        high-water-memory-pct 60
        default-ttl 4d # use 0 to never expire/evict.
        storage-engine memory
}
namespace CACHE {
        replication-factor 2
        memory-size 128M
        high-water-memory-pct 60
        default-ttl 4d # use 0 to never expire/evict.
        storage-engine memory
}

Log example only for several seconds (because of a lot of characters) :

The log

Apr 02 2021 10:23:30 GMT: DETAIL (service-list): (service_list.c:376) handling info value peers-generation
Apr 02 2021 10:23:30 GMT: DETAIL (service-list): (service_list.c:1248) info val peers-generation -> 0
Apr 02 2021 10:23:30 GMT: DETAIL (service-list): (service_list.c:376) handling info value peers-generation
Apr 02 2021 10:23:30 GMT: DETAIL (socket): (socket.c:1275) Blocking send on FD 29, size = 71
Apr 02 2021 10:23:30 GMT: DETAIL (service-list): (service_list.c:1248) info val peers-generation -> 0
Apr 02 2021 10:23:30 GMT: DETAIL (socket): (socket.c:1275) Blocking send on FD 31, size = 71
Apr 02 2021 10:23:30 GMT: DETAIL (socket): (socket.c:1306) Blocking send on FD 29 complete
Apr 02 2021 10:23:30 GMT: DETAIL (service-list): (service_list.c:376) handling info value peers-generation
Apr 02 2021 10:23:30 GMT: DETAIL (service-list): (service_list.c:1248) info val peers-generation -> 0
Apr 02 2021 10:23:30 GMT: DETAIL (service-list): (service_list.c:376) handling info value peers-generation
Apr 02 2021 10:23:30 GMT: DETAIL (socket): (socket.c:1275) Blocking send on FD 32, size = 71
Apr 02 2021 10:23:30 GMT: DETAIL (service-list): (service_list.c:1248) info val peers-generation -> 0
Apr 02 2021 10:23:30 GMT: DETAIL (socket): (socket.c:1275) Blocking send on FD 30, size = 71
Apr 02 2021 10:23:30 GMT: DETAIL (socket): (socket.c:1306) Blocking send on FD 30 complete
Apr 02 2021 10:23:30 GMT: DETAIL (service-list): (service_list.c:376) handling info value peers-generation
Apr 02 2021 10:23:30 GMT: DETAIL (service-list): (service_list.c:1248) info val peers-generation -> 0
Apr 02 2021 10:23:30 GMT: DETAIL (socket): (socket.c:1275) Blocking send on FD 28, size = 71
Apr 02 2021 10:23:30 GMT: DETAIL (socket): (socket.c:1306) Blocking send on FD 31 complete
Apr 02 2021 10:23:30 GMT: DETAIL (socket): (socket.c:1306) Blocking send on FD 32 complete
Apr 02 2021 10:23:30 GMT: DETAIL (socket): (socket.c:1306) Blocking send on FD 28 complete
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node arrived event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node departed event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:principal departed event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:peer adjacency changed event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:join request accepted event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:merge candidate seen event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:member orphaned event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2624) Last Quantum interval:61426814
Apr 02 2021 10:23:30 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:30 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:30 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:30 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node arrived event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node departed event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:principal departed event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:peer adjacency changed event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:join request accepted event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:merge candidate seen event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:member orphaned event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2624) Last Quantum interval:61426814
Apr 02 2021 10:23:30 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:30 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node arrived event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node departed event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:principal departed event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:peer adjacency changed event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:join request accepted event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:merge candidate seen event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:member orphaned event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2624) Last Quantum interval:61426814
Apr 02 2021 10:23:30 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:30 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:30 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:30 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node arrived event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node departed event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:principal departed event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:peer adjacency changed event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:join request accepted event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:merge candidate seen event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:member orphaned event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2624) Last Quantum interval:61426814
Apr 02 2021 10:23:30 GMT: DETAIL (smd): (system_metadata.c:3428) System Metadata thread - received timeout event
Apr 02 2021 10:23:30 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:30 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node arrived event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node departed event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:principal departed event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:peer adjacency changed event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:join request accepted event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:merge candidate seen event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:member orphaned event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2624) Last Quantum interval:61426814
Apr 02 2021 10:23:30 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:30 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node arrived event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node departed event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:principal departed event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:peer adjacency changed event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:join request accepted event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:merge candidate seen event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:member orphaned event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2624) Last Quantum interval:61426814
Apr 02 2021 10:23:30 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:30 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:30 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:30 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node arrived event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node departed event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:principal departed event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:peer adjacency changed event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:join request accepted event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:merge candidate seen event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:member orphaned event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2624) Last Quantum interval:61426814
Apr 02 2021 10:23:30 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:30 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node arrived event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node departed event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:principal departed event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:peer adjacency changed event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:join request accepted event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:merge candidate seen event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:member orphaned event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2624) Last Quantum interval:61426814
Apr 02 2021 10:23:30 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:30 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:30 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:30 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node arrived event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node departed event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:principal departed event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:peer adjacency changed event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:join request accepted event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:merge candidate seen event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:member orphaned event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2624) Last Quantum interval:61426814
Apr 02 2021 10:23:30 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:30 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node arrived event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node departed event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:principal departed event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:peer adjacency changed event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:join request accepted event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:merge candidate seen event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:member orphaned event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2624) Last Quantum interval:61426814
Apr 02 2021 10:23:30 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:30 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:30 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:30 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node arrived event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node departed event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:principal departed event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:peer adjacency changed event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:join request accepted event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:merge candidate seen event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:member orphaned event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2624) Last Quantum interval:61426814
Apr 02 2021 10:23:30 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:30 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node arrived event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node departed event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:principal departed event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:peer adjacency changed event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:join request accepted event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:merge candidate seen event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:member orphaned event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2624) Last Quantum interval:61426814
Apr 02 2021 10:23:30 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:30 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:30 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:30 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node arrived event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node departed event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:principal departed event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:peer adjacency changed event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:join request accepted event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:merge candidate seen event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2620) Fault:member orphaned event_ts:0
Apr 02 2021 10:23:30 GMT: DETAIL (clustering): (clustering.c:2624) Last Quantum interval:61426814
Apr 02 2021 10:23:30 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:30 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node arrived event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node departed event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:principal departed event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:peer adjacency changed event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:join request accepted event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:merge candidate seen event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:member orphaned event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2624) Last Quantum interval:61426814
Apr 02 2021 10:23:31 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:31 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:31 GMT: DETAIL (service-list): (service_list.c:376) handling info value peers-generation
Apr 02 2021 10:23:31 GMT: DETAIL (service-list): (service_list.c:1248) info val peers-generation -> 0
Apr 02 2021 10:23:31 GMT: DETAIL (service-list): (service_list.c:376) handling info value peers-generation
Apr 02 2021 10:23:31 GMT: DETAIL (socket): (socket.c:1275) Blocking send on FD 32, size = 71
Apr 02 2021 10:23:31 GMT: DETAIL (service-list): (service_list.c:1248) info val peers-generation -> 0
Apr 02 2021 10:23:31 GMT: DETAIL (socket): (socket.c:1275) Blocking send on FD 30, size = 71
Apr 02 2021 10:23:31 GMT: DETAIL (service-list): (service_list.c:376) handling info value peers-generation
Apr 02 2021 10:23:31 GMT: DETAIL (service-list): (service_list.c:1248) info val peers-generation -> 0
Apr 02 2021 10:23:31 GMT: DETAIL (socket): (socket.c:1275) Blocking send on FD 28, size = 71
Apr 02 2021 10:23:31 GMT: DETAIL (service-list): (service_list.c:376) handling info value peers-generation
Apr 02 2021 10:23:31 GMT: DETAIL (service-list): (service_list.c:1248) info val peers-generation -> 0
Apr 02 2021 10:23:31 GMT: DETAIL (socket): (socket.c:1306) Blocking send on FD 32 complete
Apr 02 2021 10:23:31 GMT: DETAIL (socket): (socket.c:1275) Blocking send on FD 31, size = 71
Apr 02 2021 10:23:31 GMT: DETAIL (service-list): (service_list.c:376) handling info value peers-generation
Apr 02 2021 10:23:31 GMT: DETAIL (service-list): (service_list.c:1248) info val peers-generation -> 0
Apr 02 2021 10:23:31 GMT: DETAIL (socket): (socket.c:1275) Blocking send on FD 29, size = 71
Apr 02 2021 10:23:31 GMT: DETAIL (socket): (socket.c:1306) Blocking send on FD 28 complete
Apr 02 2021 10:23:31 GMT: DETAIL (socket): (socket.c:1306) Blocking send on FD 29 complete
Apr 02 2021 10:23:31 GMT: DETAIL (socket): (socket.c:1306) Blocking send on FD 31 complete
Apr 02 2021 10:23:31 GMT: DETAIL (socket): (socket.c:1306) Blocking send on FD 30 complete
Apr 02 2021 10:23:31 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:31 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node arrived event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node departed event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:principal departed event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:peer adjacency changed event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:join request accepted event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:merge candidate seen event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:member orphaned event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2624) Last Quantum interval:61426814
Apr 02 2021 10:23:31 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:31 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node arrived event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node departed event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:principal departed event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:peer adjacency changed event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:join request accepted event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:merge candidate seen event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:member orphaned event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2624) Last Quantum interval:61426814
Apr 02 2021 10:23:31 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:31 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node arrived event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node departed event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:principal departed event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:peer adjacency changed event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:join request accepted event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:merge candidate seen event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:member orphaned event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2624) Last Quantum interval:61426814
Apr 02 2021 10:23:31 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:31 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:31 GMT: DETAIL (smd): (system_metadata.c:3428) System Metadata thread - received timeout event
Apr 02 2021 10:23:31 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:31 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node arrived event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node departed event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:principal departed event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:peer adjacency changed event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:join request accepted event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:merge candidate seen event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:member orphaned event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2624) Last Quantum interval:61426814
Apr 02 2021 10:23:31 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:31 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node arrived event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node departed event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:principal departed event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:peer adjacency changed event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:join request accepted event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:merge candidate seen event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:member orphaned event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2624) Last Quantum interval:61426814
Apr 02 2021 10:23:31 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:31 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:31 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:31 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node arrived event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node departed event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:principal departed event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:peer adjacency changed event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:join request accepted event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:merge candidate seen event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:member orphaned event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2624) Last Quantum interval:61426814
Apr 02 2021 10:23:31 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:31 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node arrived event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node departed event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:principal departed event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:peer adjacency changed event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:join request accepted event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:merge candidate seen event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:member orphaned event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2624) Last Quantum interval:61426814
Apr 02 2021 10:23:31 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:31 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node arrived event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node departed event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:principal departed event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:peer adjacency changed event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:join request accepted event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:merge candidate seen event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:member orphaned event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2624) Last Quantum interval:61426814
Apr 02 2021 10:23:31 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:31 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:31 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:31 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node arrived event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node departed event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:principal departed event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:peer adjacency changed event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:join request accepted event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:merge candidate seen event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:member orphaned event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2624) Last Quantum interval:61426814
Apr 02 2021 10:23:31 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:31 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node arrived event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node departed event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:principal departed event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:peer adjacency changed event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:join request accepted event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:merge candidate seen event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:member orphaned event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2624) Last Quantum interval:61426814
Apr 02 2021 10:23:31 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:31 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:31 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:31 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node arrived event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node departed event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:principal departed event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:peer adjacency changed event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:join request accepted event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:merge candidate seen event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:member orphaned event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2624) Last Quantum interval:61426814
Apr 02 2021 10:23:31 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:31 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node arrived event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node departed event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:principal departed event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:peer adjacency changed event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:join request accepted event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:merge candidate seen event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:member orphaned event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2624) Last Quantum interval:61426814
Apr 02 2021 10:23:31 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:31 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:31 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:31 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node arrived event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node departed event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:principal departed event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:peer adjacency changed event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:join request accepted event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:merge candidate seen event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2620) Fault:member orphaned event_ts:0
Apr 02 2021 10:23:31 GMT: DETAIL (clustering): (clustering.c:2624) Last Quantum interval:61426814
Apr 02 2021 10:23:32 GMT: DETAIL (service-list): (service_list.c:376) handling info value peers-generation
Apr 02 2021 10:23:32 GMT: DETAIL (service-list): (service_list.c:1248) info val peers-generation -> 0
Apr 02 2021 10:23:32 GMT: DETAIL (socket): (socket.c:1275) Blocking send on FD 30, size = 71
Apr 02 2021 10:23:32 GMT: DETAIL (service-list): (service_list.c:376) handling info value peers-generation
Apr 02 2021 10:23:32 GMT: DETAIL (service-list): (service_list.c:376) handling info value peers-generation
Apr 02 2021 10:23:32 GMT: DETAIL (service-list): (service_list.c:1248) info val peers-generation -> 0
Apr 02 2021 10:23:32 GMT: DETAIL (service-list): (service_list.c:1248) info val peers-generation -> 0
Apr 02 2021 10:23:32 GMT: DETAIL (socket): (socket.c:1275) Blocking send on FD 31, size = 71
Apr 02 2021 10:23:32 GMT: DETAIL (socket): (socket.c:1275) Blocking send on FD 29, size = 71
Apr 02 2021 10:23:32 GMT: DETAIL (socket): (socket.c:1306) Blocking send on FD 30 complete
Apr 02 2021 10:23:32 GMT: DETAIL (socket): (socket.c:1306) Blocking send on FD 31 complete
Apr 02 2021 10:23:32 GMT: DETAIL (socket): (socket.c:1306) Blocking send on FD 29 complete
Apr 02 2021 10:23:32 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:32 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:32 GMT: DETAIL (service-list): (service_list.c:376) handling info value peers-generation
Apr 02 2021 10:23:32 GMT: DETAIL (service-list): (service_list.c:1248) info val peers-generation -> 0
Apr 02 2021 10:23:32 GMT: DETAIL (socket): (socket.c:1275) Blocking send on FD 32, size = 71
Apr 02 2021 10:23:32 GMT: DETAIL (service-list): (service_list.c:376) handling info value peers-generation
Apr 02 2021 10:23:32 GMT: DETAIL (service-list): (service_list.c:1248) info val peers-generation -> 0
Apr 02 2021 10:23:32 GMT: DETAIL (socket): (socket.c:1275) Blocking send on FD 28, size = 71
Apr 02 2021 10:23:32 GMT: DETAIL (socket): (socket.c:1306) Blocking send on FD 28 complete
Apr 02 2021 10:23:32 GMT: DETAIL (socket): (socket.c:1306) Blocking send on FD 32 complete
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node arrived event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node departed event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:principal departed event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:peer adjacency changed event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:join request accepted event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:merge candidate seen event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:member orphaned event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2624) Last Quantum interval:61426814
Apr 02 2021 10:23:32 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:32 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:32 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:32 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node arrived event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node departed event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:principal departed event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:peer adjacency changed event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:join request accepted event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:merge candidate seen event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:member orphaned event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2624) Last Quantum interval:61426814
Apr 02 2021 10:23:32 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:32 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node arrived event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node departed event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:principal departed event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:peer adjacency changed event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:join request accepted event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:merge candidate seen event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:member orphaned event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2624) Last Quantum interval:61426814
Apr 02 2021 10:23:32 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:32 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:32 GMT: DETAIL (smd): (system_metadata.c:3428) System Metadata thread - received timeout event
Apr 02 2021 10:23:32 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:32 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node arrived event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node departed event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:principal departed event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:peer adjacency changed event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:join request accepted event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:merge candidate seen event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:member orphaned event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2624) Last Quantum interval:61426814
Apr 02 2021 10:23:32 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:32 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node arrived event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node departed event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:principal departed event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:peer adjacency changed event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:join request accepted event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:merge candidate seen event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:member orphaned event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2624) Last Quantum interval:61426814
Apr 02 2021 10:23:32 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:32 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node arrived event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node departed event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:principal departed event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:peer adjacency changed event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:join request accepted event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:merge candidate seen event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:member orphaned event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2624) Last Quantum interval:61426814
Apr 02 2021 10:23:32 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:32 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:32 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:32 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node arrived event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node departed event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:principal departed event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:peer adjacency changed event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:join request accepted event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:merge candidate seen event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:member orphaned event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2624) Last Quantum interval:61426814
Apr 02 2021 10:23:32 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:32 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node arrived event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node departed event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:principal departed event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:peer adjacency changed event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:join request accepted event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:merge candidate seen event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:member orphaned event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2624) Last Quantum interval:61426814
Apr 02 2021 10:23:32 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:32 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:32 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:32 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node arrived event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node departed event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:principal departed event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:peer adjacency changed event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:join request accepted event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:merge candidate seen event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:member orphaned event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2624) Last Quantum interval:61426814
Apr 02 2021 10:23:32 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:32 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node arrived event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node departed event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:principal departed event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:peer adjacency changed event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:join request accepted event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:merge candidate seen event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:member orphaned event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2624) Last Quantum interval:61426814
Apr 02 2021 10:23:32 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:32 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:32 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:32 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node arrived event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node departed event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:principal departed event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:peer adjacency changed event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:join request accepted event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:merge candidate seen event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:member orphaned event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2624) Last Quantum interval:61426814
Apr 02 2021 10:23:32 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:32 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node arrived event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node departed event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:principal departed event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:peer adjacency changed event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:join request accepted event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:merge candidate seen event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:member orphaned event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2624) Last Quantum interval:61426814
Apr 02 2021 10:23:32 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:32 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:32 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:32 GMT: DETAIL (hb): (hb.c:4785) done tending channel
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node arrived event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:node departed event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:principal departed event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:peer adjacency changed event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:join request accepted event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:merge candidate seen event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2620) Fault:member orphaned event_ts:0
Apr 02 2021 10:23:32 GMT: DETAIL (clustering): (clustering.c:2624) Last Quantum interval:61426814
Apr 02 2021 10:23:32 GMT: DETAIL (hb): (hb.c:4757) tending channel
Apr 02 2021 10:23:32 GMT: DETAIL (hb): (hb.c:4785) done tending channel

lucien · April 3, 2021, 11:41pm

From your logs, it looks like the node is trying to cluster with other nodes. But you only have a loopback ip address configured.

If this is a single node cluster try commenting out mesh-seed-address-port.

Other thoughts:

There is high IO that would be resulting from detail log level that you have dynamically enabled.
There is the nsup process that would consume cpu. Nsup handles expirations, evictions. Which version of Aerospike are you running? On newer versions nsup-period is set to zero by default. Assuming no expirations and default-ttl of 0. Please see: Configuration Reference | Aerospike Documentation

" As of Aerospike version 4.9, the server will not start if nsup-period is 0 (the default) but default-ttl is non-zero, unless allow-ttl-without-nsup is set true."

Have you also looked at using Docker contaner?

benbor · April 5, 2021, 11:04am

Hi!

try commenting out mesh-seed-address-port

Doesn’t really help for me (No changes in CPU consumption)

Which version of Aerospike are you running?

Aerospike Community Edition build 4.5.0.10

There is the nsup process that would consume cpu

Probably it is. Unfortunately for me, nsup configs introduced only in 4.5.1 So, I’ll update aerospike and repeat later

Have you also looked at using Docker contaner?

Not yet. But could you explain, how it would help me in this problem?

Thanks for your attention!

lucien · April 6, 2021, 7:36pm

The container vs. VM is more of a general recommendation since a VM may be using more resources. That said, I use both. (docker and vagrant)

Keep us posted on the upgrade and disabling nsup.

Topic		Replies	Views
Aerospike Configuration Review Configuration	6	5812	April 21, 2017
Aerospike slow performance write/batch-read	3	3431	October 16, 2017
High server CPU usage on google cloud Google Compute Engine (GCE)	4	8162	April 17, 2015
Multiple performance problems Tuning	4	3530	September 4, 2015
Poor performance after migrate environment from gcloud Tuning	3	334	February 14, 2023

Help to configure Aerospike for lowest CPU consumption (for develop environment)

Related Topics