Asbackup takes too long


#1

i have about 116,000,000 record , when i tried to backup with asbackup , it will take more than one day , so how to reduce the operation time ?


#2

To answer your question I need following information:

  • Which version of aerospike server you are running.

  • Get average record size by running “asbackup -n test -e”

  • Are you using all default values while running asbackup? If yes then run with “-f 3”, which will increase backup speed and affect read/write load.


#3

we are using aerospike server 3.6.0 i tried estimate command this is output

2016-02-18 07:51:13 GMT [INF] [18854] Backed up enough samples for estimate
2016-02-18 07:51:13 GMT [INF] [18854] Node scan for BB93A938D01B95C aborted
2016-02-18 07:51:14 GMT [INF] [18853] Backed up 10000 record(s), 7 secondary index(es), 2 UDF file(s) from 3 node(s), 3838465 byte(s) in total (~383 B/rec)
2016-02-18 07:51:16 GMT [INF] [18855] Node scan for BB99E7F8D01B95C aborted
2016-02-18 07:51:25 GMT [INF] [18856] Node scan for BB932938D01B95C aborted
2016-02-18 07:51:25 GMT [INF] [18835] Estimated overall record size is 388 byte(s)

#4

It looks like you have approximately 45 GB of data, Which should not take more than an hour to backup.

Can you just provide us what exact command you are using? How much time it takes for current backup?


#5

LOCALPATH=/root/bakcup

datestamp=$(date +%Y-%m-%d)

asbackup -n push -d $LOCALPATH/asbackup_$datestamp


#6

You have 3 node cluster. So the only way you increase your backup speed is- increase backup priority. Run following:

asbackup -n push -d $LOCALPATH/asbackup_$datestamp -f 3


#7

I am having a similar problem, but what happens is with both backup and asbackup -n mynamespace -e I get 2016-02-25 20:10:31 GMT [INF] [ 8499] Starting 100% backup of 127.0.0.1:3000 (namespace: userinfo, set: [all], bins: [all]) to [none] 2016-02-25 20:10:31 GMT [INF] [ 8499] [src/main/aerospike/as_cluster.c:302][as_cluster_seed_nodes] Add node BB9AC299F0B0022 127.0.0.1:3000 2016-02-25 20:10:31 GMT [INF] [ 8499] [src/main/aerospike/as_cluster.c:368][as_cluster_find_nodes_to_add] Add node BB9E4C19E0B0022 10.13.220.125:3000 2016-02-25 20:10:31 GMT [INF] [ 8499] [src/main/aerospike/as_cluster.c:360][as_cluster_find_nodes_to_add] Duplicate node found BB9AC299F0B0022 127.0.0.1:3000 2016-02-25 20:10:31 GMT [INF] [ 8499] Processing 2 node(s) 2016-02-25 20:10:31 GMT [INF] [ 8499] Node ID Objects Replication
2016-02-25 20:10:31 GMT [INF] [ 8499] BB9AC299F0B0022 16386535 2
2016-02-25 20:10:31 GMT [INF] [ 8499] BB9E4C19E0B0022 16386535 2
2016-02-25 20:10:31 GMT [INF] [ 8499] Namespace contains 16386535 record(s) 2016-02-25 20:10:31 GMT [INF] [ 8518] Starting backup for node BB9AC299F0B0022 2016-02-25 20:10:31 GMT [INF] [ 8519] Starting backup for node BB9E4C19E0B0022 2016-02-25 20:10:31 GMT [INF] [ 8518] No secondary indexes 2016-02-25 20:10:31 GMT [INF] [ 8518] Backing up 0 UDF file(s) 2016-02-25 20:10:32 GMT [INF] [ 8517] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec) 2016-02-25 20:10:42 GMT [INF] [ 8517] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)

There are 16M records in the biggest set of this namespace. We have another namespace with total 8M records that backups no problem in under 5 min. With asbackup when printing the progress a lot of times it is (~0 KiB/s, ~0 rec/s, ~0 B/rec) and once in a while there is non-zero values. It takes almost a full day to backup the namespace. The -f 3 option didn’t affect the performance.

We are using aerospike-amc-community-3.6.3.

Command we are using asbackup --file-limit 3000 --namespace $namespace --directory $output_directory