Backup not working when theres scan jobs


#1

Hello,

Looks like asbackup has issues if theres already a scan (with higher priority) running:

Even if its not getting any data so far it should not abort should it?

Issue is reproducable with 3.15.3.6 as well as 3.15.3.2

2018-05-02 21:22:42 GMT [INF] [91993] Starting 100% backup of 10.0.1.1 (namespace: backupns, set: backupset, bins: [all], after: [none], before: [none]) to /home/b/backupns/105/backupset
2018-05-02 21:22:42 GMT [INF] [91993] [src/main/aerospike/as_cluster.c:124][as_cluster_add_nodes_copy] Add node NODEID1 10.0.1.1:3000
2018-05-02 21:22:42 GMT [INF] [91993] [src/main/aerospike/as_cluster.c:124][as_cluster_add_nodes_copy] Add node NODEID2 10.0.1.2:3000
2018-05-02 21:22:42 GMT [INF] [91993] Processing 2 node(s)
2018-05-02 21:22:42 GMT [INF] [91993] Node ID             Objects        Replication    
2018-05-02 21:22:42 GMT [INF] [91993] NODEID1     19228526       2              
2018-05-02 21:22:42 GMT [INF] [91993] NODEID2     19228526       2              
2018-05-02 21:22:42 GMT [INF] [91993] Namespace contains 19228526 record(s)
2018-05-02 21:22:42 GMT [INF] [91993] Directory /home/b/backupns/105/backupset prepared for backup
2018-05-02 21:22:42 GMT [INF] [92013] Starting backup for node NODEID1
2018-05-02 21:22:42 GMT [INF] [92013] Created new backup file /home/b/backupns/105/backupset/FILEID1_00000.asb
2018-05-02 21:22:42 GMT [INF] [92012] Starting backup for node NODEID2
2018-05-02 21:22:42 GMT [INF] [92012] Created new backup file /home/b/backupns/105/backupset/FILEID2_00000.asb
2018-05-02 21:22:42 GMT [INF] [92013] No secondary indexes
2018-05-02 21:22:42 GMT [INF] [92013] Backing up 0 UDF file(s)
2018-05-02 21:22:43 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:22:53 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:23:03 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:23:13 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:23:23 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:23:33 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:23:43 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:23:53 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:24:03 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:24:13 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:24:23 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:24:33 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:24:43 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:24:53 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:25:03 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:25:13 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:25:23 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:25:33 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:25:43 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:25:53 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:26:03 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:26:13 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:26:23 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:26:33 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:26:43 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:26:53 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:27:03 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:27:13 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:27:23 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:27:33 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:27:43 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:27:53 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:28:03 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:28:13 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:28:23 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:28:33 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:28:43 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:28:53 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:29:03 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:29:13 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:29:23 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:29:33 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:29:43 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:29:53 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:30:03 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:30:13 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:30:23 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:30:33 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:30:43 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:30:53 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:31:03 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:31:13 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:31:23 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:31:33 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:31:43 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:31:53 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:32:03 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:32:13 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:32:23 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:32:33 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:32:42 GMT [ERR] [92012] Error while running node scan for NODEID2 - code -10: Socket read error: 11, 10.0.1.2:3000, 47302 at src/main/aerospike/as_socket.c:248
2018-05-02 21:32:42 GMT [ERR] [92013] Error while running node scan for NODEID1 - code -10: Socket read error: 11, 10.0.1.1:3000, 42094 at src/main/aerospike/as_socket.c:248
2018-05-02 21:32:43 GMT [INF] [92011] 0% complete (~0 KiB/s, ~0 rec/s, ~0 B/rec)
2018-05-02 21:32:43 GMT [INF] [92011] Backed up 0 record(s), 0 secondary index(es), 0 UDF file(s) from 2 node(s), 77 byte(s) in total (~0 B/rec)