Scan using C client giving segmentation error after scanning bunch of records

Client Libraries C Client Library
1

Hi Team, I am getting memory segmentation error while running scan (concurrent is False) after scanning subset of records . Scan abruptly fails after processing few records so not sure what is causing this issue.

Below is the high level steps for the implemented code.

  1. connection to aerospike server is made in main function
  2. aerospike_scan_foreach is call with cb and connection pointer
  3. in callback first we read the list of map under the scanned record PK and then reinsert the same after applying the as_hashmap_sort_flag to sort the list of map (this we do in overwrite mode)

Note: Failure is not specific to record it fails randomly. When we only scan then no issues but when we scan and perform read and write in callback then it fails after processing 150k records out of 200k.

Thanks

2

I would need to see your scan source code to help.

3

Hi Brian, Now the code is working fine (updated code below) though unable to find the root cause behind failures (was getting segmentation error).

bool mpr_cleanup_cb(const as_val* p_val, void* udata) {

if (p_val == NULL) { // scan is complete printf(“\n\nScan callback returned null - scan is complete”); return true; }

    as_bin* b;
    as_list* list_val;
    as_string* list_key_val;
    as_map* list_map;
    const char *name_token[2];
    as_record* updated_rec  = NULL;
    aerospike *as_new = (aerospike*)udata;
    int   j=0,i=0;
    uint32_t list_len;
    as_status ret;
    const char* key_val=NULL;
    as_error err;
    as_operations* ops = as_operations_new(2);
    as_key key;



    as_record* rec =NULL;
    rec=as_record_fromval(p_val);

    if (! rec) {
        //printf("scan callback returned non-as_record object");
        return false;
    }

    as_key_init_digest(&key, g_namespace, g_set, rec->key.digest.value);
	 list_val = as_record_get_list(rec,g_list_bin_nm);
            list_len = as_list_size(list_val);
            key_val=as_record_get_str(rec,g_list_key_nm);
            printf("\n printing list key value: %s ",key_val);

            as_arraylist *as_si_list = as_arraylist_new(list_len, 0);
            //itterate over the list
            for(j=0;j<list_len;j++)
            {
                    list_map = as_map_fromval((as_val*)as_list_get(list_val,j));
                    as_hashmap_set_flags((as_hashmap*)list_map, AS_MAP_KEY_ORDERED);
                    as_arraylist_append_map(as_si_list, (as_map*)list_map);

            }
                list_len = as_list_size((as_list*)as_si_list);
                if(list_len >0)
                {

                      as_operations_add_write(ops, g_list_bin_nm,(as_bin_value*)as_si_list);
                    ret=aerospike_key_operate((aerospike*)udata, &err, NULL, &key, ops, NULL);

                      if(ret!=AEROSPIKE_OK)
                      {
                        printf("\n Return code: AEROSPIKE_NOT_OK - %d for %s ",err.code,key_val);
                      }
                      else
                      {
                            printf("\n Return code: AEROSPIKE_OK - %d for %s ",err.code,key_val);
                            name_token[0]=g_list_bin_nm;
                            name_token[1]=NULL;
                            aerospike_key_select(as_new, &err, NULL,&key,name_token, &updated_rec);
                            as_list* updated_list_val = as_record_get_list(updated_rec,g_list_bin_nm);
                            uint32_t updated_list_len = as_list_size(updated_list_val);
                            printf("\n Updated list length: %d",updated_list_len);
                            //as_arraylist_destroy(as_si_list);

                            if(updated_list_len==list_len)
                            {
                            printf("\n Before and After list length for: %s is_matching",key_val);
                            }
                            else
                            {
                                    printf("\n Before and After list length for: %s is_not_matching",key_val);
                            }
                      }

                }
                else

                    {  
					
					    printf("\n List is empty for the scanned key %s",key_val);

                    }

as_record_destroy(rec); as_record_destroy(updated_rec); as_operations_destroy(ops);

return true;

}

Below are the statements which have been removed and replaced with given statements. removed: list_key_val = (as_string*)as_val_reserve(as_record_get_string(rec,g_list_key_nm)); char* key_val = list_key_val->value; ret=aerospike_key_operate((aerospike*)udata, &err, NULL, &key, &ops, &rec);

replaced with : key_val=as_record_get_str(rec,g_list_key_nm); ret=aerospike_key_operate((aerospike*)udata, &err, NULL, &key, &ops, NULL);

4

Do not call as_record_destroy(rec) in scan callbacks. The client does that for you when the callback completes.

5

Hi Brian, I do see it fails when i put as_operations_destroy but not sure why. Do you see any challenge in provided code? Or everything looks fine?

6

If ops is created on the heap in the callback, then ops should be destroyed when done using ops.

7

Hi Brian, yes we are creating ops with below statement. as_operations* ops = as_operations_new(2);

we are destroying ops using below command. as_operations_destroy(ops);

But it fails with memory fault issue not sure why.

8

aerospike_key_operate() sets the as_record* arg to NULL which is wrong. Instead, use the following code snippet.

as_operations* ops = as_operations_new(1);
as_operations_add_write(ops, g_list_bin_nm,(as_bin_value*)as_si_list);

as_record* updated_rec = NULL;

ret=aerospike_key_operate((aerospike*)udata, &err, NULL, &key, ops, &updated_rec);

// process rec here.
as_record_destroy(updated_rec);
as_operations_destroy(ops);
9

Why do we need to pass updated_rec in aerospike_key_operate API? In this we are performing write operation and not returning anything.

updated_record we are using while doing select.

10

I stand corrected. When all operations are write operations, the as_record argument can be NULL. You should still allocate ops closer to aerospike_key_operate() to avoid leaking when the function returns early.