Multiple operations on CDT return the result of the last operation only

Client Libraries Ruby Client
,
1

Hi,

We need to do multiple operations on a large hashmap bin; essentially, multiple get by keys. The keys are in no order lexicographically. So we send multiple get_key operations on the same bin with different keys but contrary to the expectation the result contains the value for just the last operation and not the values for all the keys.

client.put(key, {'map' => {'a' => 1,  'b' => 2, 'c' => 3, 'd' => 4}})
ops = []
ops << CDT::MapOperation.get_key('map', 'a', return_type: 8) # MapReturnType::KEY_VALUE
ops << CDT::MapOperation.get_key('map', 'b', return_type: 8)
ops << CDT::MapOperation.get_key('map', 'c', return_type: 8)
result = client.operate(key, ops)

Expectation : result.bins to be {'map' => {'a' => 1, 'b' => 2, 'c' => 3}}

What really happens is that result.bins contains only the value for the last operation i.e. {'map' => {'c' => 3}}.

This thing happens for list data type as well. We tried doing these same operations using the java client and the returned data is correct for both map and list. In ruby only the last operation is returned.

@Jan Please, help :slight_smile:

2

@zingoba,

The different Aerospike clients are unfortunately not consistent in this behaviour. The server actually returns the values for all operations, but the Ruby client currently only retains the last value when it aggregates all the values into the result Hash map.

I’m thinking of adopting the Java client behaviour of simply wrapping multiple values for the same bin name in an array. This would be a fairly easy change to make. There are some draw backs to this approach, esp. that it would be difficult to tell whether a returned array value is the result of a single operation or multiple operations that got combined. The alternative would be to use a true multimap implementation [1].

If you don’t mind, can you please open a feature request at our Github repo and I will consider this for the next release.

Cheers, Jan

[1] multimap | RubyGems.org | your community gem host

3

Hi,

I have overridden the parse_record method for the OperateCommand class to maintain a list of operations performed for each of the bins as follows :

class Aerospike::OperateCommand < Aerospike::ReadCommand
  def parse_record(op_count, field_count, generation, expiration)
    bins = op_count > 0 ? {} : nil
    receive_offset = 0

    if field_count > 0
      i = 0
      while i < field_count
        field_size = @data_buffer.read_int32(receive_offset)
        receive_offset += (4 + field_size)
        i = i.succ
      end
    end

    i = 0
    while i < op_count
      op_size = @data_buffer.read_int32(receive_offset)
      particle_type = @data_buffer.read(receive_offset+5).ord
      name_size = @data_buffer.read(receive_offset+7).ord
      name = @data_buffer.read(receive_offset+8, name_size).force_encoding('utf-8')
      receive_offset += 4 + 4 + name_size


      particle_bytes_size = op_size - (4 + name_size)
      value = Aerospike.bytes_to_particle(particle_type, @data_buffer, receive_offset, particle_bytes_size)
      receive_offset += particle_bytes_size
      bins[name] ||= []
      bins[name] << value
      i = i.succ
    end
    bins = Hash[bins.map do |name, value|
      [name, value.size == 1 ? value.first : value]
    end]

    Record.new(@node, @key, bins, generation, expiration)
  end
end

This behaves in line with the java client.

4

Himanshu, I’ll include a similar change in the next Ruby client update. (Probably some time next week.)

Cheers, Jan

5

v2.2.0 is out on RubyGems.org now.

You will need to pass an OperatePolicy with record_bin_multiplicity set to the value RecordBinMultiplicity::ARRAY on your Client#operate call in order to receive all results for multiple read operations on the same record bin.

Example:

require 'aerospike'
include Aerospike
client = Client.new()
key = Key.new('test', 'test', 'map')
client.put(key, {'map' => {'a' => 4,  'b' => 3, 'c' => 2, 'd' => 1}})
return_type = CDT::MapReturnType::VALUE
ops = []
ops << CDT::MapOperation.get_index('map', 0, return_type: return_type)
ops << CDT::MapOperation.get_by_rank('map', 0, return_type: return_type)
result = client.operate(key, ops)
result.bins // => { "map" => [ 1 ] }
policy = OperatePolicy.new(record_bin_multiplicity: RecordBinMultiplicity::ARRAY)
result = client.operate(key, ops, policy)
result.bins // => { "map" => [ 4, 1 ] }

Cheers, Jan