Hi Team.
I have been evaluating Aerospike for one of our data store needs. Before moving to enterprise edition, I wanted to see if this can really handle our usecases quickly. For this I have generated tons of data and checked for read and write throughput/latencies with the available class of hardware. Before describing the issue that we are hitting, I have done lot of googling around it and surprised that nobody had see this error before. Lets go for more details now,
-
OS version:
uname -a Linux temp-1042071 4.9.0-0.bpo.5-amd64 #1 SMP Debian 4.9.65-3+deb9u2~bpo8+1 (2017-01-05) x86_64 GNU/Linux -
Aerospike server version: aerospike-server-community-4.0.0.4-debian8.tgz
-
Aerospike Go client version: ## November 29 2017: v1.31.0
-
Disc Hardware type: SSD
-
Version of Go: go version go1.10 linux/amd64
The row under contention: I have inserted a row with primary key: P2898 in two different sets, signal1 & signal2 with AQL I can access and see the contents for this key in both sets, even with digest of the row, I can access it,
**aql> select * from signals.signal1 where PK = 'P2898'**
**+---------------------------------------------------------------------------------------------------------------------------------------+**
**| value |**
**+---------------------------------------------------------------------------------------------------------------------------------------+**
**| LIST('[2898.2899, 2898.789234565288, 2898.789234565288, 2898.789234565288, 2898.789234565232, 2898.789234565232, 2898.789234565232]') |**
**+---------------------------------------------------------------------------------------------------------------------------------------+**
**1 row in set (0.001 secs)**
**OK**
**aql> select * from signals.signal2 where PK = 'P2898'**
**+---------------------------------------------------------------------------------------------------------------------------------------+**
**| value |**
**+---------------------------------------------------------------------------------------------------------------------------------------+**
**| LIST('[2898.2899, 2898.723238923457, 2898.723238923457, 2898.723238923457, 2898.723238923457, 2898.723238923456, 2898.723238923456]') |**
**+---------------------------------------------------------------------------------------------------------------------------------------+**
**1 row in set (0.001 secs)**
**OK**
**aql> select * from signals.signal2 where DIGEST='c59a06ff228fce104be523cdbec987ad557fcfc0'**
**+---------------------------------------------------------------------------------------------------------------------------------------+**
**| value |**
**+---------------------------------------------------------------------------------------------------------------------------------------+**
**| LIST('[2898.2899, 2898.723238923457, 2898.723238923457, 2898.723238923457, 2898.723238923457, 2898.723238923456, 2898.723238923456]') |**
**+---------------------------------------------------------------------------------------------------------------------------------------+**
**1 row in set (0.001 secs)**
**OK**
**aql> select * from signals.signal2 where DIGEST='05a4f03e6e8ea5ec9a17df6ccd0c92fd39b122fc'**
**+---------------------------------------------------------------------------------------------------------------------------------------+**
**| value |**
**+---------------------------------------------------------------------------------------------------------------------------------------+**
**| LIST('[2898.2899, 2898.789234565288, 2898.789234565288, 2898.789234565288, 2898.789234565232, 2898.789234565232, 2898.789234565232]') |**
**+---------------------------------------------------------------------------------------------------------------------------------------+**
**1 row in set (0.001 secs)**
Now the problem is that, (consistently reproducible) I am not able to get data using the Go client .Get() or BatchGet() method. I see the following error when I invoke the get command.
**Skip the unpacker error**
**/usr/local/go/src/runtime/panic.go:505 (0x42b1c8)**
** gopanic: reflectcall(nil, unsafe.Pointer(d.fn), deferArgs(d), uint32(d.siz), uint32(d.siz))**
Just to ensure that this is indeed a client-side issue, I have written one get UDF which just returns the value for this record, and there I am able to see the data flowing in,
The UDF is simple get as follows: function getRecord(rec, binName, pk) if aerospike:exists(rec) then return rec[“value”] end return nil end
If I look closely on what data is transferred between systems, I see that go client is getting following bytes for this,
The problem is only for the 15th pair(index 14 in buffer array) which I am receiving in problematic record, as, 199 instead of 203 which is getting in for the good records. The switch-case code in client which gets executed because of this is,
for 199 . this returns non-nil error -
in unpacker.go:
case 0xc7: // Skip over type extension with 8 bit header and bytes
count := int(upckr.buffer[upckr.offset] & 0xff)
upckr.offset += count + 1 + 1
return nil, skipHeaderErr
for 203,
in unpacker.go:
case 0xcb:
val := Buffer.BytesToFloat64(upckr.buffer, upckr.offset)
upckr.offset += 8
return val, nil
I want to use the batch calls (because of inbuilt workgroup code and concurrency instead of building my own around the same). Any suggestions on why this must be happening and why I am getting results of 199 for type for this particular record+set combination.
One option in parallel I would try out is upgrading the go client version which is released in March. Your assistance is highly appreciated for this.
