Wire protocol

vovagp · October 26, 2014, 7:48pm

Due to the fact that the original erlang client is very limited in its ability, i read this page http://www.aerospike.com/wire-protocol/ and tried to write code to read the data from test ns, zzz set and 1 key.

This is tcp packet data (from php client) for this request:

2, 3, 0,0,0,0,0,57,

22,3,0,0,0,0,0,0, 0,0,0,0,0,0,0,0, 3,e8,0,2,0,0,0,0, 0,5,0,74,65,73,74,0, 0,0,15,4,a9,44,ff,82, 4e,3d,a3,b4,ad,1f,3,37, 49,f9,dd,bb,aa,46,6e,9a, 0

In 0,5,0,74,65,73,74,0 bytes 74,65,73,74 mean “test” namespace, right? In 0,0,15,4,a9,44,ff,82 bytes 15 mean 21 bytes of data and 4 mean RIPEMD160 digest representing the key, right?

My main issue is that I do not understand how test->zzz->1 (or, maybe, zzz->1) turns into a9,44,ff,82, 4e,3d,a3,b4,ad,1f,3,37, 49,f9,dd,bb,aa,46,6e,9a.

Help P.S. Sorry for my bad english.

vovagp · October 27, 2014, 4:12pm

If i read from test->zzz->1 data [“h”=>1] (in php notation) i get such a package:

02 03 00 00 00 00 00 27 16 00 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00 01 : n_ops; number of operations to follow in the data payload, which will follow the fields 00 00 00 0d : field1 (13 bytes) 01 : type of data -The different types allowed are described below, CL_MSG_FIELD_TYPE_SET, a particular “set” within the namespace 01 00 01 68 00 00 00 00 00 00 00 01

Who can explain me last 12 bytes? How can i unpack this data?

vovagp · October 30, 2014, 4:30am

I studied the implementation of the data decoder in golang client code. Thanks anyway

vovagp · October 30, 2014, 4:55am

1, particle type (eq INTEGER, STRING, LIST etc) 0, version 1, key name size 68, key name and 00 00 00 00 00 00 00 01 is key value by offset+length

wchu · October 30, 2014, 8:34pm

Thanks for updating with the information, and sorry for the late response. We will be updating our wire protocol document soon to have more detailed explanation.

Sergiy · March 13, 2015, 10:42am

Hello. I also tried create (using C language) tcp data packet, and I not understand how to right their collect?! Can someone tell me how? For example send data to server: key = {name_key : “test_key”, namespace : “test”, set : “demo_set”} data_for_send = {“bin1” : “MyBinString”}

Show me example tcp packet, please.

wchu · March 16, 2015, 1:21am

Best to point to the C client source code, as of version 3.1.8

github.com

aerospike/aerospike-client-c/blob/3.1.8/src/main/aerospike/aerospike_key.c#L253-L262


      
          	uint8_t* cmd = as_command_init(size);
          	uint8_t* p = as_command_write_header(cmd, 0, AS_MSG_INFO2_WRITE, policy->commit_level, 0, policy->exists, policy->gen, rec->gen, rec->ttl, policy->timeout, n_fields, n_bins);
          		
          	p = as_command_write_key(p, policy->key, key);
          
          
	for (uint32_t i = 0; i < n_bins; i++) {
          		p = as_command_write_bin(p, AS_OPERATOR_WRITE, &bins[i], &buffers[i]);
          	}
          	
          	size = as_command_write_end(cmd, p);

Hope helps

Sergiy · March 16, 2015, 9:50am

Thanks for answer. I read that article (code)… But, I need structure TCP packet for iOS (Objective-C). Example: for READ to server data

key = {name_key : "test_key", namespace : "test", set : "demo_set"} 
data_for_send = {"bin1" : "MyBinString"}

— Header Section — >

header_size : 22
info1 : 1
info2 : 0
info3 : 0
unused : 0
result_code : 0 // 0 on requests - I have received msg for succeeded
generation : ? //don’t understand what set
expiration : 0
transaction : 0
n_field : ? //what set
n_ops : ? //what set
data : ? //what set

When i collected all data - > I must to send to socket. ) Tell me please, how to correctly create header section for added to NSOutputStream?

Sergiy · March 16, 2015, 10:19am

vovagp tell me please how to collected all data (tcp packet)?

Sergiy · March 17, 2015, 8:06am

I send data

bin1 : 1234
bin2 : "LyahovchukS"

Wireshark show me tcp packet what to send to server

02 03 00 00 00 00 00 70 
16 00 01 00 00 00 00 00 
00 00 00 00 00 00 00 00 
03 e8 00 03 00 02 00 00 
00 05 00 74 65 73 74 00 
00 00 09 01 64 65 6d 6f 
5f 73 65 74 00 00 00 15
04 29 60 89 81 09 b3 de 
3c d8 b4 84 3e a8 1c a7
54 cd 8f f7 4e 00 00 00
10 02 01 00 04 62 69 6e
31 00 00 00 00 00 00 04
d2 00 00 00 13 02 03 00 
04 62 69 6e 32 4c 79 61 
68 6f 76 63 68 75 6b 53

Maybe somebody explain all according to the protocol?

wchu · March 19, 2015, 6:34am

http://www.aerospike.com/wire-protocol/

First byte is “2”, which is version Second byte is “3”, indicating its a AS_MSG, Next 6 bytes is size, 0xl70 0x16 is header_size of message

etc etc

Sergiy · March 19, 2015, 7:34am

Thanks, but I understand the beginning. Please, look my picture, and tell me your comment.

wchu · March 19, 2015, 10:12pm

You are off-by-one byte after the “info2” byte. There should be an “info3” byte. This will shift all your subsequent data. This solves your issues 1-3.

For 4-5 - the key “test-key”, we have already converted it to the 20 byte RIPEMD digest. This is the 21 bytes starting: 04 29 60 89 … where 04 = CL_MSG_FIELD_TYPE_DIGEST_RIPE