Strange consistency problem [Resolved]


#1

Hi, I’m a JEE Developer who is quite new to Aerospike. We are trying to test if we can migrate to Aerospike from MySQL. So, I have built a very simple and small Spring project to run our test.

Till now everything was fine and we were very excited, but after I changed my code to do a sequence of read>modify>read the same record I faced with a strange problem. When I read a record, I extract one of its bins, say Age, then I write that key again with a new Age value, then I read that record again and extract its Age bin value.

The problem is, sometimes the value I get from the second read is not the value I have written AND it’s not the value I have read from the first read, and also sometimes I get an Error Code 3: Generation error error. I don’t know where I am doing wrong.

Note: I wrote my program in Spring 4, for Tomcat 8 AS, and we test our benchmarks with Apache Jmeter with 10000 request per second with random values for recordId and Age.

My AeroSpike config file:

service {

	user root
	group root
	paxos-single-replica-limit 1 
	pidfile /var/run/aerospike/asd.pid
	service-threads 4
	transaction-queues 4
	transaction-threads-per-queue 4
	proto-fd-max 15000
	transaction-pending-limit 0
	transaction-retry-ms 20000
}

logging {

	# Log file must be an absolute path.
	file /var/log/aerospike/aerospike.log {
		context any info
	}
}

network {

	service {
		address any
		port 3000
		access-address 172.8.1.29
	}

	heartbeat {
		mode multicast
		address 172.8.1.29 	#239.1.99.222
		port 9918
		interval 150
		timeout 10
	}

	fabric {
		port 3001
	}

	info {
		port 3003
	}
}

namespace test {

	replication-factor 1
	memory-size 8G
	default-ttl 0
	read-consistency-level-override all	# read from all nodes
	write-commit-level-override all		# write in all nodes
	
	storage-engine memory
}

////////////////////////////////// My Spring Controller method is:

@RequestMapping("/")
	public String welcome(Long recordId, Integer newAge, ModelMap model) throws Exception{

		System.out.println("record id = " + recordId);
		System.out.println("new age = " + newAge);
		
		if(recordId == null)
			recordId = 1L;
		if(newAge == null)
			newAge = 1;
		
		AerospikeConnector.getInstance();
		AerospikeClient client = AerospikeConnector.getConnection();

		try {

			Key key1 = new Key("test", "nyt11", Long.valueOf(recordId));
			Record rec = client.get(null, key1);
			
			if(rec != null){
				long age1 = rec.getInt("Age");
				
				WritePolicy policy = new WritePolicy();
				policy.generationPolicy = GenerationPolicy.EXPECT_GEN_EQUAL;
				policy.generation = rec.generation;
				policy.timeout = 50;
				Key key2 = new Key("test", "nyt11", Long.valueOf(recordId));
				Bin bin = new Bin("Age", Integer.valueOf(newAge));
				
				String message = "Writing newAge = " + newAge + " in record = " + recordId + " which currently has age = " + age1;
				System.out.println(message);
				
				client.put(policy, key2, bin);
				long age2 = Integer.valueOf(newAge);
				
				Key key3 = new Key("test", "nyt11", Long.valueOf(recordId));
				Record rec2 = client.get(null, key3);
				long age3 = rec2.getInt("Age");
				
				model.addAttribute("newAge", message);
				if(rec2.getLong("ID") == Long.valueOf(recordId) && rec2.getLong("Age") != Integer.valueOf(newAge)){
					throw new Exception(message + " \nBad number. For record id: " + recordId + " I read this number: " + age1 +
											" I wrote this number: " + age2 + 
											" And then when I read the number it is: " + age3);
				}
			}
		}
		catch(Exception e){
			System.out.println(e);
			throw new Exception(e);
		}
		return "login";
	}

Example of errors:

org.springframework.web.util.NestedServletException: Request processing failed; nested exception is java.lang.Exception: com.aerospike.client.AerospikeException: Error Code 3: Generation error org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:978) …

AND

org.springframework.web.util.NestedServletException: Request processing failed; nested exception is java.lang.Exception: java.lang.Exception: Writing newAge = 5273 in record = 6521 which currently has age = 5304 Bad number. For record id: 6521 I read this number: 5304 I wrote this number: 5273 And then when I read the number it is: 2137 org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:978) …

And no error is logged in the Aerospike log file. Sorry if my question is very simple and thanks for your time!

If you need more information please tell me.


#2

Hi Rita,

you are using RecordId as your key and expecting it to be returned back when you read the object with this key. By default, Aerospike doesn’t save key in the record, but this could be enabled by the configuration setting as explained at http://www.aerospike.com/apidocs/java/com/aerospike/client/policy/Policy.html#sendKey

Please try your code with this sendKey modification.

-samir


#3

Hi! Thank you! I will fix it now and remember it in future :smile: