Hello, I have a question on TTL modification for namespace from 1 year to 4 months. Current setup has Community Edition 3.6.0 installed and it would get upgraded to an enterprise edition in couple of months time, possibly. The namespace has few billions of records which has an age varying from 1 day to close to 12 months. I want to know what to expect when we reduce the TTL from 1 year to 4 months and is there any difference here in terms of using a CE (ver 3.6.0) vs latest Enterprise edition? Also, is there any specific config change which would help us or which we would need to tweak for a smooth transition? Any specific metrics to watch out for during this process? If you can provide the steps to be followed for reduction of TTL for a smooth transition of recordâs TTL, that would be appreciated a lot. Thanks!
Since you have enterprise support, you should open a case at support.aerospike.com. That said, I do not recall any reason this shouldnât be a smooth transition.
Thank You @kporter! Appreciate your response! We currently have only CE 3.6.0 and are planning to use an Enterprise edition in couple of months. Assuming that we are using CE 3.6.0 for this transition, do you recommend any specific steps to do this transition ? Or we just need to go ahead with a dynamic update of the âdefault-ttlâ config value? Thanks again!
You will need to upgrade to 3.13 (jump version release) to reach the latest version, see the 3.13.0.3 release notes. There shouldnât be a problem changing the default-ttl value though Iâm not sure what your expectations of changing that value are; the default-ttl applies to new writes/updates, it will have no effect on existing records.
One of the problems with using a version that is nearly two years out of date is the difficulty that people will have in remembering any âwar storiesâ regarding gotchas that might happened around that time. Since the 3.6 release, Aerospike has made a lot of changes, especially in the area of defragmentation ( which will kick in more when you reduce the TTL ), and addition of the in-RAM last modified time.
Regarding âreducing the TTLâ. Youâll see in the documentation there is the default TTL - ie, the TTL of the record if, on writes, you use the default value. If, on the other hand, you had set an explicit value in the TTL field on your writes, those are written to the database durably, and the only way to change those values is to read the record ( like with a scan ) and re-write it.
Thus, if youâd like advice, it would be better to say why you are doing this. Do you intend to expire a lot of records early? Is that because you are reducing the cluster size? Or are you reducing it because you intend to write new data - in which case the âsmoothâ thing to do will be to simply add more data and let nature take its toll?
Which parameters to modify will also change depending on what you are doing, and why. If you are expecting many records to pass their TTL quickly, you would want to monitor object size ( because youâre intending the number of objects to go down, right? ), the disk free percentage ( which should be reclaimed over time), and the ânsupâ parameters, which is the background thread which notices elements past their TTL value, evicts them, and puts elements on the free queue.
If you are considering getting enterprise support, why not just get enterprise support, then you can ask Aerospike?
Thanks for taking time out for this detailed response @bbulkow. Really appreciate it. We have figured out that 1 year TTL (default-TTL) is not really required for our use cases to work well. Hence moving to a more favourable TTL. When we update the default-TTL, there would be a significant percentage of records in the cluster which ideally would be out dated and would need to be expired/removed. This is to make sure that we use the cluster resources efficiently. As per my understanding (along with the inputs from @kporter from this thread), we would need to scan and remove the older records (records which are older than the new TTL) and we would keep a watch on the object size , disk free percentage and ânsupâ parameters.
Personally I also support the use of Enterprise edition which additionally gives us access to Aerospike Support.
Thanks again!
Unni - if you can give me one bit of data, I am very curious. Most of the people I know using Aerospike in cases where they are using expiration simply end up setting the storage size ( or cluster size ) to the point where they are evicting, and then ârun fullâ. If they decide their use case doesnât need that much data, they cut the storage and/or cluster size.
Are you using multiple datasets in the cluster, with using our âsetâ functionality, thus you are managing sizes through the TTL feature more than namespace / storage size?
And - thanks for your support of Aerospike. You can help by tweeting / mentioning etc. that Aerospike is a core part of your architecture, and how well it works. Youâll especially like how well it works when you upgrade a few versions, but the fact that you like our vintage 2015 version is also appreciated.