LSTACK remove individual element


#1

I’m developing a social network ‘like’ feature using the Go client. The majority of activity will be to read the most recent say 50 likes for a post; and add new likes.

For this LSTACK is perfect, however there are times when a user will ‘unlike’ a post, and thus the existing ‘like’ needs to be removed from the lstack. Is there a way I can scan over the entire lstack or use a filter to select an individual element and remove it from the lstack?


#2

LSTACK is built to be optimal for fetch the last few elements which makes it really inefficient for the cases where you want to get random access. No such facility is provided in LSTACK right now.

For ability to have random access please use LLIST. That would also mean for access recent 50 likes it would become less optimal than LSTACK. To be able to optimize read of 50 likes from a record optimally you may want to store it in hot list of your own. The record would like

  • HotLike -> LIST of recent 50 likes in sorted order
  • Likes -> LLIST of all the likes.

Operations

  • Write Will add to hotlist may want to move likes from LIST to LLIST in chuncks of may be 10 on every 10th write
  • Reads would read it from the HOTLIST
  • Delete would try to see if like is found in HOTLIST remove it from it and possibly delete it from LLIST (this should be effiecient)

HTH

– R


#3

Thank you for a very detailed answer Raj.

Given that the number of deleted likes should relatively small, I am also considering the option of using a LIST of deleted likes alongside the LSTACK of likes, in which case I can read from the LSTACK then filter out any records which exist in the deleted LIST. This could presumably be done inside a Record UDF for performance.

In addition to the original question, I also need the ability to paginate through the list of likes - this could be done in a less than optimal fashion using an LSTACK peek (i.e. for results 50-100 one could peek 100 and discard the first 50).

In Redis this whole scenario is relatively easy using Sorted Sets with the ‘SCORE’ being an epoch timestamp to enable correct ordering.

I found an answer related to this on stackoverflow from Nov '14 - are there now any examples of how one can emulate the functionality of a Redis Sorted Set in Aerospike?


#4

Hi,

List is implemented using B+Tree internally so it is a actually LSortedMap by nature. So when you insert based on timestamp you can actually query using it timestamp range. In LLIST sorting key (i.e search key) and uniqueness key both are same.

Now I would think there are two keys in your setup

  1. Timestamp of like so you would like use to do range query one.
  2. Like-id or user-id on which you want to maintain uniqueness

Given Aerospike does not implement Sorted Set, you probably have to do what Redis does internally. Maintain a map (like_id -> score) and ordered list score->(like_id:data for like)

May be I can cook up example for you. Can you answer few questions

  1. How many likes do you want to store per user …
  2. How big is data for like above

– R


#5

Hi Raj,

A map + ordered list does sound like a good approach. You are correct, there would be two keys in the setup: the timestamp of like and the user-id of the like.

To answer your questions specifically:

  1. Likes per user would vary from 0 to say 10,000, with 80% being max 100 and 20% max 10,000
  2. A like consists of the timestamp, a user-id and a utf-8 user name. Assume say, 128 bytes max total.

I’m intending to use a cursor system for pagination i.e. get me the next 25 likes after cursor=“timestamp + perhaps a random string”. The random string being used so that the likes with the same timestamp are distinguishable.

Thanks Jon


#6

Sounds good …

– R


#7

@memunix,

Thank you for posting about LDTs in our forum. Please see the LDT Feature Guide for current LDT recommendations and best practices.


#8

@memunix,

Effective immediately, we will no longer actively support the LDT feature and will eventually remove the API. The exact deprecation and removal timeline will depend on customer and community requirements. Instead of LDTs, we advise that you use our newer List and SortedMap APIs, which are now available in all Aerospike-supported clients at the General Availability level. Read our blog post for details.