How to avoid LDT-Unique Key or Value Violation on Ordered List


#1

How to avoid “LDT-Unique Key or Value Violation”, when adding something that already exists into the Ordered List?


#2

Hi,

The Large Ordered List LDT can handle Unique keys and non-Unique keys. Unique is set by default because that is what we assumed would be the larger use-case.

When you’re in unique mode, you have the option of overwriting a value using the update() function (which will get you around the “value already exists()” error).

When you’re in non-unique mode (you want duplicate values), then you need to change the configuration setting via the “createSpec” parameter on the very first insert of a new LDT. This is not something you can change after the LDT instance has already been created.

The “createSpec” parameter (sometimes called the “userModule” or “createModule” in the documentation), refers in this case to a UDF module that contains one or more UDFs (User Defined Functions).

There is a specially named UDF that is called by the LDT create logic if the module exists.

So, here are the steps: (1) Create a Lua UDF (with the contents below) called “myCreateModule.lua” (2) Register the UDF Module (3) In the llist.add() call, use the UDF Module Name (“myCreateModule”) in the parameter position for “userModule”

Then, when you call add() for the first time on this LDT instance, it will use the settings that are set in the internal “adjust_settings()” function.

Here is your “myCreateModule.lua” file:

– ====================================================================== – CreateModuleLLIST (UserModule Example) – ====================================================================== – Global Print Flags (set “F” to true to print) local GP; local F=false;

– Used for version tracking in logging/debugging local MOD = “CM_LLIST:2014_11_17.A”;

– ====================================================================== – Must define a table that holds the functions that will be exported – and used by the LDT instance. – ====================================================================== local userModule = {};

– ====================================================================== – Lua Imports – ====================================================================== – Import the functions we will be using in Large List to perform the – LLIST LDT configuration. local llist_settings = require(‘ldt/settings_llist’);

– ====================================================================== – adjust_settings() – Set this LLIST for best performance using – + 100byte objects – + 8 byte keys – ====================================================================== – This is a specially named function “adjust_settings()” that the LDT – configure code looks for. If an “adjust_settings()” function exists – in this module (it’s a Lua Table entry), it will be called, with the – ldtMap as a parameter, to initialize this LDT instance on either a – create call or on the first insert. – ====================================================================== function userModule.adjust_settings( ldtMap )

– Use a medium amount of Top Record space for the B+ Tree Root Node llist_settings.set_root_list_max( ldtMap, 100 );

– With keys + digest == 30bytes, keep the overall sub-rec size under 7kb. llist_settings.set_node_list_max( ldtMap, 200 );

– With Object Sizes == 100bytes, keep the overall sub-rec size under 7kb. llist_settings.set_leaf_list_max( ldtMap, 70 );

– Keep no more than 40 100b objects in the Record Compact List. llist_settings.set_compact_list_threshold( ldtMap, 40 );

– Allow for NON-UNIQUE keys (i.e. allow duplicate values). – (Unfortunately, this is not yet a standard method, so we have to use – secret internal notation. local uniqueField = ‘U’; local AS_FALSE = ‘F’; ldtMap[uniqueField] = AS_FALSE;

end – adjust_settings()

– ====================================================================== – Return the value of this module’s table so that others importing this – module get the table reference. – ====================================================================== return userModule;

– ======================================================================== – – – – – – – – – – ========================================================================


Error when adding new value to LLIST
Allow duplicate keys in Large List in aerospike server 3.5.12
#3

Hey Toby,

I use the python client to upload data to Aerospike and I see no API for the update() function you mentioned. Is there any other way we can gracefully allow duplicates without throwing errors?

Ashwin


#4

@arypurnomoz and @ashwin_saval:

Thank you for posting about LDTs in our forum. Please see the LDT Feature Guide for current LDT recommendations and best practices.


#5

@arypurnomoz and @ashwin_saval,

Effective immediately, we will no longer actively support the LDT feature and will eventually remove the API. The exact deprecation and removal timeline will depend on customer and community requirements. Instead of LDTs, we advise that you use our newer List and SortedMap APIs, which are now available in all Aerospike-supported clients at the General Availability level. Read our blog post for details.