LList items count limited?


#1

Hi,

When try to add an LList bin with 947 items to an existing record, I get the error

/opt/aerospike/sys/udf/lua/ldt/lib_llist.lua:5590: 1409:LDT-Input Parameter Error

But if I try to separate these into two LList bins with 819 and 128 items respectively, then not error occurs.

The total memory I want to store, the 947 items, it’s about 550Kb (not too much…) Please, can you help me with why this error occurs?

Configuration settings:

ldt-enabled true
replication-factor 2
memory-size 10G

default-ttl 0 
storage-engine device {         
	file /opt/aerospike/data/sse  
	filesize 2G                    
	data-in-memory true             
	write-block-size 1M
}

Other technical information:

Server version: 3.6.2

PHP-Client version: 3.4.5

Thanks at all!!


#2

The mentioned error message is raised when the value that is added is considered to be nil/null/empty. How did you call the insert? You may find the exact value that caused this problem (most likely empty) in the log at file: /var/log/aerospike/aerospike.log It should look like: [ERROR]<%s:%s> Input Parameter < value> is bad. If you want to check out their source code, you can find it here: http://bit.ly/1YFvCsf (Github)

However, providing the exact value(s) that caused this will most likely help the developers at AS to prevent queries like these to be sent from the PHP client (if data is publishable, ofc)

Cheers, Manuel


#3

Thanks for your reply!

Yes, I know that this error is raised when a value it’s null, I’ve checked the LUA code at github before create the topic.

I checked on my php script for each value to be inserted if it’s null, but no null value found. The data I will insert it’s an array with 947 items where each one is an array with the mandatory value key for LList and another array with string values, like:

`

[0]=>
    array(2) {
        ["key"]=>
        int(0)
        ["string_index_not_fix"]=>
        array(2) {
            ["id"]=>
            string(4) "4645"
            ["errors"]=>
            array(1) {
                [0]=>
                string(4) "string_value_not_fix"
            }
        }
    }
`

Where the value of index ‘errors’ it’s an array with variable count of items.

The fact is that I’m worried with the error because it’s raised when I try to add 947 items like previous to an LList, but if I split I try to add first 819 items and after the 128 leftover no error raised and when I scan the LList can see the 947 items. Also I tried to add each item independently and not error raised another time.

Then, I think, no empty value can exists at my php array or bad format for any one.

FYI:

  • Log extract when the error is raised:

    Dec 02 2015 11:32:47 GMT: INFO (info): (hist.c::163) (05: 0000000001) (06: 0000000007)

    Dec 02 2015 11:32:47 GMT: INFO (info): (hist.c::137) histogram dump: query (0 total) msec

    Dec 02 2015 11:32:47 GMT: INFO (info): (hist.c::137) histogram dump: query_rec_count (0 total) count

    Dec 02 2015 11:32:49 GMT: INFO (drv_ssd): (drv_ssd.c::2088) device /opt/aerospike/data/sse: used 2414848, contig-free 2044M (2044 wblocks), swb-free 0, w-q 0 w-tot 0 (0.0/s), defrag-q 0 defrag-tot 5 (0.0/s) defrag-w-tot 0 (0.0/s)

    Dec 02 2015 11:32:55 GMT: INFO (udf): ([C]::-1) [ERROR]<lib_llist_2014_12_20.B:llist.add()> Input Parameter is bad

    Dec 02 2015 11:32:57 GMT: INFO (info): (thr_info.c::4812) system memory: free 5339356kb ( 65 percent free )

    Dec 02 2015 11:32:57 GMT: INFO (info): (thr_info.c::4820) migrates in progress ( 0 , 0 ) ::: ClusterSize 1 ::: objects 185 ::: sub_objects 380

    Dec 02 2015 11:32:57 GMT: INFO (info): (thr_info.c::4828) rec refs 565 ::: rec locks 0 ::: trees 0 ::: wr reqs 0 ::: mig tx 0 ::: mig rx 0

    Dec 02 2015 11:32:57 GMT: INFO (info): (thr_info.c::4833) replica errs :: null 0 non-null 0 ::: sync copy errs :: master 0

  • How to add items to LList:

`

    $this->oAerospike= new Aerospike($aConfig);
    $keyTuple = $this->oAerospike->initKey($this->sNamespace, $this->sSet, $sKey);
    $newLList = new \Aerospike\LDT\LList($this->oAerospike, $keyTuple, $sBinName);

    if ($newLList->errorno() != Aerospike::OK) {
        throw new AerospikeDBException($this->oAerospike, __METHOD__.':'.$this->sNamespace.':'.$this->sSet);
    }

    $status = $newLList->add($aData);
    if ($status == \Aerospike\LDT\LList::ERR_BIN_DAMAGED) {
        throw new AerospikeDBException($this->oAerospike, __METHOD__.':'.$this->sNamespace.':'.$this->sSet, 'This bin already exists with not LDT data');
    } else if ($status != Aerospike::OK) {
        throw new AerospikeDBException($this->oAerospike, __METHOD__.':'.$this->sNamespace.':'.$this->sSet);
    }`

An the error was raised after ‘->add’. I also tried change to ‘addMany’ but the same error is raised.

Cheers,

Mònica.


#4

I can’t spot any mistake in your code. I’m sure that one of the AS engineers will take a look at your problem within the next days. Even if it’s not a bug in the php client or server code, there should be a validation on client-side for such cases… But the fact that it works when you insert into a second LList sounds like a strong hint to a bug, from my POV. Till then the only thing I can add here is that LDTs don’t have a limitation on the amount of elements other than what your namespace can physically host (obviously not an issue here). Keys are all positive, unique integers (and != 0) ? There used to be some issues with negative integers, i’m not sure whether those have already been fixed or not. Oh and I would try without the ‘errors’-data because I fear there might be a limit to the deepness/nesting (maybe only a map of lists or a list of maps allowed? Not sure. Might be worth trying).

Cheers, Manuel


#5

Hi Mònica,

The addMany() method currently only differs from add() due to some extra type checking, but you should use it for that purpose. If you are adding many map elements, those need to all have unique values in the key field. However, a uniqueness violation in the key field should trigger an Error 1402: LDT-Unique Key or Value Violation.

It would be useful to have a dump of those values, so that we can try and reproduce the problem. It may be a bug, it may be that the type of the keys is not consistent (they should all be the same type), or it might be something else. What is clear is that the error is happening on the server-side, not the client-side.

Anyway, please upload test data that triggers that error in your cluster.

Thanks, Ronen


#6

@Monica_Marquez and @ManuelSchmidt:

Thank you for posting about LDTs in our forum. Please see the LDT Feature Guide for current LDT recommendations and best practices.


#7

@Monica_Marquez and @ManuelSchmidt,

Effective immediately, we will no longer actively support the LDT feature and will eventually remove the API. The exact deprecation and removal timeline will depend on customer and community requirements. Instead of LDTs, we advise that you use our newer List and SortedMap APIs, which are now available in all Aerospike-supported clients at the General Availability level. Read our blog post for details.