How To Skip A Set With XDR Filter Expressions

Knowledge Base XDR
1

The Aerospike Knowledge Base has moved to https://support.aerospike.com. Content on https://discuss.aerospike.com is being migrated to either https://support.aerospike.com or https://docs.aerospike.com. Maintenance on articles stored in this repository ceased on December 31st 2022 and this article may be stale. If you have any questions, please do not hesitate to raise a case via https://support.aerospike.com.

How To Skip A Set With XDR Filter Expressions

Context

When XDR is not able to ship a specific set successfully to a destination due to some issue on the destination (typically when leveraging a connector where different sets could be shipped to different routes/backends), the set would be retried continuously on the XDR side.

It will block all the transactions behind this and XDR lag would keep increasing. We may want to skip the specific set causing the shipping failure to allow other sets to progress.

As of version 5.7, the ignore-set configuration would not work in this case as it will not apply to the transactions on the retry path and only applies to the transaction that occurred after the parameter was set.

Skip a set from XDR retry path with XDR Filter Expressions

Generate the base64 of the XDR filter with java code for set ‘set1’

FilterBase.java

import java.util.Base64;
import com.aerospike.client.exp.Exp;
import com.aerospike.client.exp.Expression;
public class FilterBase {
    public static void main(String args[]) {
        Expression filter = Exp.build(Exp.not(Exp.eq(Exp.setName(), Exp.val("set1"))));
        System.out.println(filter.getBase64());
    }
}

Run the code and get base64 of filter expression

# javac -cp .:java/aerospike-client-java-5.1.8/client/target/aerospike-client-5.1.8.jar:java/aerospike-client-java-5.1.8/client/target/aerospike-client-5.1.8-jar-with-dependencies.jar FilterBase.java 
# java -cp .:java/aerospike-client-java-5.1.8/client/target/aerospike-client-5.1.8.jar:java/aerospike-client-java-5.1.8/client/target/aerospike-client-5.1.8-jar-with-dependencies.jar FilterBase 
khKTAZFGpQNzZXQx

Run the asinfo command on the source cluster using the above base64 output of filter expression

root@2b70591c2f28:/# asinfo -v "xdr-set-filter:dc=dc2;namespace=test;exp=khKTAZFGpQNzZXQx"
ok
root@2b70591c2f28:/# asinfo -v "xdr-get-filter:dc=dc2;namespace=test"
namespace=test:exp=not(eq(set_name(), <string#4>))

XDR will stop shipping the set1 transactions from the retry path and will process transactions on other sets. It would also skip any new transactions for set1 as well.

Notes

The xdr filter can be set directly from Java code without using the asinfo command:

import com.aerospike.client.AerospikeClient;
import com.aerospike.client.AerospikeException;
import com.aerospike.client.command.ParticleType;
import com.aerospike.client.exp.Exp;
import com.aerospike.client.exp.Expression;

public class ExpressionTest {

    public static void main(String args[]) throws AerospikeException {

        AerospikeClient client = new AerospikeClient("172.17.0.4", 3000);

        Expression filter = Exp.build(Exp.not(Exp.eq(Exp.setName(), Exp.val("set1"))));

        try {
            client.setXDRFilter(null, "dc2", "test", filter);
        } catch (AerospikeException e) {
            System.out.println("Failed to set filter " + e.getMessage());
        }

        client.close();
    }

Applies To

Aerospike version 5.3 and later

Keywords

SKIP SET RETRY PATH XDR FILTER EXPRESSIONS

Timestamp

October 2021