Turning on keysend in Python client

Client Libraries Python Client
1

Hi,

A very basic question/confirmation in Python.

In Spark if I want to send key to the set, I do the following:

     spark.conf.set("aerospike.sendKey", true)

What is the equivalent of the above setting in Python client please?

      config = {
          'hosts': [(self.dbHost, self.dbPort)],
          'policy': {'aerospike.POLICY_KEY_SEND': 'true', 'read': {'total_timeout': 1000}}
      }

Is above correct please and I should be able to see it in AQL through

aql> set key_send true
KEY_SEND = true
aql> --set output json
aql> set record_print_metadata true
RECORD_PRINT_METADATA = true
aql> select * from test.oracletoaerospike2
+--------------+------------+-----------+--------------+------------------------------------------------------+-----------+--------+--------------------------------+----------------------+---------+-------+
| SMALL_VC     | RANDOMISED | SCATTERED | PADDING      | RANDOM_STRING                                        | CLUSTERED | ID     | {edigest}                      | {set}                | {ttl}   | {gen} |
+--------------+------------+-----------+--------------+------------------------------------------------------+-----------+--------+--------------------------------+----------------------+---------+-------+
| "        10" | "8.0"      | "9.0"     | "xxxxxxxxxx" | "InblvsVdzHVVPFElOsSWtRLkqAJwytraTSsWMoEaoYNAuAoIrI" | "0.0"     | "10.0" | "mlCmGc4yuYN8dXpgAY1bvD4SM4g=" | "oracletoaerospike2" | 2587973 | 27    |
| "         5" | "31.0"     | "4.0"     | "xxxxxxxxxx" | "vDsFoYAOcitwrWNXCxPHzIIIxwKpTlrsVjFFKUDivytqJqOHGA" | "0.0"     | "5.0"  | "4kL+GIrLingKfKDNRb1MiS1Pdrw=" | "oracletoaerospike2" | 2587973 | 27    |
| "         1" | "63.0"     | "0.0"     | "xxxxxxxxxx" | "rMLTDXxxqXOZnqYRJwInlGfGBTxNkAszBGEUGELqTSRnFjRGbi" | "0.0"     | "1.0"  | "mFOCvMwz3qrt7kpY/2+u7Pmui34=" | "oracletoaerospike2" | 2587973 | 27    |
| "         2" | "926.0"    | "1.0"     | "xxxxxxxxxx" | "UEDJsfIgoYqwreSuuvjIcPZarpxMdCthpDCsgPlJfvIiylLiBS" | "0.0"     | "2.0"  | "QwYTCU5usJWwB4J/uwTnlEWGNRY=" | "oracletoaerospike2" | 2587973 | 27    |
| "         4" | "294.0"    | "3.0"     | "xxxxxxxxxx" | "HsLQJstfFDYSqtSIMfRDoLDpgfpxSqAQmToFMEwzmcqEIkHzmG" | "0.0"     | "4.0"  | "V8Y9JtXqNZpzqERzOcMcHJQTAjI=" | "oracletoaerospike2" | 2587973 | 27    |
| "         6" | "291.0"    | "5.0"     | "xxxxxxxxxx" | "yvOKszDuIKsHySzCCrQFyvIyJyWQDwZnPABkVnzNVMKgNxvsBH" | "0.0"     | "6.0"  | "oz2w8WA+1/XG3fWTrbPlG7Wjav0=" | "oracletoaerospike2" | 2587973 | 27    |
| "         7" | "391.0"    | "6.0"     | "xxxxxxxxxx" | "FCJNHulbvwdhXKusUFzPXEbmLtEqKFrWGFnvJlcgSQhkgHDASw" | "0.0"     | "7.0"  | "IQCwhQXCNC+8odHBC6U6JwgbUiE=" | "oracletoaerospike2" | 2587973 | 27    |
| "         8" | "216.0"    | "7.0"     | "xxxxxxxxxx" | "QFiGRwNERAcxCCQcgwXZmVgADLfneqPGPoyoCTrMKOajUhKRce" | "0.0"     | "8.0"  | "rvCgRW7Qcvcc9vTcXdxjHaiDc9I=" | "oracletoaerospike2" | 2587973 | 27    |
| "         3" | "504.0"    | "2.0"     | "xxxxxxxxxx" | "jrIWMuAWViCOdbspYtDDZrsqaBlItIXvUKQujEYOPRzmGoxGrp" | "0.0"     | "3.0"  | "k/FnJZNfA3IfjRhqrWs0zt78rVc=" | "oracletoaerospike2" | 2587973 | 27    |
| "         9" | "549.0"    | "8.0"     | "xxxxxxxxxx" | "nodLTROuwmdLhpalFVIprLxIHDrRgqHbTTbahCskowCuzpMoqq" | "0.0"     | "9.0"  | "SmryZ8Ciaqqzo8ixPckB1d2aZ8o=" | "oracletoaerospike2" | 2587973 | 27    |
+--------------+------------+-----------+--------------+------------------------------------------------------+-----------+--------+--------------------------------+----------------------+---------+-------+
10 rows in set (0.093 secs)

The bin “ID” is the primary key. But it is not shown as primary key

If I try to use spark to read the set, I get

   val dfRead  = sqlContext.read.
      format("com.aerospike.spark.sql").
      option("aerospike.batchMax", 10000).
      load
dfRead.select("*").orderBy("ID").show(1,false)

+-----+-------------------------------------------------------------+---------+------------+-------+---+--------------------------------------------------+---------+----------+----------+----------+---------+
|__key|__digest                                                     |__expiry |__generation|__ttl  |ID |RANDOM_STRING                                     |CLUSTERED|SMALL_VC  |PADDING   |RANDOMISED|SCATTERED|
+-----+-------------------------------------------------------------+---------+------------+-------+---+--------------------------------------------------+---------+----------+----------+----------+---------+
|null |[98 53 82 BC CC 33 DE AA ED EE 4A 58 FF 6F AE EC F9 AE 8B 7E]|314398046|27          |2587494|1.0|rMLTDXxxqXOZnqYRJwInlGfGBTxNkAszBGEUGELqTSRnFjRGbi|0.0      |         1|xxxxxxxxxx|63.0      |0.0      |
+-----+-------------------------------------------------------------+---------+------------+-------+---+--------------------------------------------------+---------+----------+----------+----------+---------+
only showing top 1 row

So Spark brings all bins back but __key is null!

Obviously key is not stored through Python?

Thanks

2

Hi Mich,

You can enable keysend in the python client with the following config.

write_policy = {"key": aerospike.POLICY_KEY_SEND}
policies = {"write": write_policy}

config = {
    "hosts": [("127.0.0.1", 3000)],
    "policies": policies
}

Here is a short example using the python client that will write one record and store its primary key.

from __future__ import print_function
import aerospike
from aerospike import exception as ex
import sys

# Configure the client.
write_policy = {"key": aerospike.POLICY_KEY_SEND}
policies = {"write": write_policy}

config = {
    "hosts": [("127.0.0.1", 3000)],
    "policies": policies
}

# Create a client and connect it to the cluster.
try:
    client = aerospike.client(config).connect()
except ex.ClientError as e:
    print("Error: {0} [{1}]".format(e.msg, e.code))
    sys.exit(1)

TEST_NS = "test"
TEST_SET = "key_demo"
record = {"name": "John", "id": 100}
key = (TEST_NS, TEST_SET, 3)

# Write the record.
try:
    client.put(key, record)
except ex.RecordError as e:
    print("Error: {0} [{1}]".format(e.msg, e.code))

# Close the connection to the Aerospike cluster.
client.close()

Using aql we can see that the primary key was stored.

aql> set record_print_metadata true
RECORD_PRINT_METADATA = true
aql> select * from test.key_demo
+----+--------+-----+--------------------------------+------------+---------+-------+
| PK | name   | id  | {edigest}                      | {set}      | {ttl}   | {gen} |
+----+--------+-----+--------------------------------+------------+---------+-------+
| 3  | "John" | 100 | "5NoZ7P+jiB26nMfpQQi5Tp5kXZ8=" | "key_demo" | 2591964 | 1     |
+----+--------+-----+--------------------------------+------------+---------+-------+
1 row in set (0.137 secs)

OK

aql>

Note that the above config instructs the python client to send keys for all write operations. If you would rather specify sendkeys for individual operations, you can supply a similar policy to any operation that accepts write policies. For example,

write_policy = {"key": aerospike.POLICY_KEY_SEND}

TEST_NS = "test"
TEST_SET = "key_demo"
record = {"name": "John", "id": 100}
key = (TEST_NS, TEST_SET, 3)

client.put(key, record, policy=write_policy)

I hope this helps! :slight_smile:

For more info see.

client example that sets write policies

list of client policies

3

Hi Dylan,

Thanks. That worked with PK displayed.

write_policy = {'key': aerospike.POLICY_KEY_SEND}
policies = {'write': write_policy, 'total_timeout': 1000}
config = {
  'hosts': [(v.dbHost, v.dbPort)],
  'policies': policies
}
client = aerospike.client(config).connect(v.dbConnection, v.dbPassword)

aql> set key_send true
KEY_SEND = true
aql> --set output json
aql> set record_print_metadata true
RECORD_PRINT_METADATA = true
aql> select * from test.oracletoaerospike2
+------+--------------+------------+-----------+--------------+------------------------------------------------------+-----------+--------+--------------------------------+----------------------+---------+-------+
| PK   | SMALL_VC     | RANDOMISED | SCATTERED | PADDING      | RANDOM_STRING                                        | CLUSTERED | ID     | {edigest}                      | {set}                | {ttl}   | {gen} |
+------+--------------+------------+-----------+--------------+------------------------------------------------------+-----------+--------+--------------------------------+----------------------+---------+-------+
| "11" | "        10" | "8.0"      | "9.0"     | "xxxxxxxxxx" | "InblvsVdzHVVPFElOsSWtRLkqAJwytraTSsWMoEaoYNAuAoIrI" | "0.0"     | "10.0" | "mlCmGc4yuYN8dXpgAY1bvD4SM4g=" | "oracletoaerospike2" | 2591803 | 20    |
| "6"  | "         5" | "31.0"     | "4.0"     | "xxxxxxxxxx" | "vDsFoYAOcitwrWNXCxPHzIIIxwKpTlrsVjFFKUDivytqJqOHGA" | "0.0"     | "5.0"  | "4kL+GIrLingKfKDNRb1MiS1Pdrw=" | "oracletoaerospike2" | 2591803 | 20    |
| "2"  | "         1" | "63.0"     | "0.0"     | "xxxxxxxxxx" | "rMLTDXxxqXOZnqYRJwInlGfGBTxNkAszBGEUGELqTSRnFjRGbi" | "0.0"     | "1.0"  | "mFOCvMwz3qrt7kpY/2+u7Pmui34=" | "oracletoaerospike2" | 2591803 | 20    |
| "3"  | "         2" | "926.0"    | "1.0"     | "xxxxxxxxxx" | "UEDJsfIgoYqwreSuuvjIcPZarpxMdCthpDCsgPlJfvIiylLiBS" | "0.0"     | "2.0"  | "QwYTCU5usJWwB4J/uwTnlEWGNRY=" | "oracletoaerospike2" | 2591803 | 20    |
| "5"  | "         4" | "294.0"    | "3.0"     | "xxxxxxxxxx" | "HsLQJstfFDYSqtSIMfRDoLDpgfpxSqAQmToFMEwzmcqEIkHzmG" | "0.0"     | "4.0"  | "V8Y9JtXqNZpzqERzOcMcHJQTAjI=" | "oracletoaerospike2" | 2591803 | 20    |
| "7"  | "         6" | "291.0"    | "5.0"     | "xxxxxxxxxx" | "yvOKszDuIKsHySzCCrQFyvIyJyWQDwZnPABkVnzNVMKgNxvsBH" | "0.0"     | "6.0"  | "oz2w8WA+1/XG3fWTrbPlG7Wjav0=" | "oracletoaerospike2" | 2591803 | 20    |
| "8"  | "         7" | "391.0"    | "6.0"     | "xxxxxxxxxx" | "FCJNHulbvwdhXKusUFzPXEbmLtEqKFrWGFnvJlcgSQhkgHDASw" | "0.0"     | "7.0"  | "IQCwhQXCNC+8odHBC6U6JwgbUiE=" | "oracletoaerospike2" | 2591803 | 20    |
| "9"  | "         8" | "216.0"    | "7.0"     | "xxxxxxxxxx" | "QFiGRwNERAcxCCQcgwXZmVgADLfneqPGPoyoCTrMKOajUhKRce" | "0.0"     | "8.0"  | "rvCgRW7Qcvcc9vTcXdxjHaiDc9I=" | "oracletoaerospike2" | 2591803 | 20    |
| "4"  | "         3" | "504.0"    | "2.0"     | "xxxxxxxxxx" | "jrIWMuAWViCOdbspYtDDZrsqaBlItIXvUKQujEYOPRzmGoxGrp" | "0.0"     | "3.0"  | "k/FnJZNfA3IfjRhqrWs0zt78rVc=" | "oracletoaerospike2" | 2591803 | 20    |
| "10" | "         9" | "549.0"    | "8.0"     | "xxxxxxxxxx" | "nodLTROuwmdLhpalFVIprLxIHDrRgqHbTTbahCskowCuzpMoqq" | "0.0"     | "9.0"  | "SmryZ8Ciaqqzo8ixPckB1d2aZ8o=" | "oracletoaerospike2" | 2591803 | 20    |
+------+--------------+------------+-----------+--------------+------------------------------------------------------+-----------+--------+--------------------------------+----------------------+---------+-------+
10 rows in set (0.098 secs)

OK
1 Like