Presence in `user-path` does not suffice for udf registration

[avi]

for example i am installing aerospike via the helm chart and modifying the templates to mount a configmap with the udf’s .lua file, i see the .udf in /opt/aerospike/usr/udf/lua but show modules in aql returns none. is this the intended behavior? at least in the docker image’s docs Docker Hub in the Persistent Lua Cache section it implies that the presence of those files would suffice. is there an admin command i can use to sync them to the user-path, so i don’t have to do register module’s through aql?

[avi]

also when i add this postStart to the lifecycle of the stateful set

postStart:
  exec:
    command:
    - echo "register module '/opt/aerospike/usr/udf/lua/profile.lua'" | aql

the pod fails to start and i see this from the container logs

WARNING (udf): (udf_cask.c:645) Failed to remove the file /opt/aerospike/usr/udf/lua/profile.lua. Error 16

this does not happen without the postStart, the pod starts fine, and manually running the above command inside the pod successfully registers the udf

edit: actually this is complaining because aql isn’t in the stateful set pod?

Exec lifecycle hook ([echo "register module '/opt/aerospike/usr/udf/lua/profile.lua'" | aql]) for Container "aerospike" in Pod "aerospike-aerospike-0_aerospike(ab5ff606-82dd-4f9f-b069-61c50d8c3b70)" failed - error: rpc error: code = Unknown desc = failed to exec in container: failed to start exec "d99baf07d0deed55a0ace0f5ec7f4a572bd6a412a3cc36a182e7aff7019061cc": OCI runtime exec failed: exec failed: container_linux.go:380: starting container process caused: exec: "echo \"register module '/opt/aerospike/usr/udf/lua/profile.lua'\" | aql": stat echo "register module '/opt/aerospike/usr/udf/lua/profile.lua'" | aql: no such file or directory: unknown, message: ""

[aanderson]

Hi @avi, based on our documentation this path should be writable, but a config map mounted in Kubernetes is not writable.

You could try using an empty dir volume for “opt/aerospike/usr/udf/lua” and via a custom init container, copy the contents of the configmap to the final destination directory.

[avi]

this didn’t exactly work for me, aerospike wasn’t auto registering the module even with the empty dir strategy, so i just ran register module on it, here’s my ytt overlay for posterity

#@ load("@ytt:overlay", "overlay")

#@overlay/match by=overlay.subset({"kind": "StatefulSet"})
---
spec:
  template:
    spec:
      initContainers:
      - name: copy-udfs
        image: busybox:1.35
        command: ["/bin/sh", "-c", "cp /udfs/profile.lua /temp/"]
        volumeMounts:
        - name: udf-volume
          mountPath: /udfs/
          subpath: profile.lua
        - name: temp
          mountPath: /temp/
      containers:
      #@overlay/match by=overlay.all
      #@overlay/match-child-defaults missing_ok=True
      - lifecycle:
          postStart:
            exec:
              command: ["/bin/sh", "-c", "sleep 20 && echo \"register module '/temp/profile.lua'\" | aql && asinfo -v \"set-config:context=namespace;id=test;nsup-period=600\""]
      #@overlay/match by=overlay.all
      - volumeMounts:
        #@overlay/append
        - name: temp
          mountPath: /temp/
      volumes:
      #@overlay/append
      - name: udf-volume
        configMap:
          name: udf-config
      - name: temp
        emptyDir: {}