Aerospike go client is using huge amount of memory, especially in “newPartitions” function (I don’t know what it is). How can i reduce memory usage. I am using only one “Client Objest” in golang code, which is initialized in main function.
Our cluster size is below.
cluster: 1
cluster nodes: 6
machine-type: CPU 2cores, Memory 15GB
ConnectionQueueSize = 512
paxos-single-replica-limit 1
pidfile /var/run/aerospike/asd.pid
service-threads 8
transaction-queues 8
transaction-threads-per-queue 4
proto-fd-max 30000
batch-threads 8
~ # go tool pprof main http://localhost:6000/debug/pprof/heap
Fetching profile from http://localhost:6000/debug/pprof/heap
Saved profile in /root/pprof/pprof.main.localhost:6000.alloc_objects.alloc_space.inuse_objects.inuse_space.003.pb.gz
Entering interactive mode (type "help" for commands)
(pprof) top
11383.14MB of 11600.15MB total (98.13%)
Dropped 875 nodes (cum <= 58MB)
Showing top 10 nodes out of 52 (cum >= 135.40MB)
flat flat% sum% cum cum%
7680.93MB 66.21% 66.21% 7680.93MB 66.21% github.com/my-app/vendor/github.com/aerospike/aerospike-client-go.newPartitions
3095.59MB 26.69% 92.90% 3100.05MB 26.72% github.com/my-app/vendor/github.com/aerospike/aerospike-client-go.NewConnection
321.09MB 2.77% 95.67% 321.09MB 2.77% github.com/my-app/vendor/github.com/aerospike/aerospike-client-go.clonePartitions
120.36MB 1.04% 96.71% 120.36MB 1.04% github.com/my-app/vendor/github.com/aerospike/aerospike-client-go.newSingleConnectionQueue
67.95MB 0.59% 97.29% 84.84MB 0.73% github.com/my-app/vendor/github.com/aerospike/aerospike-client-go.(*info).parseMultiResponse
40.12MB 0.35% 97.64% 375.11MB 3.23% github.com/my-app/vendor/github.com/aerospike/aerospike-client-go.(*Cluster).tend
17.13MB 0.15% 97.78% 78.83MB 0.68% github.com/my-app/vendor/github.com/aerospike/aerospike-client-go.(*info).sendCommand
16.90MB 0.15% 97.93% 167.02MB 1.44% github.com/my-app/vendor/github.com/aerospike/aerospike-client-go.parsePeers
13.67MB 0.12% 98.05% 103.91MB 0.9% github.com/my-app/vendor/github.com/aerospike/aerospike-client-go.newInfo
9.40MB 0.081% 98.13% 135.40MB 1.17% github.com/my-app/vendor/github.com/aerospike/aerospike-client-go.newNode
(pprof) list github.com/my-app/vendor/github.com/aerospike/aerospike-client-go.newPartitions
Total: 11.33GB
ROUTINE ======================== github.com/my-app/vendor/github.com/aerospike/aerospike-client-go.newPartitions in /go/src/github.com/my-app/vendor/github.com/aerospike/aerospike-client-go/partitions.go
7.50GB 7.50GB (flat, cum) 66.21% of Total
Error: open /go/src/github.com/my-app/vendor/github.com/aerospike/aerospike-client-go/partitions.go: no such file or directory
(pprof) list github.com/my-app/vendor/github.com/aerospike/aerospike-client-go.newPartitionParser
Total: 11.33GB
