# Scanning all records in a set

**URL:** <https://discuss.aerospike.com/t/scanning-all-records-in-a-set/4667>\
**Category:** PHP Client Library\
**Created:** [October 2, 2017, 11:52am UTC](https://discuss.aerospike.com/t/scanning-all-records-in-a-set/4667 "2017-10-02T11:52:36Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Konstantin\_Mirin](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.aerospike.com/konstantin_mirin/32/911_2.png) [@Konstantin\_Mirin](https://discuss.aerospike.com/u/Konstantin_Mirin)\
**Post date:** [October 2, 2017, 11:52am UTC](https://discuss.aerospike.com/t/scanning-all-records-in-a-set/4667/1 "2017-10-02T11:52:37Z")

</div>

Hi, I have around 6M records in a set. I would like to loop through them and perform certain actions (use as a base record to create an index elsewhere) In the first place, I am getting a timeout error. OK, lets increase timeout. 300 seconds. Still failing after ~30s. OK, let’s try concurrent scanning. Still failing with the timeout error OK, let’s ignore bins - bingo, it’s scanning through the records, but… how do I get an actual data? Accumulate keys in batch and use getMany?

Is this an intended way of handling things? Any other alternatives?

---

<div class="post-metadata">

**Author:** ![Konstantin\_Mirin](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.aerospike.com/konstantin_mirin/32/911_2.png) [@Konstantin\_Mirin](https://discuss.aerospike.com/u/Konstantin_Mirin)\
**Post date:** [October 2, 2017, 11:57am UTC](https://discuss.aerospike.com/t/scanning-all-records-in-a-set/4667/2 "2017-10-02T11:57:28Z")

</div>

OK, it failed miserably even in that case: An error occured while scanning[9] Timeout: timeout=999999 iterations=1 failedNodes=0 failedConns=0 3562595 records found

So… How do I fetch ALL my records?

---

<div class="post-metadata">

**Author:** ![Albot](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.aerospike.com/albot/32/2076_2.png) [@Albot](https://discuss.aerospike.com/u/Albot)\
**Post date:** [October 2, 2017, 11:57pm UTC](https://discuss.aerospike.com/t/scanning-all-records-in-a-set/4667/3 "2017-10-02T23:57:05Z")

</div>

Are you trying to do this through AQL? What is the server/client version? How many hops between client calling application and aerospike server? Does the client system show its hitting upper limits of cpu/memory/network?

---

<div class="post-metadata">

**Author:** ![kporter](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.aerospike.com/kporter/32/515_2.png) [@kporter](https://discuss.aerospike.com/u/kporter)\
**Post date:** [October 3, 2017, 12:16am UTC](https://discuss.aerospike.com/t/scanning-all-records-in-a-set/4667/4 "2017-10-03T00:16:40Z")

</div>

1. What version of the server?
2. Which error is raised by the client?

---

<div class="post-metadata">

**Author:** ![Konstantin\_Mirin](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.aerospike.com/konstantin_mirin/32/911_2.png) [@Konstantin\_Mirin](https://discuss.aerospike.com/u/Konstantin_Mirin)\
**Post date:** [October 3, 2017, 11:29am UTC](https://discuss.aerospike.com/t/scanning-all-records-in-a-set/4667/5 "2017-10-03T11:29:18Z")

</div>

Latest PHP client, not AQL. No hops, they are on the same network (same AWS zone, same VPC)

I use r3.4xlarge instance which should be good enough. No replication at the moment.

3.14 server.

Error is shown above:

`An error occured while scanning[9] Timeout: timeout=999999 iterations=1 failedNodes=0 failedConns=0`

I tried querying the same aerospike set using node.js and it worked like a charm - all 8M records were processed with no issues.

---

<div class="post-metadata">

**Author:** ![meher](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.aerospike.com/meher/32/1644_2.png) [@meher](https://discuss.aerospike.com/u/meher)\
**Post date:** [October 4, 2017, 3:32am UTC](https://discuss.aerospike.com/t/scanning-all-records-in-a-set/4667/6 "2017-10-04T03:32:41Z")

</div>

Could be the PHP client and your code is taking too much time to process and causing the socket to timeout.

With server 3.14, you should be able to set the socket timeout for scan. This is possible since version 3.12.1:

[AER-5510] - (SCAN) Write idle-time-out now configurable from client.

I am not sure if the PHP client supports setting this property on scan, but I would hope it does. Otherwise, you typically have only a 10 second timeout at the socket level after which the server will close the connection. I believe this is different than the read timeout you are passsing. (You can confirm this is happening by checking the server side logs).

---

<div class="post-metadata">

**Author:** ![Konstantin\_Mirin](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.aerospike.com/konstantin_mirin/32/911_2.png) [@Konstantin\_Mirin](https://discuss.aerospike.com/u/Konstantin_Mirin)\
**Post date:** [October 4, 2017, 1:20pm UTC](https://discuss.aerospike.com/t/scanning-all-records-in-a-set/4667/7 "2017-10-04T13:20:21Z")

</div>

Thank you. I’ve coded a prototype in node.js that I’m happy with - consistent read speed and no lost WRITEs.

So probably I have to blame PHP implementation for Aerospike, it didn’t work for my case…

---

<div class="post-metadata">

**Author:** ![rbotzer](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.aerospike.com/rbotzer/32/2114_2.png) [@rbotzer](https://discuss.aerospike.com/u/rbotzer)\
**Post date:** [October 5, 2017, 6:11pm UTC](https://discuss.aerospike.com/t/scanning-all-records-in-a-set/4667/8 "2017-10-05T18:11:06Z")

</div>

[Release 7.0.2](https://github.com/aerospike/aerospike-client-php/releases/tag/7.0.2) of the [PHP client](https://packagist.org/packages/aerospike/aerospike-client-php) adds the necessary `Aerospike::OPT_SOCKET_TIMEOUT` param to the options for the query and scan methods.
