# Device overload when map size is too big

**URL:** <https://discuss.aerospike.com/t/device-overload-when-map-size-is-too-big/5206>\
**Category:** Tuning\
**Tags:** map\
**Created:** [May 25, 2018, 7:20pm UTC](https://discuss.aerospike.com/t/device-overload-when-map-size-is-too-big/5206 "2018-05-25T19:20:01Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![buptljy](https://avatars.discourse-cdn.com/v4/letter/b/e19b73/32.png) [@buptljy](https://discuss.aerospike.com/u/buptljy)\
**Post date:** [May 25, 2018, 7:20pm UTC](https://discuss.aerospike.com/t/device-overload-when-map-size-is-too-big/5206/1 "2018-05-25T19:20:01Z")

</div>

We got “device overload” error after the program ran successfully on production for a few months. And we find that some maps’ sizes are very big, which may be bigger than 1,000.

After I inspected the source code, I found that the reason of “devcie overload” is that the write queue is beyond limitations, and the length of the write queue is related to the effiency of processing.

So I checked the “particle\_map” file, and I suspect that the whole map will be rewritten even if we just want to insert one pair of KV into the map.

But I am not so sure about this. Any advice ?

Cross-posted here: [Aerospike: Device Overload Error when size of map is too big - Stack Overflow](https://stackoverflow.com/questions/50535678/aerospike-device-overload-error-when-size-of-map-is-too-big)

---

<div class="post-metadata">

**Author:** ![kporter](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.aerospike.com/kporter/32/515_2.png) [@kporter](https://discuss.aerospike.com/u/kporter)\
**Post date:** [May 25, 2018, 8:42pm UTC](https://discuss.aerospike.com/t/device-overload-when-map-size-is-too-big/5206/2 "2018-05-25T20:42:17Z")

</div>

> So I checked the “particle\_map” file, and I suspect that the whole map will be rewritten even if we just want to insert one pair of KV into the map.

You are correct. When using persistence, Aerospike does not update records in-place. Each update/insert is buffered into an in-memory write-block which, when full, is queued to be written to disk. This queue allows for short bursts that exceed your disks max IO but if the burst is sustained for too long the server will begin to fail the writes with the ‘device overload’ error you have mentioned. How far behind the disk is allowed to get is controlled by the [max-write-cache](https://www.aerospike.com/docs/reference/configuration#max-write-cache) namespace storage-engine parameter.

You can find more about our storage layer at [https://www.aerospike.com/docs/architecture/index.html](https://www.aerospike.com/docs/architecture/index.html).

---

<div class="post-metadata">

**Author:** ![meher](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.aerospike.com/meher/32/1644_2.png) [@meher](https://discuss.aerospike.com/u/meher)\
**Post date:** [May 25, 2018, 10:25pm UTC](https://discuss.aerospike.com/t/device-overload-when-map-size-is-too-big/5206/3 "2018-05-25T22:25:52Z")

</div>

Take a look at this article as well: [https://discuss.aerospike.com/t/device-overload/733](https://discuss.aerospike.com/t/device-overload/733)

---

<div class="post-metadata">

**Author:** ![buptljy](https://avatars.discourse-cdn.com/v4/letter/b/e19b73/32.png) [@buptljy](https://discuss.aerospike.com/u/buptljy)\
**Post date:** [May 26, 2018, 1:37am UTC](https://discuss.aerospike.com/t/device-overload-when-map-size-is-too-big/5206/5 "2018-05-26T01:37:43Z")

</div>

But am I right about the single put of a map ? Does aerospike really need to rewrite the whole map even if we just want to put one pair key-value?

---

<div class="post-metadata">

**Author:** ![meher](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.aerospike.com/meher/32/1644_2.png) [@meher](https://discuss.aerospike.com/u/meher)\
**Post date:** [May 28, 2018, 11:49pm UTC](https://discuss.aerospike.com/t/device-overload-when-map-size-is-too-big/5206/6 "2018-05-28T23:49:30Z")

</div>

Yes. Records always have to be written contiguously… would not be possible to efficiently read them otherwise…
