# Swaping memory to disk with large datasets **URL:** \ **Category:** General Discussion\ **Created:** [September 24, 2017, 11:41am UTC](https://discuss.aerospike.com/t/swaping-memory-to-disk-with-large-datasets/4645 "2017-09-24T11:41:54Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![Juliano\_Galhiego\_Vie](https://avatars.discourse-cdn.com/v4/letter/j/8491ac/32.png) [@Juliano\_Galhiego\_Vie](https://discuss.aerospike.com/u/Juliano_Galhiego_Vie)\ **Post date:** [September 24, 2017, 11:41am UTC](https://discuss.aerospike.com/t/swaping-memory-to-disk-with-large-datasets/4645/1 "2017-09-24T11:41:54Z")
Hi guys. I have a case where I need to access my data (+20 billions) through 2 secondary indexes (user-id and global-id) and it wont be accessed very often and it doesn’t need to be very fast. My problem is that I won’t have enough memory to store these indexes as each index needs 64B. As I dont need it to be very fast I was thinking about configuring my memory to swap to disk so I can have all the memory I need to store my indexes. Do you guys think its a good idea? ---
**Author:** ![Albot](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.aerospike.com/albot/32/2076_2.png) [@Albot](https://discuss.aerospike.com/u/Albot)\ **Post date:** [September 24, 2017, 4:41pm UTC](https://discuss.aerospike.com/t/swaping-memory-to-disk-with-large-datasets/4645/2 "2017-09-24T16:41:32Z")
If you mean you’re opting to run aerospike on SSD’s rather than data-in-memory than yeah it works great if you have good disks. Another couple of thoughts are - You could model your data and write reference sets so that you do not need indexes - If they are not accessed frequently and do not need to be quick, you could opt to use scans/predicate scans instead of queries The best way to know what fits you best of course would be to test your use cases against the different setups and compare! 🙂 ---
**Author:** ![Juliano\_Galhiego\_Vie](https://avatars.discourse-cdn.com/v4/letter/j/8491ac/32.png) [@Juliano\_Galhiego\_Vie](https://discuss.aerospike.com/u/Juliano_Galhiego_Vie)\ **Post date:** [October 7, 2017, 5:11pm UTC](https://discuss.aerospike.com/t/swaping-memory-to-disk-with-large-datasets/4645/3 "2017-10-07T17:11:28Z")
Hi @AlbotWhat do you mean when u say “write reference sets so that you do not need indexes”? I did a little search about it but had no success, do you know any doc talking about this? Thanks for your support. ---
**Author:** ![Albot](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.aerospike.com/albot/32/2076_2.png) [@Albot](https://discuss.aerospike.com/u/Albot)\ **Post date:** [October 9, 2017, 4:02pm UTC](https://discuss.aerospike.com/t/swaping-memory-to-disk-with-large-datasets/4645/4 "2017-10-09T16:02:10Z")
Say you’re writing a table to store stock symbol information. Let’s say you have a record like this: PK,SymbolName,SymbolType,Price Let’s assume the PK is some symbolID we can use for our primary use case. Getting the data is easy, you just form a PK and run client.get. In some use cases though you would need ability to lookup by SymbolName or SymbolType. Depending on the cardinality and use case, you may choose a variety of different methods. The easiest is of course secondary index. Another method to do this would be to write a reference record. ex… setSymbols may contain records like this: PK,SymbolName,SymbolType,Price. Lets assume we are looking at 1 record… with values “1234,GOOG,F,123” Now if we need a way to look this record up from not just the PK (1234) but also the SymbolName (GOOG) we can write a reference set. setSymbolReference: PK,SymbolID… continuing this example, we could have the values of the record we need to lookup goog as “GOOG, 1234”. So step 1 is to lookup the reference key which points to the real primary key, step 2 is to get the actual data. 1. get(GOOG) =\> 1234 2. get(1234) =\> data! The pitfall is that you need to update 2 records whenever 1 update comes through.