|
virtmem
virtual memory library for Arduino
|
|
virtmem
virtual memory library for Arduino
|
virtmem is an Arduino library that allows your project to easily use an external memory source to extend the (limited) amount of available RAM. This library supports several memory resources, for instance, SPI ram (e.g. the 23LC1024 chip from Microchip), an SD card or even a computer connected via a serial connection. The library is made in such a way that managing and using this virtual memory closely resembles working with data from 'normal' memory.
Before delving into specifics, here is a simple example to demonstrate how virtmem works and what it can do.
This Arduino sketch demonstrates how to use a SD card as virtual memory store. By using a virtual memory pointer wrapper class, using virtual memory becomes quite close to using data residing in 'normal' memory.
More examples can be found on the examples page.
As the name suggests, virtmem works similar to virtual memory management found on computers.
The library uses a paging system to work efficiently with virtual memory. A memory page is a static buffer that resides in regular RAM and contains a copy of a part of the total virtual memory pool. Multiple pages are typically used (usually four), where each page contains a copy of a different virtual memory region. Similar to data allocated from the heap, data access from virtual memory happens through a virtual pointer.
Whenever a virtual pointer is dereferenced, the library first checks whether the requested data resides in one of the memory pages, and if not, the library will copy the data from virtual memory to a suitable memory page. All data access now happens through this memory page.
When virtual memory has to be copied to a memory page and no pages are free, the library will first have to free a page. During this process any modified data will be written back to the virtual memory pool and the requested data will be loaded to the page. This process is sometimes called swapping. Swapping a page is generally a relative time consuming process, since a large amount of data has to be transferred for example over SPI. To minimize swapping, the library always tries to free pages that were not recently loaded in (FIFO like). The time spent on swapping is further reduced by having multiple memory pages and only writing out data that was modified.
Because memory pages reside in regular RAM, (repeated) data access to paged memory is quite fast.
The virtmem/ subdirectory contains all the code needed to compile Arduino sketches, and should therefore be copied as any other library. The file layout follows the new Arduino library specification.
Other files include documentation and code for testing the library, and are not needed for compilation. A summarized overview is given below.
| Directory | Contents |
|---|---|
| <root> | Internal development and some docs |
| benchmark/ | Internal code for benchmarking |
| doc/ | Files used for Doxygen created documentation |
| doc/html/ | This manual |
| gtest/ and test/ | Code for internal testing |
| virtmem/ | Library code. This directory needs to be copied to your libraries folder. |
| virtmem/extras/ | Contains python scripts needed for the serial memory allocator. |
Virtual memory in virtmem is managed by a virtual memory allocator. These are C++ template classes which are responsible for allocating and releasing virtual memory and contain the datablocks utilized for memory pages. Most of this functionality is defined in the virtmem::BaseVAlloc and virtmem::VAlloc classes. In addition to this, several allocator classes are derived from these base classes that actually implement the code necessary to deal with virtual memory (e.g. reading and writing data). For example, the class virtmem::SDVAlloc implements an allocator that uses an SD card as a virtual memory pool.
The virtmem library supports the following allocators:
| Allocator | Description | Header |
|---|---|---|
| virtmem::SDVAllocP | Uses a FAT formatted SD card as memory pool. Requires SD fat library. | #include <alloc/sd_alloc.h> |
| virtmem::SPIRAMVAllocP | Uses SPI ram (Microchip's 23LC series) as memory pool. Requires serialram library. | #include <alloc/spiram_alloc.h> |
| virtmem::MultiSPIRAMVAllocP | Like virtmem::SPIRAMVAlloc, but supports multiple memory chips. | #include <alloc/spiram_alloc.h> |
| virtmem::SerialVAllocP | Uses RAM from a computer connected through serial as memory pool. The computer should run the virtmem/extras/serial_host.py Python script. | #include <alloc/serial_alloc.h> |
| virtmem::StaticVAllocP | Uses regular RAM as memory pool (for debugging). | #include <alloc/static_alloc.h> |
| virtmem::StdioVAllocP | Uses files through regular stdio functions as memory pool (for debugging purposes on PCs). | #include <alloc/stdio_alloc.h> |
The following code demonstrates how to setup a virtual memory allocator:
To use the virtmem library you should include the virtmem.h header file. Furthermore, the specific header file of the allocator has to be included (alloc/sd_alloc.h, see table above). Finally, since some allocators depend on other libraries, they also may need to be included (SdFat.h in this example).
All classes, functions etc. of the virtmem library resides in the virtmem namespace. If you are unfamiliar with namespaces, you can find some info here. This is purely for 'organizational purposes', however, for small programs it may be easier to simply pull the virtmem namespace in the global namespace:
All allocator classes are singleton, meaning that only one (global) instance can be defined (however, instances of different allocators can co-exist, see Multiple allocators). Before the allocator can be used it should be initialized by calling its start function. Please note that, since this example uses the SD fat lib allocator, SD fat lib has to be initialized prior to the allocator (see virtmem::SDVAlloc).
Two interfaces exist to actually use virtual memory.
The first approach is to interface with raw memory directly through functions defined in virtmem::BaseVAlloc (e.g. read() and write()). Although dealing with raw memory might be slightly more efficient performance wise, this approach is not recommended as it is fairly cumbersome to do so.
The second approach is to use virtual pointer wrappers. These template classes were designed to make virtual memory access as close as 'regular' memory access as possible. Here is an example:
In this example we defined a virtual pointer to an int, which is linked to the SD allocator. An alternative syntax to define a virtual pointer is through virtmem::VAlloc::TVPtr:
If your platform supports C++11, this can shortened further:
Staying on C++11 support, using auto further reduces the syntax quite a bit:
Memory allocation is done through the alloc() function. In the above example no arguments were passed to alloc(), which means that alloc will automatically deduce the size required for the pointer type (int). If you want to allocate a different size (for instance to use the data as an array) then the number of bytes should be specified as the first argument to alloc:
Besides all standard types (char, int, short, etc.), virtual pointers can also work with custom types (structs/classes):
Note that there are a few imitations when using structs (or classes) with virtmem. For details: see virtmem::VPtr.
Finally, to free memory the free() function can be used:
A portion of the virtual memory can be locked to a memory page. Whenever such a lock is made, the data is guaranteed to stay in a memory page and will not be swapped out. The same data can be locked multiple times, and the data may only be swapped when all locks are released.
One reason to lock data is to improve performance. As soon as data is locked it can safely be accessed through a regular pointer, meaning that no additional overhead exists to use the data.
Another reason to lock data is to work with code that only accepts data residing in regular memory space. For instance, if a function needs to be called that requires a pointer to the data, the pointer to the locked memory page region can then be passed to such a function.
Each virtual data lock will essentially block a memory page. Since the number of memory pages is rather small, care should be taken to not to create too many different data locks.
To decrease the likelyhood of running out of free memory pages, virtmem supports two additional sets of smaller memory pages which are specifically used for data locking. They are much smaller in size, so they will not use a large deal of RAM, while still providing extra capacity for smaller data locks. Having a set of smaller memory pages is especially useful for locks created when accessing data in structures/classes.
The default size and amount of memory pages is dependent upon the MCU platform and can be customized as described in Configuring allocators.
To create a lock to virtual memory the virtmem::VPtrLock class is used:
The virtmem::makeVirtPtrLock function is used to create a lock. The last parameter to this function (optional, by default false) tells whether the locked data should be treated as read-only: if set to false the data will be written back to the virtual memory (even if unchanged) after the lock has been released. If you know the data will not be changed (or you don't care about changes), then it is more efficient to pass true instead.
Accessing locked data is simply done by dereferencing the lock variable (i.e. *lock).
Sometimes it is not possible to completely lock the memory region that was requested. For instance, there may not be sufficient space available to lock the complete data to a memory page or there will be overlap with another locked memory region. For this reason, it is important to always check the actual size that was locked. After a lock has been made, the effective size of the locked region can be requested by the virtmem::VPtrLock::getLockSize() function. Because it is rather unpredictable whether the requested data can be locked in one go, it is best create a loop that iteratively creates locks until all bytes have been dealt with:
Note that a memset overload is provided by virtmem which works with virtual pointers.
After you are finished working with a virtual memory lock it has to be released. This can be done manually with the virtmem::VPtrLock::unlock function. However, the virtmem::VPtrLock destructor will call this function automatically (the class follows the RAII principle). This explains why calling unlock() in the above example was not necessary, as the destructor will call it automatically when the lock variable goes out of scope at the end of every iteration.
Whenever virtual data is accessed, virtmem first has to make sure that this data resides in regular RAM (i.e. a memory page). In addition, if the data is changed, virtmem has to flag this data as 'dirty' so that it will be synchronized during the next page swap. To achieve these tasks, virtmem returns a proxy class whenever a virtual pointer is dereferenced, instead of returning the actual data. This proxy class (virtmem::VPtr::ValueWrapper) acts as it is the actual data, and is mostly invisible to the user:
The first line of the above example demonstrates a read operation: in this case the proxy class (returned by *myIntVPtr) will be converted to an int (automatic type cast), during which it will return a copy the data from virtual memory. The second line is a write operation: here the proxy class signals virtmem that data is changed and needs to be synchronized during the next page swap.
For accessing data members of structures (everything discussed here also applies to classes) in virtual memory the situation is more complicated. When a member is accessed through the -> operator of of the virtual pointer, we must return an actual pointer to the structure. While it is possible to make this data available through a memory page (as is done normally) in the -> overload function, synchronizing the data back is more tricky. For this reason, another proxy class is used (virtmem::VPtr::MemberWrapper) which is returned when the -> operator is called. This proxy class has also its -> operator overloaded, and this overload returns the actual data. The lifetime of this proxy class is important and matches that of the lifetime the data needs to be available in a memory page (for more details, see Stroustrup's general wrapper paper). Following from this, the proxy class will create a data lock to the data of the structure during construction and release this lock during its destruction.
Sometimes it may be handy to assign a regular pointer to a virtual pointer:
In the above example we have a (nonsensical) function f, which only accepts virtual pointers. Hence, the final line in this example will fail. Of course we could overload this function and provide an implementation that supports regular pointers. Alternatively, you can also 'wrap' the regular pointer inside a virtual pointer:
When a regular pointer is wrapped inside a virtual pointer, it can be used as it were a virtual pointer. If you want to obtain the original pointer then you can use the unwrap() function:
Please note that calling unwrap() on a non-wrapped virtual pointer yields an invalid pointer address. To avoid this, the isWrapped() function can be used.
Consider the following code:
The size of MyStruct contains a large buffer and the total size exceeds 1 kilobyte. A good reason to put it in virtual memory! However, when assignment occurs in the example above, dereferencing the virtual pointer causes a copy of the whole data structure to a virtual page (more info here). This means that the memory page must be sufficient in size. One option is to configure the allocator and make sure memory pages are big enough. However, since this will increase RAM usage, this may not be an option.
For the example outlined above, another option may be to let MyStruct::buffer be a virtual pointer to a buffer in virtual memory:
Since MyStruct only stores a virtual pointer, its much smaller and can easily fit into a memory page.
While not more than one instance of a memory allocator type should be defined, it is possible to define different allocators in the same program:
The number and size of memory pages can be configured in config.h. Alternatively, these settings can be passed as a template parameter to the allocator. For more info, see the description about virtmem::DefaultAllocProperties.
It might be necessary to obtain a pointer to a member of a structure (or class) which resides in virtual memory. The way to obtain such a pointer with 'regular' memory is by using the address-of operator ('&'):
You may be tempted to do the same when the structure is in virtual memory:
However, this should never be done! The problem is that the above code will set p to an address inside one of the virtual memory pages. This should be considered as a temporary storage location and the contents can be changed anytime.
To obtain a 'safe' pointer, one should use the getMembrPtr() function:
Overloads of some common C functions for dealing with memory and strings are provided by virtmem. They accept virtual pointers or a mix of virtual and regular pointers as function arguments. Please note that they are defined in the virtmem namespace like any other code from virtmem, hence, they will not "polute" the global namespace unless you want to (i.e. by using the using directive).
The following function overloads exist:
memcpymemsetmemcmpstrncpystrcpystrncmpstrcmpstrlenWhen working with regular pointers, a void pointer can be used to store whatever pointer you like. With virtual pointers something similar can be achieved by using the base class for virtual pointers, namely virtmem::BaseVPtr:
When dealing with both regular and virtual pointers, it may be handy to use a single value that can set both types to zero. For this virtmem::NILL can be used:
Note that on platforms that support C++11 you can simply use nullptr instead.
Similar to regular pointers, sometimes it may be necessary to convert from one pointer type to another. For this a simple typecast works as expected:
Some platforms such as the latest Arduino versions (>= 1.6.6) or Teensyduino support the fairly recent C++11 standard. In virtmem you can take this to your advantage to shorten the syntax, for instance by using template aliasing:
In fact, this feature is already used by allocator classes:
The new auto keyword support means that we can further reduce the syntax quite a bit:
Another, small feature with C++11 support, is that nullptr can be used to assign a zero address to a virtual pointer.
Maybe, possibly not. A quick google reveales that most SD cards should have something called 'wear leveling', meaning that you probably don't have to worry about it. In any case, virtmem tries to reduce writes whenever possible. Furthermore, to get an indication of how much is written, have a look at the statistics functions.
Several factors may influence the speed.
In general the fastest allocator is the SPI RAM allocator, followed by the multi SPI RAM allocator, SD allocator and serial allocator.
The speed of the first three all largely depend on SPI speeds. These are typically much higher on ARM based boards (e.g. Teensy 3.X) compared to AVR boards (e.g. Arduino Uno).
The speed of the serial allocator depends mainly on the baudrate. Increasing this number will signficantly improve read and write speeds. Note that even the AVR based boards can handle speeds much higher than the 115200 kbps that is sometimes marked as maximum speed. Also note that some boards, like Teensy 3.X and Arduino Due, have 'virtual serial ports', which are only limited by USB transfer speeds – not baudrates.
Finally, the convenience of virtual pointers will add some overhead compared to accessing data in regular memory. Beeing a software solution, more steps have to be performed for data access. Using virtual data locks can signifcantly reduce this overhead.
When accessing virtual data, a proxy class is returned. This class behaves as much as the data as possbile, but sometimes the compiler needs some help. Simply casting it to the right type should be enough:
The serial allocator needs to be connected with a computer which runs a script (virtmem/extras/serial_host.py). During initialization of the allocator it will wait for an connection indefinitely. For more information, please see the documentation about the serial alloactor.
All allocators are template classes. This was mainly done so that memory page settings can be configured, for instance:
However, often we don't need to configure anything, and the default is fine:
We don't actually have to specify virtmem::DefaultAllocProperties to stay with defaults:
To shorten it further, the 'non P' allocator shortcuts were made:
Note that allocators such as virtmem::MultiSPIRAMVAllocP always require template parameters, hence, only a 'P version' exists.
Some benchmarking results are shown below. Note that these numbers are generated with very simple, and possibly not so accurate tests, hence they should only be used as a rough indication (source code can be found in virtmem/examples/benchmark).
| Allocator | Teensy 3.2 (96 MHz) | Teensy 3.2 (144 MHz) | Arduino Uno | |||
|---|---|---|---|---|---|---|
| Read / Write (kB/s) | Read / Write locks (kB/s) | Read / Write (kB/s) | Read / Write locks (kB/s) | Read / Write (kB/s) | Read / Write locks (kB/s) | |
| Native (write only) | 13333 | 20000 | 970 | |||
| virtmem::StaticVAllocP | 333 / 228 | 6000 / 7900 | 458 / 313 | 8905 / 11607 | 25 / 22 | 265 / 357 |
| virtmem::SerialVAllocP | 227 / 152 | 500 / 373 | 250 / 182 | 496 / 378 | 5 / 2 (14 / 9) | 6 / 4 (30 / 20) |
| virtmem::SDVAllocP | 266 / 70 | 1107 / 98 | 347 / 72 | 1102 / 91 | 23 / 15 | 156 / 44 |
| virtmem::SPIRAMVAllocP | 284 / 193 | 1887 / 1159 | 380 / 253 | 2083 / 1207 | 23 / 19 | 150 / 118 |
Some notes:
Copyright (c) 2015-2016, Rick Helmus All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
1.8.9.1