Contents

 1 Introduction to NVRAM
 2 Available hardware
  2.1 Technology
   2.1.1 MRAM-based microcontrollers, boards and modules
   2.1.2 ReRAM-based microcontroller
   2.1.3 FRAM-based microcontrollers
   2.1.4 RAM + battery
 3 Scientific topics
  3.1 NVRAM-based architectures
  3.2 Optimize mass storage system
   3.2.1 Other optimization of mass storage
  3.3 Orthogonal persistence
   3.3.1 Persistent heaps
   3.3.2 Orthogonal persistence by checkpointing or transactions
   3.3.3 Whole system persistence
  3.4 NVRAM without persistence goal
   3.4.1 RAM as a cache for NVRAM
   3.4.2 RAM next to NVRAM
   3.4.3 NVRAM as a replacement for DRAM
  3.5 Various
   3.5.1 Approximate computing
  3.6 NVRAM in devices other than the processor
 4 Various approaches
 5 Emulating NVRAM
  5.1 Emulator-based solutions
  5.2 Circuit-level simulator
Bibliography

1 Introduction to NVRAM

First of all, some definitions:

Memristor
A hypothetical non-linear, passive, two-terminal electrical component relating electric charge and magnetic flux linkage. The resistance of a memristor should depend on how much electric charge has flowed in what direction through it in the past. No component exist that match the initial formal definition of the memristor, and the feasibility of such a component is today doubtful. However, some components have been designed that have similar behavior (extended memristive systems.
Non-Volatile Flip-Flop (NVFF)
A flip-flop that retains its content when not powered by saving the content of a flip-flop in a retention circuit during the power-down.
NVRAM
Memristive systems can be used to build non-volatile memories. Existing NVRAM are described below;
SCM
Storage-class memory denotes memory types that are byte-adressable, faster than Flash, but not fast enough to be used as working memory;
NVM
Non Volatile Memory. Denotes all types of non-volatile memories including – contrary to NVRAM – Flash or EEPROM.

I will not talk of beyond-CMOS logic transistors, such as tunneling field-effect transistors (TFETs) and negative capacitance field-effect transistors (NCFETs) because, well, I don’t understand them (yet).

So, non-volatile random-access memory (NVRAM) is a type of random-access memory (RAM) that retains its information when power is turned off (non-volatile). This is in contrast to DRAM or SRAM which maintain data as long as power is applied.

Since decades, computers load instructions and data from a non-volatile memory (NVM), typically a Flash drive, which is slow but has big storage capacity. They are loaded into RAM, which has small storage capacity because it is not dense. The processor can then execute the instructions, allowing to read data, compute new one, and to save them back to NVM.

But what if RAM was persistent and dense and, why not, as fast as SRAM ? What would the architecture of the processor look like ? The notion of file would it still be useful ? In what form ? How to manage a computer which has no need to save data ? Which impact on performances ?

Lots of researchers are working to design a NVRAM that would be dense, cheap, fast to write, fast to read, consume low power, has a great endurance. Many technologies are being investigated to obtain different compromises between these qualities. The graal, a type of memory that would have all of them, is usually referred to as universal memory.

Recently, some NVRAM technologies have become mature enough to build prototypes and the impact on computer science can start to be investigated. In the following, I give some information on the hardware (including manufacturers and technologies), then I briefly give some computer science topics that are impacted by the existence of NVRAM.

2 Available hardware

2.1 Technology

Lots of manufacturers are working on the design of NVRAM around the world : Toshiba, Panasonic, Fujitsu, Samsung, Everspin, Ramtron, Micron, SanDisk, Lapis, Qualcomm, Infineon, Adesto Technologies, Avalanche Technology, Cypress, Spin Transfer Technology, IMEC, Crocus technology.

In France, ST MicroElectronics and Spintec are industrials investigating in this area. On the academic part, the CEA Leti and the LIRMM are working on various NVRAM technologies.

For more details on technologies and trends, you can read [Su et al., 2017Boukhobza et al., 2017Yu and Chen, 2016]. Older interesting readings include:

2.1.1 MRAM-based microcontrollers, boards and modules

2.1.2 ReRAM-based microcontroller

2.1.3 FRAM-based microcontrollers

2.1.4 RAM + battery

This is not really NVRAM (not dense, requires electrical power):

3 Scientific topics

Numerous scientific fields are impacted by the presence of NVRAM and it is a difficult task to classify existing works. Below, I list some keys for classifying them:

Which qualities of NVRAM is of interest in the work ?
Which part of the hardware architecture is modified ?
Which part of the software architecture is modified ?
What kind of applications are targetted
The size of the computing systems targetted appears to be one important criteria to distinguish between works:
Big machines
  • Database
  • Computational-intensive
  • Web servers
  • ...
Small machines
  • Transiently-powered systems
  • Normally-off systems
  • Sensors
  • ...
All machines
Some works concern nonetheless all types of computing systems

In the remainder of this article, I rather classify existing works with respect to the scientific goal that is pursued, but I tag each citation with some keywords giving information on the above criterias.

Some position papers give an overview of the impact of NVRAM on computer science :

3.1 NVRAM-based architectures

Lots of work change the processor architecture. Basically, the goal is to replace parts of the memory hierarchy, e.g. cache, with NVRAM, and simulate the gains (performances, energy etc.). I am not exhaustive here. If you want to read only one article in this category, read this one [Boukhobza et al., 2017]. And two others ? Those ones [Mittal et al., 2015Su et al., 2017].

3.2 Optimize mass storage system

Some works replace Hard/Flash drives with NVRAM. Some of them invent new file systems adapted to the byte-adressable nature of NVRAM:

3.2.1 Other optimization of mass storage

3.3 Orthogonal persistence

If the working memory contains NVRAM, data can be made persistent without the use of a File System API, achieving orthogonal or transparent persistency.

From https://en.wikipedia.org/wiki/Persistence_(computer_science)

Persistence is said to be ”orthogonal” or ”transparent” when it is implemented as an intrinsic property of the execution environment of a program. An orthogonal persistence environment does not require any specific actions by programs running in it to retrieve or save their state.

3.3.1 Persistent heaps

Orthogonal persistence can be achieved by the use of a persistent heap. In this case, applications use a transactional API to allocate, deallocate, and modify persistent objects. On recovery, the application retrieves a special “root” object from which it can reach the others. Persistent heaps provide persistence to the in-memory data structure and provides the ability to remove the serialization process between the in-memory data structure and the file.

3.3.2 Orthogonal persistence by checkpointing or transactions

Other works achieve orthogonal persistence without requiring programmers to use a dedicated API:

3.3.3 Whole system persistence

3.4 NVRAM without persistence goal

Lots of works investigate the use of hybrid design NVRAM+DRAM in order to benefit from properties other than the persistence of NVRAM. For instance, the density and low energy consumption of of PCM.

3.4.1 RAM as a cache for NVRAM

3.4.2 RAM next to NVRAM

A key concern of these works is to increase the endurance of PCM, using various wear-levelling techniques:

The use of other types of memory has been investigated:

3.4.3 NVRAM as a replacement for DRAM

Some works presents techniques to improve PCM’s lifetime:

Other works deal with other types of memory, typically MRAM, removing the need for wear-levelling:

3.5 Various

Various aspects that I don’t manage to arrange in a specify category:

3.5.1 Approximate computing

A bunch of works study the performances of the memory hierarchy while considering Approximate computing:

3.6 NVRAM in devices other than the processor

What kind of optimizations can we expect from the introduction of NVRAM in peripherals themselves ? I have found no work on that but some ideas I want to investigate, with other researchers (thanks O. Sentieys) include:

4 Various approaches

- nvram seems to make near-data and in-memory computing possible - neural networks

5 Emulating NVRAM

As few hardware is available yet, it is difficult for computer scientists to setup systems dedicated to NVRAM and evaluate their performances. Although the realism of the simulation may be a real issue, the benefits of emulation are numerous:

Simulating NVRAM is an active scientific topic. Some tools already exist; I briefly detail them in the following.

Another approach is to extrapolate from the results with DRAM. A great article demonstrate the limits of this approach:

5.1 Emulator-based solutions

Bochs
is an open source IA-32 (x86) PC emulator written in C++. Some works have extended it to simulate NVRAM-based machine performances [Traue et al., 2014].
MSPSim
is an emulator of MSP430 processors. It is used by some other works to simulate Flash [Ransford et al., 2011] or NVRAM [Ait Aoudia et al., 2014] performances.
NVMPro, a DRAM-based performance emulation, by HPLabs [Sengupta et al., 2015]. It injects software delays after given time intervals called epoch].
Perma
[Essen et al., 2012] implements a NVRAM architecture simulator (PerMA) in a ramdisk-style device that stores an application’s data in privately held DRAM pages, and can insert a customizable delay when a page of data is accessed. It is calibrated thanks to existing technology (Virident PCIe-attached Flash card, DRAM, and FusionIO). Link to the code: https://bitbucket.org/vanessen/di-mmap/src.
Tuna
The Tuna http://opennvram.org/ board uses a Zinq to emulate various access latencies thanks to an FPGA design between the ARM Cortex A9 and the DDR3.
PMFS
POSIX-compliant file system optimized for persistent memory [Dulloor et al., 2014]. Maps the entire persistent memory in the kernel address space. Allows to change the latency of emulated NVRAM. Used by several recent articles for evaluation.
[Kannan et al., 2017] throttles the per-socket DRAM bandwidth and latency through modification of PCI-based thermal registers.
Haste
can model PCIe-based SSDs. It holds 64 GB of 667 MHz DDR2 DRAM, running at 250 MHz DDR (500M transfers per second), under the control of four Xilinx Virtex 5 FPGAs.

5.2 Circuit-level simulator

The advantage of circuit-level simulators is that they may be very precise. The problem is to connect them to whole-system emulators (with memory management, cache system, etc.).

NVSim
models integrated circuits [Dong et al., 2012].

Bibliography

[Ait Aoudia et al., 2014]   Ait Aoudia, F., Marquet, K., and Salagnac, G. (2014). Incremental checkpointing of program state to nvram for transiently-powered systems. In ReCoSoC - 7th International Workshop on Reconfigurable Communication-centric Systems-on-Chip, Montpellier, France.

[Badam and Pai, 2011]   Badam, A. and Pai, V. S. (2011). Ssdalloc: Hybrid ssd/ram memory management made easy. In Proceedings of the 8th USENIX Conference on Networked Systems Design and Implementation, NSDI’11, pages 211–224, Berkeley, CA, USA. USENIX Association.

[Bailey et al., 2011]   Bailey, K., Ceze, L., Gribble, S. D., and Levy, H. M. (2011). Operating system implications of fast, cheap, non-volatile memory. In 13th Workshop on Hot Topics in Operating Systems, HotOS XIII, Napa, California, USA, May 9-11, 2011. USENIX Association.

[Ben-David et al., 2016]   Ben-David, N., Blelloch, G. E., Fineman, J. T., Gibbons, P. B., Gu, Y., McGuffey, C., and Shun, J. (2016). Parallel algorithms for asymmetric read-write costs. In Proceedings of the 28th ACM Symposium on Parallelism in Algorithms and Architectures, SPAA ’16, pages 145–156, New York, NY, USA. ACM.

[Bhaskaran et al., 2014]   Bhaskaran, M. S., Xu, J., and Swanson, S. (2014). Bankshot: Caching slow storage in fast non-volatile memory. SIGOPS Oper. Syst. Rev., 48(1):73–81.

[Blelloch et al., 2015]   Blelloch, G. E., Fineman, J. T., Gibbons, P. B., Gu, Y., and Shun, J. (2015). Sorting with asymmetric read and write costs. In Proceedings of the 27th ACM Symposium on Parallelism in Algorithms and Architectures, SPAA ’15, pages 1–12, New York, NY, USA. ACM.

[Boukhobza et al., 2017]   Boukhobza, J., Rubini, S., Chen, R., and Shao, Z. (2017). Emerging nvm: A survey on architectural integration and research challenges. ACM Trans. Des. Autom. Electron. Syst., 23(2):14:1–14:32.

[Cargnini et al., 2014]   Cargnini, L. V., Torres, L., Brum, R. M., Senni, S., and Sassatelli, G. (2014). Embedded memory hierarchy exploration based on magnetic random access memory. Journal of Low Power Electronics and Applications, 4(3):214–230.

[Caulfield et al., 2010a]   Caulfield, A. M., Coburn, J., Mollov, T., De, A., Akel, A., He, J., Jagatheesan, A., Gupta, R. K., Snavely, A., and Swanson, S. (2010a). Understanding the impact of emerging non-volatile memories on high-performance, io-intensive computing. In Proceedings of the 2010 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis, SC ’10, pages 1–11, Washington, DC, USA. IEEE Computer Society.

[Caulfield et al., 2010b]   Caulfield, A. M., De, A., Coburn, J., Mollow, T. I., Gupta, R. K., and Swanson, S. (2010b). Moneta: A high-performance storage array architecture for next-generation, non-volatile memories. In Proceedings of the 2010 43rd Annual IEEE/ACM International Symposium on Microarchitecture, MICRO ’43, pages 385–395, Washington, DC, USA. IEEE Computer Society.

[Chakrabarti et al., 2014]   Chakrabarti, D. R., Boehm, H.-J., and Bhandari, K. (2014). Atlas: Leveraging locks for non-volatile memory consistency. In Proceedings of the 2014 ACM International Conference on Object Oriented Programming Systems Languages & Applications, OOPSLA ’14, pages 433–452, New York, NY, USA. ACM.

[Chen et al., 2012]   Chen, J., Chiang, R. C., Huang, H. H., and Venkataramani, G. (2012). Energy-aware writes to non-volatile main memory. SIGOPS Oper. Syst. Rev., 45(3):48–52.

[Chen et al., 2015]   Chen, R., Wang, Y., Hu, J., Liu, D., Shao, Z., and Guan, Y. (2015). Unified non-volatile memory and nand flash memory architecture in smartphones. In The 20th Asia and South Pacific Design Automation Conference, pages 340–345.

[Chien et al., 2016]   Chien, T. K., Chiou, L. Y., Sheu, S. S., Lin, J. C., Lee, C. C., Ku, T. K., Tsai, M. J., and Wu, C. I. (2016). Low-power mcu with embedded reram buffers as sensor hub for iot applications. IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 6(2):247–257.

[Coburn et al., 2011]   Coburn, J., Caulfield, A. M., Akel, A., Grupp, L. M., Gupta, R. K., Jhala, R., and Swanson, S. (2011). Nv-heaps: Making persistent objects fast and safe with next-generation, non-volatile memories. In Proceedings of the Sixteenth International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS XVI, pages 105–118, New York, NY, USA. ACM.

[Condit et al., 2009]   Condit, J., Nightingale, E. B., Frost, C., Ipek, E., Lee, B., Burger, D., and Coetzee, D. (2009). Better i/o through byte-addressable, persistent memory. In Proceedings of the ACM SIGOPS 22Nd Symposium on Operating Systems Principles, SOSP ’09, pages 133–146, New York, NY, USA. ACM.

[Dhiman et al., 2009]   Dhiman, G., Ayoub, R., and Rosing, T. (2009). Pdram: A hybrid pram and dram main memory system. In Proceedings of the 46th Annual Design Automation Conference, DAC ’09, pages 664–469, New York, NY, USA. ACM.

[Doh et al., 2009]   Doh, I. H., Lee, H. J., Moon, Y. J., Kim, E., Choi, J., Lee, D., and Noh, S. H. (2009). Impact of nvram write cache for file system metadata on i/o performance in embedded systems. In Selected Areas in Cryptography, 16th Annual International Workshop, SAC 2009, Calgary, Alberta, Canada, August 13-14, 2009, Revised Selected Papers, volume 5867 of Lecture Notes in Computer Science, pages 1658–1663. Springer.

[Dong et al., 2012]   Dong, X., Xu, C., Xie, Y., and Jouppi, N. (2012). Nvsim: A circuit-level performance, energy, and area model for emerging nonvolatile memory. Computer-Aided Design of Integrated Circuits and Systems, IEEE Transactions on, 31(7):994–1007.

[Dulloor et al., 2014]   Dulloor, S. R., Kumar, S., Keshavamurthy, A., Lantz, P., Reddy, D., Sankaran, R., and Jackson, J. (2014). System software for persistent memory. In Proceedings of the Ninth European Conference on Computer Systems, EuroSys ’14, pages 15:1–15:15, New York, NY, USA. ACM.

[Essen et al., 2012]   Essen, B. V., Pearce, R. A., Ames, S., and Gokhale, M. (2012). On the role of nvram in data-intensive architectures: An evaluation. In 26th IEEE International Parallel and Distributed Processing Symposium, IPDPS 2012, Shanghai, China, May 21-25, 2012, pages 703–714. IEEE Computer Society.

[Gao et al., 2013]   Gao, T., Strauss, K., Blackburn, S. M., McKinley, K. S., Burger, D., and Larus, J. (2013). Using managed runtime systems to tolerate holes in wearable memories. In Proceedings of the 34th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’13, pages 297–308, New York, NY, USA. ACM.

[Gebregiorgis et al., 2018]   Gebregiorgis, A., Bishnoi, R., and Tahoori, M. B. (2018). Spintronic normally-off heterogeneous system-on-chip design. In 2018 Design, Automation Test in Europe Conference Exhibition (DATE), pages 113–118.

[Greenan and Miller, 2007]   Greenan, K. M. and Miller, E. L. (2007). Prims: Making nvram suitable for extremely reliable storage. In Proceedings of the 3rd Workshop on on Hot Topics in System Dependability, HotDep’07, Berkeley, CA, USA. USENIX Association.

[Guerra et al., 2012]   Guerra, J., Marmol, L., Campello, D., Crespo, C., Rangaswami, R., and Wei, J. (2012). Software persistent memory. In Presented as part of the 2012 USENIX Annual Technical Conference (USENIX ATC 12), pages 319–331, Boston, MA. USENIX.

[Hu et al., 2011]   Hu, J., Xue, C. J., Zhuge, Q., Tseng, W. C., and Sha, E. H. M. (2011). Towards energy efficient hybrid on-chip scratch pad memory with non-volatile memory. In 2011 Design, Automation Test in Europe, pages 1–6.

[Hu et al., 2013]   Hu, J., Zhuge, Q., Xue, C. J., Tseng, W. C., and Sha, E. H. M. (2013). Software enabled wear-leveling for hybrid pcm main memory on embedded systems. In Design, Automation Test in Europe Conference Exhibition (DATE), 2013, pages 599–602.

[Huang et al., 2014]   Huang, J., Schwan, K., and Qureshi, M. K. (2014). Nvram-aware logging in transaction systems. Proc. VLDB Endow., 8(4):389–400.

[Hwang et al., 2014]   Hwang, T., Jung, J., and Won, Y. (2014). Heapo: Heap-based persistent object store. Trans. Storage, 11(1):3:1–3:21.

[Hwang et al., 2017]   Hwang, T., Lee, D., Noh, Y., and Won, Y. (2017). Designing persistent heap for byte addressable nvram. In 2017 IEEE 6th Non-Volatile Memory Systems and Applications Symposium (NVMSA), pages 1–6.

[Jang et al., 2012]   Jang, H., An, B. S., Kulkarni, N., Yum, K. H., and Kim, E. J. (2012). A hybrid buffer design with stt-mram for on-chip interconnects. In Networks on Chip (NoCS), 2012 Sixth IEEE/ACM International Symposium on, pages 193–200.

[Jung and Won, 2016]   Jung, J. and Won, Y. (2016). nvramdisk: A transactional block device driver for non-volatile ram. IEEE Transactions on Computers, 65(2):589–600.

[Kannan et al., 2017]   Kannan, S., Gavrilovska, A., Gupta, V., and Schwan, K. (2017). Heteroos: Os design for heterogeneous memory management in datacenter. In 2017 ACM/IEEE 44th Annual International Symposium on Computer Architecture (ISCA), pages 521–534.

[Kannan et al., 2013]   Kannan, S., Gavrilovska, A., Schwan, K., and Milojicic, D. (2013). Optimizing checkpoints using nvm as virtual memory. In Parallel Distributed Processing (IPDPS), 2013 IEEE 27th International Symposium on, pages 29–40.

[Kannan et al., 2011]   Kannan, S., Gavrilovska, A., Schwan, K., Milojicic, D., and Talwar, V. (2011). Using active nvram for i/o staging. In Proceedings of the 2Nd International Workshop on Petascal Data Analytics: Challenges and Opportunities, PDAC ’11, pages 15–22, New York, NY, USA. ACM.

[Kim et al., 2014a]   Kim, J., Min, C., and Eom, Y. I. (2014a). Reducing excessive journaling overhead with small-sized nvram for mobile devices. IEEE Trans. Consumer Electronics, 60(2):217–224.

[Kim et al., 2014b]   Kim, M., Shin, J., and Won, Y. (2014b). Selective segment initialization: Exploiting nvram to reduce device startup latency. IEEE Embedded Systems Letters, 6(2):33–36.

[Kim et al., 2016]   Kim, W.-H., Kim, J., Baek, W., Nam, B., and Won, Y. (2016). Nvwal: Exploiting nvram in write-ahead logging. In Proceedings of the Twenty-First International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS ’16, pages 385–398, New York, NY, USA. ACM.

[Komalan et al., 2018]   Komalan, M., Rock, O. H., Hartmann, M., Sakhare, S., Tenllado, C., Gómez, J. I., Kar, G. S., Furnemont, A., Catthoor, F., Senni, S., Novo, D., Gamatie, A., and Torres, L. (2018). Main memory organization trade-offs with dram and stt-mram options based on gem5-nvmain simulation frameworks. In 2018 Design, Automation Test in Europe Conference Exhibition (DATE), pages 103–108.

[Kultursay et al., 2013]   Kultursay, E., Kandemir, M., Sivasubramaniam, A., and Mutlu, O. (2013). Evaluating stt-ram as an energy-efficient main memory alternative. IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS), 0:256–267.

[Layer et al., 2016]   Layer, C., Becker, L., Jabeur, K., Claireux, S., Dieny, B., Prenat, G., Pendina, G. D., Gros, S., Paoli, P., Javerliac, V., Bernard-Granger, F., and Decloedt, L. (2016). Reducing system power consumption using check-pointing on nonvolatile embedded magnetic random access memories. J. Emerg. Technol. Comput. Syst., 12(4):32:1–32:24.

[Lee et al., 2010]   Lee, B. C., Zhou, P., Yang, J., Zhang, Y., Zhao, B., Ipek, E., Mutlu, O., and Burger, D. (2010). Phase-change technology and the future of main memory. IEEE Micro, 30(1):143–143.

[Lee and Oh, 2013]   Lee, D. and Oh, H. (2013). A lifetime aware buffer assignment method for streaming applications on dram/pram hybrid memory. ACM Trans. Embed. Comput. Syst., 12(1s):36:1–36:17.

[Lee and Won, 2013]   Lee, D. and Won, Y. (2013). Bootless boot: Reducing device boot latency with byte addressable nvram. In High Performance Computing and Communications 2013 IEEE International Conference on Embedded and Ubiquitous Computing (HPCC_EUC), 2013 IEEE 10th International Conference on, pages 2014–2021.

[Lee et al., 2016]   Lee, S., Lee, T., Park, H., Ahn, J., Yoo, S., Won, Y., and Lee, S. (2016). Differential write-conscious software design on phase-change memory: An sqlite case study. ACM Trans. Des. Autom. Electron. Syst., 21(3):47:1–47:25.

[Li et al., 2015]   Li, J., Pu, C., Chen, Y., Talwar, V., and Milojicic, D. (2015). Improving preemptive scheduling with application-transparent checkpointing in shared clusters. In Proceedings of the 16th Annual Middleware Conference, Middleware ’15, pages 222–234, New York, NY, USA. ACM.

[Li et al., 2012]   Li, X., Lu, K., and Zhou, X. (2012). Nv-ts: A fault tolerance transaction system based on persistent memory. In Computer Science and Electronics Engineering (ICCSEE), 2012 International Conference on, volume 2, pages 221–224.

[Li et al., 2016]   Li, Z., Liu, Y., Zhang, D., Xue, C. J., Wang, Z., Shi, X., Sun, W., Shu, J., and Yang, H. (2016). Hw/sw co-design of nonvolatile io system in energy harvesting sensor nodes for optimal data acquisition. In 53rd ACM/EDAC/IEEE Design Automation Conference (DAC), pages 1–6.

[Liu et al., 2011]   Liu, T., Zhao, Y., Xue, C. J., and Li, M. (2011). Power-aware variable partitioning for dsps with hybrid pram and dram main memory. In Design Automation Conference (DAC), 2011 48th ACM/EDAC/IEEE, pages 405–410.

[Liu et al., 2015]   Liu, Y., Li, Z., Li, H., Wang, Y., Li, X., Ma, K., Li, S., Chang, M. F., John, S., Xie, Y., Shu, J., and Yang, H. (2015). Ambient energy harvesting nonvolatile processors: From circuit to system. In 2015 52nd ACM/EDAC/IEEE Design Automation Conference (DAC), pages 1–6.

[Lu et al., 2014]   Lu, Y., Shu, J., Sun, L., and Mutlu, O. (2014). Loose-ordering consistency for persistent memory. In 2014 IEEE 32nd International Conference on Computer Design (ICCD), pages 216–223.

[Luo et al., 2015]   Luo, X., Liu, D., Liangy, L., Li, Y., Zhong, K., and Long, L. (2015). Mobilock: an energy-aware encryption mechanism for nvram-based mobile devices. In Non-Volatile Memory System and Applications Symposium (NVMSA), 2015 IEEE, pages 1–6.

[Ma et al., 2015]   Ma, K., Zheng, Y., Li, S., Swaminathan, K., Li, X., Liu, Y., Sampson, J., Xie, Y., and Narayanan, V. (2015). Architecture exploration for ambient energy harvesting nonvolatile processors. In 2015 IEEE 21st International Symposium on High Performance Computer Architecture (HPCA), pages 526–537.

[Meena et al., 2014]   Meena, J. S., Sze, S. M., Chand, U., and Tseng, T.-Y. (2014). Overview of emerging nonvolatile memory technologies. Nanoscale Research Letters, 9(1):1–33.

[Mirhoseini et al., 2013]   Mirhoseini, A., Songhori, E. M., and Koushanfar, F. (2013). Automated checkpointing for enabling intensive applications on energy harvesting devices. In International Symposium on Low Power Electronics and Design (ISLPED), pages 27–32.

[Mittal et al., 2015]   Mittal, S., Vetter, J. S., and Li, D. (2015). A survey of architectural approaches for managing embedded dram and non-volatile on-chip caches. IEEE Transactions on Parallel and Distributed Systems, 26(6):1524–1537.

[Narayanan and Hodson, 2012]   Narayanan, D. and Hodson, O. (2012). Whole-system persistence. In Proceedings of the Seventeenth International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS XVII, pages 401–410, New York, NY, USA. ACM.

[Oukid et al., 2017]   Oukid, I., Booss, D., Lespinasse, A., Lehner, W., Willhalm, T., and Gomes, G. (2017). Memory management techniques for large-scale persistent-main-memory systems. Proc. VLDB Endow., 10(11):1166–1177.

[Park et al., 2011]   Park, H., Yoo, S., and Lee, S. (2011). Power management of hybrid dram/pram-based main memory. In Proceedings of the 48th Design Automation Conference, DAC ’11, pages 59–64, New York, NY, USA. ACM.

[Pelley et al., 2014]   Pelley, S., Chen, P. M., and Wenisch, T. F. (2014). Memory persistency. In Proceeding of the 41st Annual International Symposium on Computer Architecuture, ISCA ’14, pages 265–276, Piscataway, NJ, USA. IEEE Press.

[Pelley et al., 2013]   Pelley, S., Wenisch, T. F., Gold, B. T., and Bridge, B. (2013). Storage management in the nvram era. PVLDB, 7(2):121–132.

[Qureshi et al., 2009]   Qureshi, M. K., Srinivasan, V., and Rivers, J. A. (2009). Scalable high performance main memory system using phase-change memory technology. In Proceedings of the 36th Annual International Symposium on Computer Architecture, ISCA ’09, pages 24–33, New York, NY, USA. ACM.

[Ramos et al., 2011]   Ramos, L. E., Gorbatov, E., and Bianchini, R. (2011). Page placement in hybrid memory systems. In Proceedings of the International Conference on Supercomputing, ICS ’11, pages 85–95, New York, NY, USA. ACM.

[Ransford et al., 2011]   Ransford, B., Sorber, J., and Fu, K. (2011). Mementos: System support for long-running computation on rfid-scale devices. In Proceedings of the Sixteenth International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS XVI, pages 159–170, New York, NY, USA. ACM.

[Ren et al., 2015]   Ren, J., Zhao, J., Khan, S., Choi, J., Wu, Y., and Mutlu, O. (2015). Thynvm: Enabling software-transparent crash consistency in persistent memory systems. In Proceedings of the 48th International Symposium on Microarchitecture, MICRO-48, pages 672–685, New York, NY, USA. ACM.

[Ryu, 2013]   Ryu, Y. (2013). Performance evaluation of page migration scheme for nvram-based wireles sensor nodes. International Journal of Distributed Sensor Networks, 2013:7.

[Sartakov et al., 2015]   Sartakov, V. A., Martens, A., and Kapitza, R. (2015). Temporality a nvram-based virtualization platform. In Reliable Distributed Systems (SRDS), 2015 IEEE 34th Symposium on, pages 104–109.

[Schwalb et al., 2015]   Schwalb, D., Berning, T., Faust, M., Dreseler, M., and Plattner, H. (2015). nvm malloc: Memory allocation for nvram. In Bordawekar, R., Lahiri, T., Gedik, B., and Lang, C. A., editors, ADMS@VLDB, pages 61–72.

[Sehgal et al., 2015]   Sehgal, P., Basu, S., Srinivasan, K., and Voruganti, K. (2015). An empirical study of file systems on nvm. In 2015 31st Symposium on Mass Storage Systems and Technologies (MSST), pages 1–14.

[Sengupta et al., 2015]   Sengupta, D., Wang, Q., Volos, H., Cherkasova, L., Li, J., Magalhaes, G., and Schwan, K. (2015). A framework for emulating non-volatile memory systemswith different performance characteristics. In Proceedings of the 6th ACM/SPEC International Conference on Performance Engineering, ICPE ’15, pages 317–320, New York, NY, USA. ACM.

[Seok et al., 2011]   Seok, H., Park, Y., Park, K.-W., and Park, K. H. (2011). Efficient page caching algorithm with prediction and migration for a hybrid main memory. SIGAPP Appl. Comput. Rev., 11(4):38–48.

[Su et al., 2017]   Su, F., Ma, K., Li, X., Wu, T., Liu, Y., and Narayanan, V. (2017). Nonvolatile processors: Why is it trending? In Design, Automation Test in Europe Conference Exhibition (DATE), 2017, pages 966–971.

[Swanson and Caulfield, 2013]   Swanson, S. and Caulfield, A. M. (2013). Refactor, reduce, recycle: Restructuring the i/o stack for the future of storage. Computer, 46(8):52–59.

[Teimoori et al., 2018]   Teimoori, M. T., Hanif, M. A., Ejlali, A., and Shafique, M. (2018). Adam: Adaptive approximation management for the non-volatile memory hierarchies. In 2018 Design, Automation Test in Europe Conference Exhibition (DATE), pages 785–790.

[Torres et al., 2013]   Torres, L., Brum, R. M., Cargnini, L. V., and Sassatelli, G. (2013). Trends on the application of emerging nonvolatile memory to processors and programmable devices. In 2013 IEEE International Symposium on Circuits and Systems (ISCAS 2013), Beijing, China, May 19-23, 2013, pages 101–104. IEEE.

[Traue et al., 2014]   Traue, J., Nolte, J., Engel, P., and Karnapke, R. (2014). Using emulation software to predict the performance of algorithms on nvram. In Proceedings of the 7th International ICST Conference on Simulation Tools and Techniques, SIMUTools ’14, pages 142–146, ICST, Brussels, Belgium, Belgium. ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering).

[Ved and Awasthi, 2018]   Ved, S. and Awasthi, M. (2018). Exploring non-volatile main memory architectures for handheld devices. In 2018 Design, Automation Test in Europe Conference Exhibition (DATE), pages 1528–1531.

[Venkataraman et al., 2011]   Venkataraman, S., Tolia, N., Ranganathan, P., and Campbell, R. H. (2011). Consistent and durable data structures for non-volatile byte-addressable memory. In Proceedings of the 9th USENIX Conference on File and Stroage Technologies, FAST’11, pages 5–5, Berkeley, CA, USA. USENIX Association.

[Volos et al., 2014]   Volos, H., Nalli, S., Panneerselvam, S., Varadarajan, V., Saxena, P., and Swift, M. M. (2014). Aerie: Flexible file-system interfaces to storage-class memory. In Proceedings of the Ninth European Conference on Computer Systems, EuroSys ’14, pages 14:1–14:14, New York, NY, USA. ACM.

[Volos et al., 2011]   Volos, H., Tack, A. J., and Swift, M. M. (2011). Mnemosyne: Lightweight persistent memory. In Proceedings of the Sixteenth International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS XVI, pages 91–104, New York, NY, USA. ACM.

[Wei et al., 2015]   Wei, W., Jiang, D., McKee, S. A., Xiong, J., and Chen, M. (2015). Exploiting program semantics to place data in hybrid memory. In Proceedings of the 2015 International Conference on Parallel Architecture and Compilation (PACT), PACT ’15, pages 163–173, Washington, DC, USA. IEEE Computer Society.

[Wei et al., 2014]   Wei, W., Jiang, D., Xiong, J., and Chen, M. (2014). Exploring opportunities for non-volatile memories in big data applications. In Zhan, J., Han, R., and Weng, C., editors, Big Data Benchmarks, Performance Optimization, and Emerging Hardware, volume 8807 of Lecture Notes in Computer Science, pages 209–220. Springer International Publishing.

[Wu et al., 2013]   Wu, X., Qiu, S., and Narasimha Reddy, A. L. (2013). Scmfs: A file system for storage class memory and its extensions. Trans. Storage, 9(3):7:1–7:23.

[Xu and Swanson, 2016]   Xu, J. and Swanson, S. (2016). Nova: A log-structured file system for hybrid volatile/non-volatile main memories. In 14th USENIX Conference on File and Storage Technologies (FAST 16), pages 323–338, Santa Clara, CA. USENIX Association.

[Xue et al., 2011]   Xue, C., Sun, G., Zhang, Y., Yang, J., Chen, Y., and Li, H. (2011). Emerging non-volatile memories: Opportunities and challenges. In Hardware/Software Codesign and System Synthesis (CODES+ISSS), 2011 Proceedings of the 9th International Conference on, pages 325–334.

[Yoon et al., 2011]   Yoon, D. H., Muralimanohar, N., Chang, J., Ranganathan, P., Jouppi, N. P., and Erez, M. (2011). Free-p: Protecting non-volatile memory against both hard and soft errors. In 2011 IEEE 17th International Symposium on High Performance Computer Architecture, pages 466–477.

[Yu and Chen, 2016]   Yu, S. and Chen, P. Y. (2016). Emerging memory technologies: Recent trends and prospects. IEEE Solid-State Circuits Magazine, 8(2):43–56.

[Zhan et al., 2016]   Zhan, J., Ouyang, J., Ge, F., Zhao, J., and Xie, Y. (2016). Hybrid drowsy sram and stt-ram buffer designs for dark-silicon-aware noc. IEEE Transactions on Very Large Scale Integration (VLSI) Systems, PP(99):1–14.

[Zhang and Li, 2009]   Zhang, W. and Li, T. (2009). Exploring phase change memory and 3d die-stacking for power/thermal friendly, fast and durable memory architectures. In Proceedings of the 2009 18th International Conference on Parallel Architectures and Compilation Techniques, PACT ’09, pages 101–112, Washington, DC, USA. IEEE Computer Society.

[Zhang and Swanson, 2015]   Zhang, Y. and Swanson, S. (2015). A study of application performance with non-volatile main memory. In 2015 31st Symposium on Mass Storage Systems and Technologies (MSST), pages 1–10.

[Zhang et al., 2015]   Zhang, Y., Yang, J., Memaripour, A., and Swanson, S. (2015). Mojim: A reliable and highly-available non-volatile memory system. In Proceedings of the Twentieth International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS ’15, pages 3–18, New York, NY, USA. ACM.

[Zhao et al., 2013]   Zhao, J., Li, S., Yoon, D. H., Xie, Y., and Jouppi, N. P. (2013). Kiln: Closing the performance gap between systems with and without persistence support. In Proceedings of the 46th Annual IEEE/ACM International Symposium on Microarchitecture, MICRO-46, pages 421–432, New York, NY, USA. ACM.

[Zhao et al., 2017]   Zhao, M., Fu, C., Li, Z., Li, Q., Xie, M., Liu, Y., Hu, J., Jia, Z., and Xue, C. J. (2017). Stack-size sensitive on-chip memory backup for self-powered nonvolatile processors. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 36(11):1804–1816.

[Zhou et al., 2009]   Zhou, P., Zhao, B., Yang, J., and Zhang, Y. (2009). A durable and energy efficient main memory using phase change memory technology. In Proceedings of the 36th Annual International Symposium on Computer Architecture, ISCA ’09, pages 14–23, New York, NY, USA. ACM.

[Zou et al., 2015]   Zou, Q., Poremba, M., He, R., Yang, W., Zhao, J., and Xie, Y. (2015). Heterogeneous architecture design with emerging 3d and non-volatile memory technologies. In The 20th Asia and South Pacific Design Automation Conference, pages 785–790.