Ramulator V2.0a

Introduction

Ramulator 2.0 is a modern, modular, and extensible cycle-accurate DRAM simulator. It is the successor of Ramulator 1.0 [Kim+, CAL'16], achieving both fast simulation speed and ease of extension. The goal of Ramulator 2.0 is to enable rapid and agile implementation and evaluation of design changes in the memory controller and DRAM to meet the increasing research effort in improving the performance, security, and reliability of memory systems. Ramulator 2.0 abstracts and models key components in a DRAM-based memory system and their interactions into shared interfaces and independent implementations, enabling easy modification and extension of the modeled functions of the memory controller and DRAM.

This Github repository contains the public version of Ramulator 2.0. From time to time, we will synchronize improvements of the code framework, additional functionalities, bug fixes, etc. from our internal version. Ramulator 2.0 is in its early stage and welcomes your contribution as well as new ideas and implementations in the memory system!

Currently, Ramulator 2.0 provides the DRAM models for the following standards:

• DDR3, DDR4, DDR5
• LPDDR5
• GDDR6
• HBM(2), HBM3

Ramulator 2.0 also provides implementations for the following RowHammer mitigation techniques:

• PARA [Kim+, ISCA'14]
• TWiCe [Lee+, ISCA'19]
• Graphene [Park+, MICRO'20]
• Hydra [Qureshi+, ISCA'22]
• Randomized Row Swap (RRS) [Saileshwar+, ASPLOS'22]
• An "Oracle" Refresh Mitigation [Kim+, ISCA'20]

A quick glance at Ramulator 2.0's other key features:

• Modular and extensible software architecture: Ramulator 2.0 provides an explicit separation of implementations from interfaces. Therefore new ideas can be implemented without intrusive changes.
• Self-registering factory for interface and implementation classes: Ramulator 2.0 automatically constructs the correct class of objects by their names as you specify in the configuration. Do not worry about boilerplate code!
• YAML-based configuration file: Ramulator 2.0 is configured via human-readable and machine-friendly configuration files. Sweeping parameters is as easy as editing a Python dictionary!

The initial release of Ramulator 2.0 is described in the following paper:

Haocong Luo, Yahya Can Tugrul, F. Nisa Bostancı, Ataberk Olgun, A. Giray Yaglıkcı, and Onur Mutlu, "Ramulator 2.0: A Modern, Modular, and Extensible DRAM Simulator," arXiv, 2023.

If you use Ramulator 2.0 in your work, please use the following citation:

@misc{luo2023ramulator2,
  title={{Ramulator 2.0: A Modern, Modular, and Extensible DRAM Simulator}}, 
  author={Haocong Luo and Yahya Can Tu\u{g}rul and F. Nisa Bostancı and Ataberk Olgun and A. Giray Ya\u{g}l{\i}k\c{c}{\i} and and Onur Mutlu},
  year={2023},
  archivePrefix={arXiv},
  primaryClass={cs.AR}
}

Using Ramulator 2.0

Dependencies

Ramulator uses some C++20 features to achieve both high runtime performance and modularity and extensibility. Therefore, a C++20-capable compiler is needed to build Ramulator 2.0. We have tested and verified Ramulator 2.0 with the following compilers:

• g++-12
• clang++-15

Ramulator 2.0 uses the following external libraries. The package manager system (Conan) will automatically download and configure these dependencies.

• argparse
• spdlog
• yaml-cpp

Getting Started

Install conan, if necessary

  $ pip install conan==1.60.2

Clone the repository

  $ git clone https://github.com/CMU-SAFARI/ramulator2

Configure the project and build the executable

  $ mkdir build
  $ cd build
  $ cmake ..
  $ make -j
  $ cp ./ramulator2 ../ramulator2
  $ cd ..

Alternate Conan build flow

  $ mkdir build
  $ cd build
  $ conan install .. --build=missing
  $ conan build ..
  $ cp ./ramulator2 ../ramulator2
  $ cd ..

Package as a Conan library

  $ cd <ramulator2 base directory>
  $ conan create .

This should produce a ramulator2 executable that you can execute standalone and a libramulator.so dynamic library that can be used as a memory system library by other simulators.

Running Ramulator 2.0 in Standalone Mode

Ramulator 2.0 comes with two independent simulation frontends: A memory-trace parser and a simplistic out-of-order core model that can accept instruction traces. To start a simulation with these frontends, just run the Ramulator 2.0 executable with the path to the configuration file specified through the -f argument

  $ ./ramulator2 -f ./example_config.yaml

To support easy automation of experiments (e.g., evaluate many different traces and sweep parameters), Ramulator 2.0 can accept the configurations as a string dump of the YAML document, which is usually produced by a scripting language that can easily parse and manipulate YAML documents (e.g., python). We provide an example python snippet to demonstrate an experiment of sweeping the nRCD timing constraint:

import os
import yaml  # YAML parsing provided with the PyYAML package

baseline_config_file = "./example_config.yaml"
nRCD_list = [10, 15, 20, 25]

base_config = None
with open(base_config_file, 'r') as f:
  base_config = yaml.safe_load(f)

for nRCD in nRCD_list:
  config["MemorySystem"]["DRAM"]["timing"]["nRCD"] = nRCD
  cmds = ["./ramulator2", str(config)]
  # Run the command with e.g., os.system(), subprocess.run(), ...

Using Ramulator 2.0 as a Library (gem5 Example)

Ramulator 2.0 packs all the interfaces and implementations into a dynamic library (libramulator.so). This can be used as a memory system library providing extensible cycle-accurate DRAM simulation to another simulator. We use gem5 as an example to show how to use Ramulator 2.0 as a library. We have tested and verified the integration of Ramulator 2.0 into gem5 as a library.

1. Clone Ramulator 2.0 into gem5/ext/ramulator2/ directory.
2. Build Ramulator 2.0. You should have libramulator.so at gem5/ext/ramulator2/ramulator2/libramulator.so
3. Create a file SConscript at gem5/ext/ramulator2/, with the following contents to add Ramulator 2.0 to gem5's build system

import os

Import('env')

if not os.path.exists(Dir('.').srcnode().abspath + '/ramulator2'):
  env['HAVE_RAMULATOR2'] = False
  Return()

env['HAVE_RAMULATOR2'] = True
ramulator2_path = os.path.join(Dir('#').abspath, 'ext/ramulator2/ramulator2/')
env.Prepend(CPPPATH=Dir('.').srcnode())
env.Append(
  LIBS=['ramulator'],
  LIBPATH=[ramulator2_path],
  RPATH=[ramulator2_path],
  CPPPATH=[
  ramulator2_path+'/src/', 
  ramulator2_path+'/ext/spdlog/include',
  ramulator2_path+'/ext/yaml-cpp/include'
])

4. Put the Ramulator2 wrapper code to gem5/src/mem/
5. Add the code to gem5/src/mem/SConscript to register the Ramulator2 SimObjects to gem5

if env['HAVE_RAMULATOR2']:
  SimObject('Ramulator2.py', sim_objects=['Ramulator2'])
  Source('ramulator2.cc')
  DebugFlag("Ramulator2")

6. Create the Ramulator2 SimObject as the memory controller and specify the path to the Ramulator 2.0 configuration file in your gem5 configuration script, e.g.,

import m5
from m5.objects import *

system = System()
system.mem_ctrl = Ramulator2()
system.mem_ctrl.config_path = "<path-to-config>.yaml" # Don't forget to specify GEM5 as the implementation of the frontend interface!

# Continue your configuration of gem5 ...

General Instructions for Writing Your Own Wrapper of Ramulator 2.0 for Another (including Your Own) Simulator

We describe the key steps and cover the key interfaces involved in using Ramulator 2.0 as a library for your own simulator.

1. Add Ramulator 2.0's key header files to your build system:

• ramulator2/src/base/base.h
• ramulator2/src/base/request.h
• ramulator2/src/base/config.h
• ramulator2/src/frontend/frontend.h
• ramulator2/src/memory_system/memory_system.h

2. Parse the YAML configuration for Ramulator 2.0 and instantiate the interfaces of the two top-level components using GEM5 as Frontend impl.

// MyWrapper.h
std::string config_path;
std::unique_ptr<Ramulator::IFrontEnd> ramulator2_frontend;
std::unique_ptr<Ramulator::IMemorySystem> ramulator2_memorysystem;

// MyWrapper.cpp
YAML::Node config = Ramulator::Config::parse_config_file(config_path, {});
ramulator2_frontend.reset(Ramulator::Factory::create_frontend(config));
ramulator2_memorysystem.reset(Ramulator::Factory::create_memory_system(config));

ramulator2_frontend->connect_memory_system(ramulator2_memorysystem.get());
ramulator2_memorysystem->connect_frontend(ramulator2_frontend.get());

3. Communicate the necessary memory system information from Ramulator 2.0 to your system (e.g., memory system clock):

float memory_tCK = ramulator2_memorysystem->get_tCK();

4. Send the memory requests from your simulator to Ramulator 2.0, with the correspoding callbacks that should be executed when the request is "completed" by Ramulator 2.0, e.g.,

if (is_read_request) {
  enqueue_success = ramulator2_frontend->
    receive_external_requests(0, memory_address, context_id, 
    [this](Ramulator::Request& req) {
      // your read request callback 
    });

  if (enqueue_success) {
    // What happens if the memory request is accepted by Ramulator 2.0
  } else {
    // What happens if the memory request is rejected by Ramulator 2.0 (e.g., request queue full)
  }
}

5. Tick the memory subsystem clock using your simulator's clocking

void cycle_advance() {
  ramulator2_memorysystem->tick();
}

6. Find a proper time and place to call the epilogue functions of Ramulator 2.0 when your simulator has finished execution, e.g.,

void my_simulator_finish() {
  ramulator2_frontend->finalize();
  ramulator2_memorysystem->finalize();
}

Example using external SystemC and TLM2.0 bridge module

#include <ramulator/base/base.h>
#include <ramulator/base/config.h>
#include <ramulator/frontend/frontend.h>
#include <ramulator/memory_system/memory_system.h>
#include <tlm>
#include <tlm_utils/simple_target_socket.h>

class ExampleTarget : public sc_core::sc_module {
public:
    tlm_utils::simple_target_socket<ExampleTarget> target_socket;

    explicit ExampleTarget(sc_core::sc_module_name name) {
        SC_HAS_PROCESS(ExampleTarget);
        SC_METHOD(tick);
        sensitive << memory_clock_event;
        target_socket.register_nb_transport_fw(this, &ExampleTarget::nb_transport_fw);
    }

    auto nb_transport_fw(tlm::tlm_generic_payload& payload, tlm::tlm_phase& phase,
                         sc_core::sc_time& time) -> tlm::tlm_sync_enum {
        auto enqueue_success = ramulator2_frontend->receive_external_requests(
                                    0, payload.get_address(), 0,
                                    [this](Ramulator::Request& req) { ExampleTarget::callback(req); });
        if (enqueue_success)
            // Success flow
        else
            // Failure flow
        payload.acquire();
        phase = END_REQ;
        return tlm::TLM_ACCEPTED;
    }

    auto before_end_of_elaboration() -> void override {
            auto config_path = "/path/to/config/file.yaml";
            auto config      = Ramulator::Config::parse_config_file(config_path, {});
            ramulator2_frontend.reset(Ramulator::Factory::create_frontend(config));
            ramulator2_memorysystem.reset(Ramulator::Factory::create_memory_system(config));
    
            ramulator2_frontend->connect_memory_system(ramulator2_memorysystem.get());
            ramulator2_memorysystem->connect_frontend(ramulator2_frontend.get());
    
            period = sc_core::sc_time{ramulator2_memorysystem->get_tCK(), sc_core::SC_NS};
    }

    auto end_of_simulation() -> void override {
        ramulator2_frontend->finalize();
        ramulator2_memorysystem->finalize();
    }

    auto callback(Ramulator::Request& req) -> void {
        // Callback flow
    }

    auto tick() {
        ramulator2_memorysystem->tick();
        memory_clock_event.notify(period);
    }
private:
      std::unique_ptr<Ramulator::IFrontEnd>     ramulator2_frontend;
      std::unique_ptr<Ramulator::IMemorySystem> ramulator2_memorysystem;
      sc_core::sc_event                         memory_clock_event;
      sc_core::sc_time                          period;
}

Extending Ramulator 2.0

Directory Structure

Ramulator 2.0

ext                     # External libraries
src                     # Source code of Ramulator 2.0
└ <component1>          # Collection of the source code of all interfaces and implementations related to the component
  └ impl                # Collection of the source code of all implementations of the component
    └ com_impl.cpp      # Source file of a specific implementation
  └ com_interface.h     # Header file that defines an interface
  └ CMakeList.txt       # Component-level CMake configuration
└ ...                    
└ CMakeList.txt         # Top-level CMake configuration of all Ramulator 2.0's source files
CMakeList.txt           # Project-level CMake configuration

Interface and Implementation

To achieve high modularity and extensibility, Ramulator 2.0 models the components in the memory system using two concepts, Interfaces and Implementations:

• An interface is a high-level abstraction of the common high-level functionality of a component as seen by other components in the system. It is an abstract C++ class defined in a .h header file, that provides its functionalities through virtual functions.
• An implementation is a concrete realization of an interface that models the actual behavior of the object. It is usually a C++ class that inherits from the interface class it is implementing that provides implementations of the interface's virtual functions.

An example interface class looks like this:

// example_interface.h
#ifndef     RAMULATOR_EXAMPLE_INTERFACE_H
#define     RAMULATOR_EXAMPLE_INTERFACE_H

// Defines fundamental data structures and types of Ramulator 2.0. Must include for all interfaces.
#include "base/base.h"  

namespace Ramulator {
class ExampleIfce {
  // One-liner macro to register this "ExampleIfce" interface with the name "ExampleInterface" to Ramulator 2.0.
  RAMULATOR_REGISTER_INTERFACE(ExampleIfce, "ExampleInterface", "An example of an interface class.")
  public:
    // Model common behaviors of the interface with virtual functions 
    virtual void foo() = 0;
};
}        // namespace Ramulator

#endif   // RAMULATOR_EXAMPLE_INTERFACE_H

An example implementation that implements the above interface looks like this

// example_impl.cpp
#include <iostream>

// An implementation should always include the header of the interface that it is implementating
#include "example_interface.h"

namespace Ramulator {
// An implementation class should always inherit from *both* the interface it is implementating, and the "Implementation" base class
class ExampleImpl : public ExampleIfce, public Implementation  {
  // One-liner macro to register and bind this "ExampleImpl" implementation with the name "ExampleImplementation" to the "ExampleIfce" interface.
  RAMULATOR_REGISTER_IMPLEMENTATION(ExampleIfce, ExampleImpl, "ExampleImplementation", "An example of an implementation class.")
  public:
    // Implements concrete behavior
    virtual void foo() override {
      std::cout << "Bar!" << std::endl;
    };
};
}      // namespace Ramulator

Adding an Implementation to an Existing Interface

Let us consider an example of adding an implementation "MyImpl" for the interface "MyIfce", defined in my_ifce.h, that belongs to a component "my_comp".

1. Create a new .cpp file under my_comp/impl/ that contains the source code for the implementation. Say this file is "my_impl.cpp". The directory structure for my_comp will look like this:

my_comp                     
  └ impl
    └ my_impl.cpp      
  └ my_interface.h     
  └ CMakeList.txt       

2. Provide the concrete class definition for MyImpl in my_impl.cpp, following the structure explained above. It is important to do the following two things when defining your class:
   • Make sure you inherit from both the interface class that you are implementing and the Implementation base class
   • Make sure to include the macro RAMULATOR_REGISTER_IMPLEMENTATION(...) inside your implementation class definition. You should always specify which interface class your class is implementing and the stringified name of your implementation class in the macro. Assume you have the following registration macro for the interface class

class MyIfce {
  RAMULATOR_REGISTER_INTERFACE(MyIfce, "MyInterfaceName", "An example of my interface class.")
}

and the following for the implementation class

class MyImpl : public MyIfce, public Implementation {
  RAMULATOR_REGISTER_IMPLEMENTATION(MyIfce, MyImpl, "MyImplName", "An example of my implementation class.")
}

If everything is correct, Ramulator 2.0 will be able to automatically construct a pointer of type MyIfce*, pointing to an object of type MyImpl, when it sees the following in the YAML configuration file:

MyInterfaceName:
  impl: MyImplName

3. Finally, add impl/my_impl.cpp to target_sources in CMakeList.txt so that CMake knows about the newly added source file.

Adding a New Interface (or New Component)

The process is similar to that of adding a new implementation, but you will need to create the interface and add them to CMakeList.txt under the corresponding component directory. If you add a new component (i.e., create a new directory under src/) you will need to add this new directory to the CMakeList.txt file under src/, i.e.,

add_subdirectory(my_comp)

Verifying Ramulator 2.0's Memory Controller and DRAM Model

We use the Verilog model from Micron to verify that the DRAM commands issued by Ramulator 2.0 do not cause timing or state transition violations.

1. Generate the instruction traces

cd verilog_verification
cd traces
python3 tracegen.py --type SimpleO3 --pattern {stream,random} --num-insts ${NUM_INSTS} --output ${TRACE_FILE} --distance ${MEMREQ_INTENSITY}

2. Collect the DRAM command trace (with addresses and time stamps) from the simulations

cd ..
./ramulator2 -f ./verification-config.yaml

3. Configure the Verilog model to match the configuration used by Ramulator 2.0:

• DRAM Organization: "DDR4_8G_X8"
• DRAM Frequency: "DDR4_2400"
• Number of Ranks: 2

We provide the already configured Verilog files in verilog_verification/sources/.

4. Convert the DRAM Command Trace to fit the testbench of the Verilog model. We provide a script verilog_verification/trace_converter.py to do so.

python3 trace_converter.py DDR4_8G_X8 2 DDR4_2400

5. Then you can just start your Verilog simulator (e.g., ModelSim) and check for violations. We provide a script to parse the simulation output and check for errors verilog_verification/trace_verifier.py

python3 trace_verifier.py <trace_filepath> <output_filepath>

Reproducing the Results in our Ramulator 2.0 paper

Simulation Performance Comparison with Other Simulators

We put all scripts and configurations in perf_comparison/

1. Get simulators from their respective sources and put their source code at perf_comparison/simulators/

cd perf_comparison
mkdir simulators
cd simulators
git clone https://github.com/umd-memsys/DRAMSim2.git    # DRAMSim2
git clone https://github.com/umd-memsys/DRAMsim3        # DRAMSim3
wget http://www.cs.utah.edu/~rajeev/usimm-v1.3.tar.gz   # USIMM v1.3
tar xvzf ./usimm-v1.3.tar.gz
git clone https://github.com/CMU-SAFARI/ramulator.git   # Ramulator 1.0
mv ramulator ramulatorv1
git clone https://github.com/CMU-SAFARI/ramulator2.git  # Ramulator 2.0
mv ramulator2 ramulatorv2

2. Apply patches to DRAMSim2, DRAMSim3, and USIMM to remove some of their hardcoded system configurations and unify the termination criteria of all simulators for a fair comparison. We do not change the core modeling and simulation code of these simulators.

cd DRAMSim2
git apply ../../DRAMSim2-patch.patch
cd ..
cd DRAMsim3
git apply ../../DRAMsim3-patch.patch
cd ..

3. Build all simulators

cd ..
./build_simulators.sh

4. Generate traces

cd traces
./gen_all_traces.sh

5. Run the simulators with comparable system and DRAM configurations at perf_comparison/configs/ and record runtimes

python3 perf_comparison.py

Cross-Sectional Study of Various RowHammer Mitigation Techniques

We put all scripts and configurations in rh_study/

1. Get the instruction traces from SPEC 2006 and 2017

cd rh_study
wget <download_link>  # We host the traces here https://drive.google.com/file/d/1CvAenRZQmmM6s55ptG0-XyeLjhMVovTx/view?usp=drive_link
tar xvzf ./cputraces.tar.gz

2. Generate workloads from trace combinations

python3 get_trace_combinations.py

3. Run the single-core and multi-core simulations (assuming using slurm, if not, please change line 68 of run_singlecore.py and line 73 of run_multicore.py to fit your job scheduler)

python3 run_singlecore.py
python3 run_multicore.py

4. Execute the notebook plot.ipynb to plot the results