Skip to content

Memory Management

Jump to bottom
Gary Miguel edited this page Oct 20, 2021 · 2 revisions

BFCArena

  • Why we need an arena? What problem does it solve?

Memory Planning

Memory Pattern

  • How does ORT come up with a peak memory consumption?

External Torch's CUDACachingAllocator

  • How does ORTModule uses pytorch allocator? See _graph_execution_manager.py
  • Advanced: What's the difference between BFCArena and CUDACachingAllocator?

Please use the learning roadmap on the home wiki page for building general understanding of ORT.

Navigation by topic

Upcoming Release Roadmap

Glossary

Development

Common Tasks

Dependencies

Core Architecture

Feature Details

Inferencing

Training

Clone this wiki locally