Add 3DS memory map #446
Add 3DS memory map #446
Conversation
The "real" addresses set here are based on 3DS virtual memory addresses, with this memory map intended to map 1:1 with 3DS userland virtual memory Unsure if this would end up being desirable in practice. This has the benefit of making points map 1:1 to RetroAchievements memory space (which is a huge win given how pointer heavy 3DS games will be and the disjointed mess that "real" memory lies in the memory map). This will, however, balloon the memory space to search in for developers. BizHawk (the current 3DS RetroAchievements capable emulator), as an implementation detail, also slightly suffers for the achievement runtime as it will allocate 1GiB here for a memory block LUT; this is an implementation detail, however, just due to being a multi-system emulator, a standalone emulator wouldn't have such, and even so 1GiB of memory is relatively free for any machine that can handle 3DS emulation anyways. A different memory map could potentially be made which only covers the "real" memory spaces, but doing so will still have a large amount of memory mapped (definitely more than 256MiB, the size of the New 3DS's FCRAM), and will be much harder for the current achievements implementation to handle pointers (given the reported problems with even handling MEM1/MEM2 on the Wii).
CasualPokePlayer
force-pushed
the
3ds_memory_map
branch
from
fba3c6e to
b1d1984
Compare
|
Does this mean that if I am searching RAM and find a pointer that points to the new linear heap , it'll have a value in the 0x30000000-0x37FFFFFF range? |
That is usually correct (it's technically more 0x30000000-0x3FFFFFFF, only 0x30000000-0x37FFFFFF if you're on the old 3DS) |
Jamiras
left a comment
Jamiras
left a comment
There was a problem hiding this comment.
Sorry it took me so long to get to this. My summer was super busy, and I've been playing catch-up ever since.
| TEST_PARAMS2(test_memory, RC_CONSOLE_NEO_GEO_CD, 0x010000); | ||
| TEST_PARAMS2(test_memory, RC_CONSOLE_NINTENDO, 0x010000); | ||
| TEST_PARAMS2(test_memory, RC_CONSOLE_NINTENDO_64, 0x800000); | ||
| TEST_PARAMS2(test_memory, RC_CONSOLE_NINTENDO_3DS, 0x40000000); |
There was a problem hiding this comment.
This will need to be updated to account for changes in #462.
TEST_PARAMS3(test_memory, RC_CONSOLE_NINTENDO_3DS, 0x40000000, 0x2CE00000);
I'm a bit weary that there's still 750MB of searchable memory given the system only has 256MB of RAM.
The Code Binary segment, IPC Buffers segment, and Shared Memory segments are each 64MB. The Regular Heap and Linear Heap segments are each 128MB. There's also another ~48MB of miscellaneous sections that aren't the primary 256MB of RAM.
Would achievement developers be expected to find data in all of those regions? Anything that's not marked as UNUSED is currently searchable. Would it make sense to add new enum values for non-searchable system ram?
Maybe they could be marked as VIRTUAL_RAM and then have that made as non-searchable. Currently VIRTUAL_RAM is only being used for echo/mirror RAM segments, and eliminating those from search results would probably actually benefit developers on those systems.
There was a problem hiding this comment.
The Code Binary segment is something I would expect would be needed to looked at, as that doesn't just contain the .text, but also contains .data and .bss which are likely going to contain variables (and especially base pointers for fun pointer hell).
IPC Buffers and Shared Memory are unlikely to contain anything actually useful, as that's for handling communications with other 3DS processes (generally speaking this just means talking to the OS for various services, or in the case of emulators, the emulator HLE'ing those services). This is why those are listed as RC_MEMORY_TYPE_HARDWARE_CONTROLLER as they effectively work as "modern" MMIO (for unprivileged userland anyways).
| { 0x1FF82000U, 0x1FFFFFFFU, 0x1FF82000U, RC_MEMORY_TYPE_SYSTEM_RAM, "Thread Local Storage" }, /* Most of this is actually unused in practice */ | ||
| { 0x20000000U, 0x2FFFFFFFU, 0x20000000U, RC_MEMORY_TYPE_UNUSED, "" }, | ||
| { 0x30000000U, 0x37FFFFFFU, 0x30000000U, RC_MEMORY_TYPE_SYSTEM_RAM, "New Linear Heap" }, /* Newer games use this as the linear heap base instead */ | ||
| { 0x38000000U, 0x3FFFFFFFU, 0x38000000U, RC_MEMORY_TYPE_SYSTEM_RAM, "New Linear Heap (New 3DS Exclusive)" } /* New 3DS exclusive space for the newer linear heap */ |
There was a problem hiding this comment.
This map makes sense given the referenced documentation, but only these last two segments are the physical memory reported in the system specs. Where does all the other SYSTEM_RAM come from?
There was a problem hiding this comment.
I'm not sure what you mean here. To be clear, not all of the ranges will necessarily be addressed to actual memory (they would be mapped on the fly as allocations happen), not even neccessarily contiguous in their mapping, they're just the maximum ranges (assuming such memory is available in practice, in practice you'd OOM before you'd be able to do that). Most of the ranges here would possibly be addressed to some part of the 128/256MiB FCRAM (including things like IPC Buffers and Shared Memory, notwithstanding them marked as RC_MEMORY_TYPE_HARDWARE_CONTROLLER).
There was a problem hiding this comment.
The hardware specs for 3DS say 128MB of RAM, and the hardware specs for New3DS say 256MB of RAM. I'm just confused where the data in all the other blocks marked as SYSTEM_RAM is being stored.
If it's just mapping data from the "New Linear Heap" segments, it could be marked as VIRTUAL_RAM instead of SYSTEM_RAM.
Ideally, we need some way to restrict the searchable memory space. Otherwise, when a dev captures memory to look for changes, they'll have to capture the full 750MB for the non-UNUSED memory blocks, and there could be redundant matches where memor is mirrored. Then, we would need some way to guide the developer into preferring one over an other.
There was a problem hiding this comment.
Again, not all of the range is necessarily going to be mapped, and there's no guarantee different mappings will be contiguous. Linear Heap allocations won't be overlapping with other memory allocations, so ranges where other memory has already been allocated will simply be "skipped" when an allocation occurs in order to find a suitable location for the allocation. For example, 0x30000000-0x30000FFF, being strictly mappable to FCRAM+0 to 0xFFF might not be able to be allocated for the linear heap if something allocated the FCRAM what would be there (noting that other memory regions, e.g. Code Binary and the Regular Heap, aren't strict about physical FCRAM locations), so it might proceed to look at 0x30001000-0x30001FFF, and so on.
There was a problem hiding this comment.
To be clear too, when I say an area is "not mapped" that means that say, if the game were to read from that area on a console, it'd just segfault (data abort exception), although on emulator and for the purpose of peeking memory, 0s would be returned.
3 participants
The "real" addresses set here are based on 3DS virtual memory addresses, with this memory map intended to map 1:1 with 3DS userland virtual memory
Unsure if this would end up being desirable in practice. This has the benefit of making pointers map 1:1 to RetroAchievements memory space (which is a huge win given how pointer heavy 3DS games will be and the disjointed mess that "real" memory lies in the memory map). This will, however, balloon the memory space to search in for developers. BizHawk (the current 3DS RetroAchievements capable emulator), as an implementation detail, also slightly suffers for the achievement runtime as it will allocate 1GiB here for a memory block LUT; this is an implementation detail, however, just due to being a multi-system emulator, a standalone emulator wouldn't have such, and even so 1GiB of memory is relatively free for any machine that can handle 3DS emulation anyways.
A different memory map could potentially be made which only covers the "real" memory spaces, but doing so will still have a large amount of memory mapped (definitely more than 256MiB, the size of the New 3DS's FCRAM), and will be much harder for the current achievements implementation to handle pointers (given the reported problems with even handling MEM1/MEM2 on the Wii).