Skip to content

Commit e6a6f74

Browse files
authored
Improve memory workaround docs (#2591)
1 parent 3708a45 commit e6a6f74

File tree

1 file changed

+54
-12
lines changed

1 file changed

+54
-12
lines changed

source/docs/software/basic-programming/java-gc.rst

Lines changed: 54 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,13 @@ If you anticipate your application creating a large number of short-lived object
3535
Fixing Out of Memory Errors
3636
---------------------------
3737

38-
If the JVM cannot allocate memory, the program will be terminated. As an embedded system with only a small amount of memory available (256 MB on the roboRIO 1, 512 MB on the roboRIO 2), the roboRIO is particularly susceptible to running out of memory. If you continue to run out of memory even after investigating with VisualVM and taking steps to minimize the number of allocated objects, a few different options are available to make additional memory available to the robot program.
38+
If the JVM cannot allocate memory, the program will be terminated. As an embedded system with only a small amount of memory available (256 MB on the roboRIO 1, 512 MB on the roboRIO 2), the roboRIO is particularly susceptible to running out of memory.
39+
40+
.. admonition :: No amount of system tuning can fix out of memory errors caused by out-of-control allocations.
41+
42+
If you are running out of memory, always investigate allocations with :doc:`VisualVM </docs/software/advanced-gradlerio/profiling-with-visualvm>` first.
43+
44+
If you continue to run out of memory even after investigating with VisualVM and taking steps to minimize the number of allocated objects, a few different options are available to make additional memory available to the robot program.
3945

4046
- Disabling the system web server
4147
- Setting sysctls (Linux kernel options)
@@ -47,26 +53,48 @@ Implementing most of these options require :doc:`connecting with SSH </docs/soft
4753
Disabling the System Web Server
4854
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
4955

50-
The built-in NI system web server provides the webpage seen when using a web browser to connect to the roboRIO. It also is used by the Driver Station's data log download functionality. However, it consumes several MB of RAM, so disabling it will free up that memory for the robot program to use. There are several ways to disable the web server:
56+
The built-in NI system web server provides the webpage (the :doc:`roboRIO Web Dashboard </docs/software/roborio-info/roborio-web-dashboard>`) seen when using a web browser to connect to the roboRIO, e.g. to change IP address settings. It also is used by the Driver Station's data log download functionality. However, it consumes several MB of RAM, so disabling it will free up that memory for the robot program to use. There are several ways to disable the web server:
5157

52-
The first and easiest is to use the RoboRIO Team Number Setter tool. Versions 2024.2.1 and later of the tool have a button to disable or enable the web server. However, a few teams have reported that this does not work or does not persist between reboots. There are two alternate ways to disable the web server; both require connecting to the roboRIO with SSH and logging in as the ``admin`` user.
58+
The first and easiest is to use the :doc:`RoboRIO Team Number Setter </docs/software/wpilib-tools/roborio-team-number-setter/index>` tool. Versions 2024.2.1 and later of the tool have a button to disable or enable the web server. However, a few teams have reported that this does not work or does not persist between reboots. There are two alternate ways to disable the web server; both require connecting to the roboRIO with SSH and logging in as the ``admin`` user.
5359

5460
1. Run ``/etc/init.d/systemWebServer stop; update-rc.d -f systemWebServer remove; sync``
5561

56-
2. Run ``chmod a-x /usr/local/natinst/etc/init.d/systemWebServer``
62+
2. Run ``chmod a-x /usr/local/natinst/etc/init.d/systemWebServer; sync``
63+
64+
To revert the alternate ways and re-enable the web server, take the corresponding step:
65+
66+
1. Run ``update-rc.d -f systemWebServer defaults; /etc/init.d/systemWebServer start; sync``
67+
68+
2. Run ``chmod a+x /usr/local/natinst/etc/init.d/systemWebServer; sync``
5769

5870
Setting sysctls
5971
^^^^^^^^^^^^^^^
6072

61-
Several Linux kernel options (called sysctls) can be set to tweak how the kernel allocates memory. Several options have been found to reduce out-of-memory errors. Setting these options requires connecting to the roboRIO with SSH and logging in as the ``admin`` user, then running the following commands:
73+
Several Linux kernel options (called sysctls) can be set to tweak how the kernel allocates memory. Several options have been found to reduce out-of-memory errors:
74+
75+
- Setting ``vm.overcommit_memory`` to 1 (the default value is 2). This causes the kernel to always pretend there is enough memory for a requested memory allocation at the time of allocation; the default setting always checks to see if there's actually enough memory to back an allocation at the time of allocation, not when the memory is actually used.
76+
- Setting ``vm.vfs_cache_pressure`` to 1000 (the default value is 100). Increasing this causes the kernel to much more aggressively reclaim file system object caches; it may slightly degrade performance.
77+
- Setting ``vm.swappiness`` to 100 (the default value is 60). This causes the kernel to more aggressively swap process memory to the swap file. Changing this option has no effect unless you add a swap file.
78+
79+
You can set some or all of these options; the most important one is ``vm.overcommit_memory``. Setting these options requires connecting to the roboRIO with SSH and logging in as the ``admin`` user, then running the following commands:
6280

6381
.. code-block:: text
6482
65-
echo "swappiness=100" > /etc/sysctl.conf
66-
echo "vfs_cache_pressure=1000" >> /etc/sysctl.conf
67-
echo "overcommit_memory=1" >> /etc/sysctl.conf
83+
echo "vm.overcommit_memory=1" >> /etc/sysctl.conf
84+
echo "vm.vfs_cache_pressure=1000" >> /etc/sysctl.conf
85+
echo "vm.swappiness=100" >> /etc/sysctl.conf
6886
sync
6987
88+
The ``/etc/sysctl.conf`` file should contain the following lines at the end when done (to check, you can run the command ``cat /etc/sysctl.conf``):
89+
90+
.. code-block:: text
91+
92+
vm.overcommit_memory=1
93+
vm.vfs_cache_pressure=1000
94+
vm.swappiness=100
95+
96+
To revert the change, edit ``/etc/sysctl.conf`` (this will require the use of the vi editor) and remove these 3 lines.
97+
7098
Periodically Calling the Garbage Collector
7199
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
72100

@@ -90,16 +118,30 @@ Sometimes the garbage collector won't run frequently enough to keep up with the
90118
Setting Up Swap on a USB Flash Drive
91119
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
92120

93-
A swap file on a Linux system provides disk-backed space that can be used by the system as additional virtual memory to put infrequently used data and programs when they aren't being used, freeing up physical RAM for active use such as the robot program. It is strongly recommended to not use the built-in non-replaceable flash storage on the roboRIO 1 for a swap file, as it has very limited write cycles and may wear out quickly. Instead, however, a FAT32-formatted USB flash drive may be used for this purpose. This does require the USB flash drive to always be plugged into the roboRIO before boot. A swap file can be set up by plugging the USB flash drive into the roboRIO USB port, connecting to the roboRIO with SSH and logging in as the ``admin`` user, and running the following commands:
121+
A swap file on a Linux system provides disk-backed space that can be used by the system as additional virtual memory to put infrequently used data and programs when they aren't being used, freeing up physical RAM for active use such as the robot program. It is strongly recommended to not use the built-in non-replaceable flash storage on the roboRIO 1 for a swap file, as it has very limited write cycles and may wear out quickly. Instead, however, a FAT32-formatted USB flash drive may be used for this purpose. This does require the USB flash drive to always be plugged into the roboRIO before boot.
122+
123+
.. caution:: Having a swap file on a USB stick means it's critical the USB stick stay connected to the roboRIO at all times it is powered.
124+
125+
This should be used as a last resort if none of the other steps above help. Generally needing swap is indicative of some other allocation issue, so use VisualVM first to optimize allocations.
126+
127+
A swap file can be set up by plugging the USB flash drive into the roboRIO USB port, connecting to the roboRIO with SSH and logging in as the ``admin`` user, and running the following commands. Note the vi step requires knowledge of how to edit and save a file in vi.
94128

95129
.. code-block:: text
96130
97131
fallocate -l 100M /u/swapfile
98132
mkswap /u/swapfile
99133
swapon /u/swapfile
100-
echo "#!/bin/sh" > /etc/init.d/addswap.sh
101-
echo "[ -x /sbin/swapon ] && swapon --ifexists /u/swapfile" >> /etc/init.d/addswap.sh
102-
echo ": exit 0" >> /etc/init.d/addswap.sh
134+
vi /etc/init.d/addswap.h
103135
chmod a+x /etc/init.d/addswap.sh
104136
update-rc.d -v addswap.sh defaults
105137
sync
138+
139+
The ``/etc/init.d/addswap.sh`` file contents should look like this:
140+
141+
.. code-block:: text
142+
143+
#!/bin/sh
144+
[ -x /sbin/swapon ] && swapon -e /u/swapfile
145+
: exit 0
146+
147+
To revert the change, run ``update-rc.d -f addswap.sh remove; rm /etc/init.d/addswap.sh; sync; reboot``.

0 commit comments

Comments
 (0)