Use rebar3 for compilation and AVM packbeam generation. #971
Conversation
|
I just did a test run with this PR and it looks like it is still building the C |
The Elixir build still requires PackBEAM. |
|
Right! I should have realized that. |
fadushin
force-pushed
the
rebar-build
branch
2 times, most recently
from
f1dd0f2 to
78c57cf
Compare
Signed-off-by: Fred Dushin <[email protected]>
fadushin
force-pushed
the
rebar-build
branch
from
78c57cf to
1c02169
Compare
libs/atomvmlib/.gitignore
Outdated
| # | ||
| # SPDX-License-Identifier: Apache-2.0 OR LGPL-2.1-or-later | ||
| rebar.config | ||
| rebar.lock |
There was a problem hiding this comment.
shouldn't we version lock files?
There was a problem hiding this comment.
Probabaly not in this case, because in most cases the dependencies are internal to this project, and we do not version them; nod do we put them in GitHub repositories structured in such a way that they can be pulled down.
libs/atomvmlib/.gitignore
Outdated
| # SPDX-License-Identifier: Apache-2.0 OR LGPL-2.1-or-later | ||
| rebar.config | ||
| rebar.lock | ||
| _checkouts/* |
There was a problem hiding this comment.
Sources directory is getting polluted in different places:
libs/atomvmlib/_checkouts/
libs/atomvmlib/rebar.config
libs/atomvmlib/rebar.lock
libs/eavmlib/rebar.config
libs/eavmlib/rebar.lock
libs/esp32boot/_checkouts/
libs/esp32boot/rebar.config
libs/esp32boot/rebar.lock
libs/esp32devmode/_checkouts/
libs/esp32devmode/rebar.config
libs/esp32devmode/rebar.lock
libs/estdlib/rebar.config
libs/estdlib/rebar.lock
libs/etest/rebar.config
libs/etest/rebar.lock
tests/code_load/rebar.config
tests/code_load/rebar.lock
tests/erlang_tests/include/code_load_pack_data.hrl
tests/erlang_tests/include/export_test_module_data.hrl
tests/erlang_tests/rebar.config
tests/erlang_tests/rebar.lock
tests/libs/eavmlib/_checkouts/
tests/libs/eavmlib/rebar.config
tests/libs/eavmlib/rebar.lock
tests/libs/estdlib/_checkouts/
tests/libs/estdlib/rebar.config
tests/libs/estdlib/rebar.lock
We need to fix this before merging. Adding _checkouts/* to .gitignore is a workaround.
Honestly my plan here is quite the opposite and I would further modularize libraries, such as I mention here #710. |
|
Tested release workflows, and I've got the following error: Log: |
| COMMAND mkdir -p _checkouts && cd _checkouts && ln -f -s ${CMAKE_SOURCE_DIR}/libs/${dep} | ||
| VERBATIM | ||
| ) | ||
| endforeach() | ||
|
|
||
| add_custom_command( | ||
| OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/src | ||
| COMMAND ln -s -f ${CMAKE_CURRENT_SOURCE_DIR}/src |
There was a problem hiding this comment.
We don't support it right now, but it would be nice (see #8) and this change would completely rule it out.
There was a problem hiding this comment.
Well, not unless we use something like mklink. I suspect there are many other issues with Windows that would need to be addressed.
|
Converted to DRAFT until I re-work the rebar3 projects in the build tree |
|
I am going to keep this branch open but close the issue, for a rethink of strategies. |
This PR changes the build of Erlang libraries and tests to use rebar3 as a build tool, instead of calling the Erlang compiler directly. The
atomvm_rebar3_packbeamrebar3 plugin is used to create PackBEAM (.avm) files, instead of the C-basedPackBEAMtool.This PR includes the following changes:
libsdirectory have been modifed to comply withrebar3build conventions, including a src directory, an<application-name>.app.srcfile for each library, and arebar.config.infile, for which CMake rules will generate arebar.configwith proper paths. Many of the changes in this directory also relate simply to file moves, in order to comply withrebar3build conventions.alispproject has been folded into theeavmlibproject, as we have no use case for building or deploying these modules separately.estdlibandeavmlib, have been modified to also userebar3. Many of the changes in this directory relate simply to file moves, in order to comply withrebar3build conventions.ExAtomVMplugin. That work can come later, once theExAtomVMtool reaches functional parity with theatomvm_rebar3_plugin.atomvm_examplesrepository in the future.rebar3build system, though some of the logic has been copied from the base build system. Due to a potentially missing feature in theatomvm_rebar3_pluginuf2createtask, we have left use of theuf2tool.erlescript. A TODO has been left to identify the issue.rebar3as well as various adjustments to resulting paths.Benchmarks
The following benchmarks were run on MacOS, FreeBSD, and Linux systems (though the Linux system is running virtualized on FreeBSD, so numbers there should be taken with a grain of salt). Comparisons are made between build times on the
masterand this branch, for parallel and sequential builds usingmakeandninja, for all three platforms.Results are represented in the following table:
master)rebar-build)make -j 8make -j 8ninjaninjamakemakemake -j 8make -j 8ninjaninjamakemakemake -j 8make -j 8ninjaninjamakemakeSummary:
masterrebar3masterare extremely slow. They are better withrebar3, as at least chunks of the Erlang code are built in parallel. This is especially true of the erlang unit tests.Future Improvements
atomvm_rebar3_plugintees up support for the OTP Application behavior, as the plugin will generateapplication.binfiles from the created.appfiles, which are used at runtime by the OTP Application controller to determine which applications to load.mixand theExAtomVMplugin, but we have some work to do on the plugin to brin git up to parity. We have left the Elixir builds as is.Notes
FreeBSD freenas.local 13.1-RELEASE-p7 FreeBSD 13.1-RELEASE-p7 n245428-4dfb91682c1 TRUENAS amd64Linux debian 4.19.0-24-amd64 #1 SMP Debian 4.19.282-1 (2023-04-29) x86_64 GNU/LinuxThese changes are made under both the "Apache 2.0" and the "GNU Lesser General
Public License 2.1 or later" license terms (dual license).
SPDX-License-Identifier: Apache-2.0 OR LGPL-2.1-or-later