Porting

Porting Soletta™ Framework to a new OS requires a good level of knowledge of both the target OS, and how Soletta deals with them. The following sections detail the necessary steps on the Soletta side, from specifying the toolchain, to how platform specific implementations are handled in the code.

TL;DR

Let's start with a quick overview of all required steps to port Soletta to a new platform:

• Set up proper toolchain, by setting the TOOLCHAIN_PREFIX variable accordinly (eg. arm-none-eabi-)
• Set proper CFLAGS and LDFLAGS
• In data/jsons/dependencies.json, add a way to uniquely identify the OS, as well as any other dependencies you might want to check
• In Kconfig, for the BASE_OS entry, add one matching your OS, as well as the respective file in tools/build/Kconfig.$BASE_OS
  • Use one of the pre-existing ones as a reference on how to enable Soletta features as they are ported
• In src/lib/common/include/sol-mainloop.h, if necessary, define SOL_MAIN_DEFAULT to match the application entry point as required by your OS
• Implement the main loop primitives in src/lib/common/sol-mainloop-impl-$YOUROS.c, add the file and any other needed in src/lib/common/Makefile
• With the basic main loop working, move on to the other subsystems

Build system and dependencies

Soletta uses Kconfig to configure the build, and a series of custom made Makefiles to execute that build. In order for the configuration to have options that make sense under the system it's going to be built for, it needs to check that any required dependencies are met. This is achieved by the build system running a script that will generate a set of basic configuration entires for Kconfig to use, and for the dependencies to be checked correctly, the right toolchain needs to be used.

The straightforward way to set the toolchain is by setting the variable TOOLCHAIN_PREFIX to the corresponding prefix before calling make, for example

TOOLCHAIN_PREFIX=i586-none-linux- make alldefconfig

The build system will also take any CFLAGS and LDFLAGS set, and these are used for the dependency checks as well, so if any flags are required for the build to work for the target, they must be passed. These flags, however, are not cached, so they must be passed every time make is ran to ensure the build will work as expected.

With the toolchain correctly set, the build system will run the dependency checker. This is a python script that reads a set of rules from data/jsons/dependencies.json inside Soletta project's source directory. This script serves several purposes.

• It sets a few define values that are used internally in Soletta.
• Checks if the compiler and linker support a set of desired flags, and if so, it adds them to the flags to be used for the build.
• Checks the availability of some optional programs that are used by some build targets, like valgrind, for the test suite, or doxygen, to generate documentation.
• Checks if build time dependencies are available. Based on these dependencies, Kconfig will enable the selection of some features, and it's also the way to detect what the base OS is being targeted.

This file contains three sections:

• definitions: Straight set of definitions to make available to the code.
• pre-dependencies: Early dependencies, these might be required during the next set of dependencies check.
• dependencies: The set of dependencies to check.

The dependencies themselves can be of several types, but all have the same form. A dependency is declared as a JSON object with two required keys, dependency and type, and others that will depend on the type. The dependency key is the name by which a particular dependency will be known by the rest of the build system. If its conditions are met, it will be made available to Kconfig as the upper case version of that name, prefixed by HAVE_. So, for example, a fulfilled dependency foo would turn into HAVE_FOO.

Dependency types

C code

Type ccode, it will create a little test program containing the code fragment given, and build using the CFLAGS and LDFLAGS passed to make, as well as those obtained from the pre-dependencies. It can be used to check the availability of header files, specific definitions or functions.

Supported keys:

• defines: An array of definitions to define in the test program. For example, "_GNU_SOURCE 1" would put in the program #define _GNU_SOURCE 1
• headers: An array of header files to include in the test program. For example, <sys/socket.h>
• fragment: Fragment of C code that will be compiled. The wrapping main() function is provided by the script.
• cflags: An object, with a single value key containing a string to be added to the CFLAGS to be used to build the test. These will be added to Soletta project's if the dependency is used during the build.
• ldflags: Same as cflags, but for LDFLAGS.

Pkg-config

Type pkg-config, uses the pkg-config tool to check if a package is available. If the conditions are met, the CFLAGS and LDFLAGS the package needs to build against will be added to Soletta project's, as well as the list of packages it needs to link against in the case of a static build.

Supported keys:

• pkgname: Required. The name of the package to check for.
• atleast-version: Minimum version the package must be.
• max-version: Maximum version the package may be.
• exact-version: The package must match that exact version.

Filesystem

Type filesystem, used to check the existence of specific files in a given list of directories. If they are found, the Kconfig entry is set to y and a variable made of the upper cased dependency, suffixed with _PATH is made available in the Makefiles. It can be used to see if optional dependencies are available, such as sub-modules in Soletta itself, or located somewhere outside its source directory. The paths in the list that will be looked in can contain {VARIABLES} that will be expanded from the environment variables used to run the script, with the special variable {TOP_SRCDIR} expanding to the absolute path of Soletta framework's source directory.

Supported keys:

• files: An array of strings with the file names to look for.
• path: An object, where each key serves mostly as documentation, and its value is the path to be searched for the files.

Executables

Type exec, checks that a given command can be executed.

Supported keys:

• exec: Required, the command to check for.
• required: If set to true, the build will fail if the command cannot be executed.

Python modules

Type python, checks that the given Python module is available. It does so by trying to import the module as given.

Supported keys:

• pkgname: The package the check for.
• required: If set to true, the build will fail if the module import fails.

CFLAGS and LDFLAGS

Types cflags and ldflags, respectively. Check if the compiler and linker support each of the given flags, and if they do, they are added to the flags used during the build.

Supported keys:

• cflags or ldflags, respectively: An array of strings, each string being a flag to test.
• append_to: The Makefile variable where the flags should be added.

Detecting the OS

In order for Kconfig to know what configuration options are available for the target OS, it needs to first know what that OS is. We do this by treating the OS as a dependency like any other. These dependencies should be something that identifies as uniquely as possible the target OS. For example, for Linux and Contiki, there are checks to see if we can build using specific headers, respectively, <linux/ioctl.h> and <contiki.h>. For RIOT, the build will always define RIOT_CPU.

Once the dependency checker recognizes the OS, the main Kconfig file checks for the presence of these variables and loads an OS specific file that tells the configuration system what features are available for that OS, given the detected dependencies. These files are located under tools/build/Kconfig.$OSNAME.

For example, when building for Linux, HAVE_LINUX will be set to y, and so the file tools/build/Kconfig.linux will be loaded.

FEATURE based configuration

The function of the OS specific Kconfig files is to select the features supported by Soletta under that OS. These features may be disabled if dependencies are not met.

The convention followed by Soletta project's build system is to have the dependency checker create variables of the form HAVE_FOO, that are then checked in Kconfig to enable other variables of the form FEATURE_BAR. These FEATURE entries are used by the rest of configuration system to make options available to the user without cluttering the Kconfig files with lists of dependencies. It also makes porting to Soletta to a new OS easier, since FEATUREs can be enabled as they are implemented, in a single Kconfig file, without worrying about changing every option to be aware of the new OS.

Main loop

With Soletta being an event oriented framework, the main loop becomes the core that ties all the other components together. As such, it should be the first thing to be ported. Some of the primitives the main loop support have a common implementation ready to be used by any port, such as idlers and timeouts. Others are highly dependant on features present in the underlying OS and may be left out if they don't make sense.

Application entry point

Different OSes might have different requirements on how the entry point of the application should be defined. Soletta takes advantage of how event driven applications have a common format, there's a setup phase, the main loop that will trigger events, and a shutdown after the main loop ends. With this in mind, the portable way to define the entry point of a Soletta application is to use the SOL_MAIN_DEFAULT macro and related constructs.

SOL_MAIN_DEFAULT takes two function arguments, setup and shutdown. The macro should be implemented by each port to the way the application expects to be started. The main thing that they all should do is call sol_init(), then the startup provided function, go into the main loop and once that ends, call the provided shutdown function, then sol_shutdown() to end.

Common

Main loop functions are implemented by a user facing set of functions, defined in src/lib/common/sol-mainloop.c, that internally will call the specific implementations. The functions each port needs to implement are declared in src/lib/common/sol-mainloop-impl.h, and should be defined by a port specific file in the form src/lib/common/sol-mainloop-impl-$PORTNAME.c, which should be conditionally added, along with any other necessary files, to the obj-core object in the Makefile under the same directory.

Timeouts and idlers have a common implementation in sol-mainloop-common.c that can be used by ports by including this file along with their implementation in the Makefile. The only requirement these have is that timeout makes use of sol_util_timespec_get_current(), to get the current monotonic time, and should be appropriately implemented by each port in src/shared/sol-util-impl-$PORTNAME.c.

Other primitives the main loop supports are file handlers and child process monitoring, currently only used by the Linux port.

Then comes the loop itself. At its most basic, there is a sol_mainloop_impl_run() function, implemented by sol-mainloop-common.c that can be used by ports if it's suitable, but this function still needs the port specific implementation of sol_mainloop_impl_iter(), which represents a single iteration of the loop. Here's where all the even processing needs to happen. Some ports may have specific needs, such as Contiki, that implements the loop itself in the SOL_MAIN_DEFAULT macro.

I/O

Access to peripherals, such as GPIO, PWM or I2C is done through the abstractions provided by the I/O library in src/lib/io.

The convention is to have a configuration struct passed to the open function of each abstraction, describing how a specific device should be opened. Unlike the main loop functions, there is no public API calling internal impl functions here, but the whole API is implemented by the port, as for example, sol-gpio-impl-linux.c. If there's a need to change the configuration of an already opened device, then it first needs to be closed and opened again with different parameters.

For devices where the OS may call user functions from an interrupt context it's up to the implementation to integrate them with the main loop. This needs to be done in a manner that's transparent for users of Soletta itself, so that the code remains portable without having to consider the implications of possibly calling other functions from such interrupt contexts.

For example, under RIOT, when the developer opens a GPIO as an input, it will pass in the configuration a callback function to be called when the value read by the device changes. The GPIO implementation will try to use hardware interrupts if they are enabled, and the OS will call the given internal function when an interrupt is triggered, but to avoid the function provided by the developer to be called from the interrupt context, this internal one will send a message to the main thread, that the RIOT implementation of the main loop will treat especially, calling back another internal GPIO function that ends up calling the one provided during sol_gpio_open(). This way, Soletta users are shielded from how an OS may work, providing a common and predictable implementation under all platforms.

Network

In the src/lib/comms directory can be found all the network related components, from interface management to specific protocols like CoAP and MQTT.

Flow

The Flow subsystem in Soletta builds upon all the others, with its core being fairly small non-dependant on much more than the main loop. Most of the code that composes this subsystem is in the node types. Still, there are some things that need consideration when this is ported to very small devices.

Though small, the flow system can be a bit complex, so call stacks may run deep. If the stack is too limited, the application may fail to function properly, or even start.

Dynamic memory is also heavily used, so in order for the flow to work, the OS and libc used need to have fully functional malloc() and free() implementations. One that let's malloc() work but never frees the memory will run out of available RAM very quick.

FBP applications

FBP applications are scripts written using FBP notation, that under Linux can be executed using sol-fbp-runner. For other cases, Soletta also has the sol-fbp-generator, that takes an FBP application and converts it into C code, that can then be compiled to run on other systems. Naturally, this means that either Soletta needs to be available on the host doing to cross compile, or at least it should have a static version of the generator built. The generator takes as input the FBP files that compose the application, plus the JSON descriptions of the node types that is generated during the build, and can be found under the build directory in soletta_sysroot/usr/share/soletta/flow/descriptions.

Example, the following FBP (located in src/samples/flow/basics/simple.fbp)

timer(timer:interval=1000) OUT -> IN toggle(boolean/toggle)
toggle OUT -> IN console_toggle(console)
toggle OUT -> IN not(boolean/not)
not OUT -> IN console_not(console)

Can be turned into a C program to be built for any system that has flow already working with

sol-fbp-generator -I $PATH_TO_SOLETTA/build/soletta_sysroot/usr/share/soletta/flow/descriptions simple.fbp simple.c

Then the generated simple.c would be the intended application's code to build for the target OS.