Q: Adding custom board variant files

[4project-co-il]

Hi,

I think I've read all the possible information on the internet, but still can't find the solution.
Most likely I'm missing something or I don't understand the concept completely.

I have a custom board I want to add to the PIO, so potential customers will be able to start a new project using that board right from the PIO menus, without the need to copy any files locally.
The board have the needed JSON and it's own vriant files, since the pin mapping is different from any other available board.
Everything is working fine locally with all needed files in boards and variants directories in .platformio folder.

In Arduino IDE I can add my JSON configuration file as an additional board manager URLs, which will tell the IDE where to take the framework package needed for my new board.

I don't see how I can do that with PIO.
I can request to add the configuration JSON here:
https://github.com/platformio/platform-atmelsam/tree/develop/boards

But I can't find any way to provide the needed variant files.

I did find that location:
https://github.com/platformio/platformio-pkg-framework-arduinosam/tree/master/variants

But I'm not sure it's maintained since I can see the "SparkFun_SAMD_Mini" variant there, but not other SparkFun's boards, like the "SparkFun_SAMD21_Dev" for example, which is used here:
https://github.com/platformio/platform-atmelsam/blob/develop/boards/sparkfun_samd21_dev_usb.json

Will appreciate an explanation what I'm missing in the process.

The board is based on SAMD21 MCU and is intented for unexperienced users, so I want to easy on the installation process and remove files copy&paste steps.

Thanks.

[valeros]

Hi @4project-co-il, this feature is not documented and not available for all dev platforms, but you can use the board_build.variants_dir option in your platformio.ini file to specify a path to a folder with your custom variant files. For example you can create a new folder my_variants in the root of your project and add the following line to your configuration:

board_build.variants_dir = my_variants

This way custom variants will be part of your project.

[4project-co-il]

Hi @valeros ,

Locally I have no problems with the paths.
I can change my platformio.ini any way I want.

When distributing the new board I'd like to have the same experience as other manufacturers have.
For example, if I look on the "SparkFun SAMD21 Dev Breakout", I can select it from the list of supported boards and start working with the board without copying anything manually or changing the platformio.ini.

The boards/sparkfun_samd21_dev_usb.json arrive from current repository:
https://github.com/platformio/platform-atmelsam/blob/develop/boards/sparkfun_samd21_dev_usb.json

While the variant files are not.

I can see that I have .platformio/packages/framework-arduino-samd-sparkfun directory installed on my computer, which have the variants directory inside.

If having a separate framework in order to have my custom board variants files is the only way, so how can I force PIO to install a separate framework for my board?

I don't see any path in SparkFun's board json file.
The main difference I think is that their file has "core": "sparkfun" parameter, while my board file have it as "core": "arduino".
I see now that SeeedStudio boards have it as "core": "seeed".
Is that the way?
What happens if I set it as "core": "custom"?
How do I point the PIO to use my framework from within the board files?

I still think I miss something in the process...

[valeros]

In a nutshell, all boards and frameworks (e.g. adafruit, sparkfun, seeed) in this repo have gone through the integration process. While there is some flexibility for adding new hardware, it can only be done via the platformio.ini file (as I mentioned above). From what I understand, you have your own hardware and framework and want to provide a similar experience to the officially supported boards. Unfortunately, that’s not currently possible. The only option I can think of is to fork and maintain your own copy of this dev-platform and have your clients install your fork manually.

[4project-co-il]

@valeros

Hmm...
Another thought direction:
Is it possible to specify a path to variants path to a github repo/directory, so PIO will pull all needed files when creating the project?
Similary to the libraries and other files it reloads based on platformio.ini content?

[valeros]

Is it possible to specify a path to variants path to a github repo/directory

Unfortunately, it's not possible. You can override the entire framework package (see platform_packages), but only in the platformio.ini file.

[4project-co-il]

Sounds like a locked garden for those who want to sell products, but not yet the size of Adafruit or SparkFun...

[valeros]

As I mentioned above, you can always fork the repo and make any changes according to your needs.

[4project-co-il]

Yes, but how do I deliver that to the platformio.ini of an unexperienced customer?

Can I point the board_build.variants_dir to a github location, so PIO will pull it's content to a cache before build?
Something similar to platform or lib_deps?

Or only the platform_packages can be pulled?

I'm ok with platformio.ini way, but what is missing in the chain is pushing those platformio.ini directives from the boards json configuration.
If I could somehow specify those parameters pointed to the github location in the board json file, that will appear in the platformio.ini when a user selects my board from the supported boards list, than the problem will be solved.

[valeros]

Can I point the board_build.variants_dir to a github location, so PIO will pull it's content to a cache before build?

Only platform_packages can override the packages (entire packages, not arbitrary parts)

If I could somehow specify those parameters pointed to the github location in the board json file

Unfortunately, it's not possible.

---

As a workaround, you can use an extra script. Here is an example how to dynamically patch a framework package. Obviously, it's a much more complicated workflow than a Project Wizard, but at least it's possible to add new hardware. Note that your custom board won't be added to this repo, so your board manifest should be placed in the boards folder in the root of your project.

[4project-co-il]

@valeros
One more thought direction...
Assuming the user used my board definition from the menus, as it was PRed to PIO system.
Plus, the user added my library to the project using the PIO system.
Now my library files are in project's .pio/libdeps/ directory...

Is there a way to point the variants_dir in the board's JSON to that .pio/libdeps/ directory?
Some directive like %LIB_PATH% or something?

I guess I can set it to relative path, similar to variants as I have now, since both are relative to the project directory.
But it doesn't sound right.

[valeros]

Assuming the user used my board definition from the menus, as it was PRed to PIO system.

Unfortunately, that's a problem in the first place. Your board can't be added if there is no proper variant for it. Mainly because it won't be usable without that additional library you described.

Plus, the user added my library to the project using the PIO system.

How is it done? Via CLI or project config?

Is there a way to point the variants_dir in the board's JSON to that .pio/libdeps/ directory?

Unfortunately, there is nothing like %LIB_PATH%, the variants_dir option can be set only via platformio.ini. However, any library can have an extra script which basically gives you the full control of the entire build process (see an example here). So you probably can come up with a very hacky solution that injects your custom variant files to the build environment. But it sounds quite unreliable and error prone, so I can't recommend it.

[4project-co-il]

@valeros

In my case the board require to use a specific library.
Others who might be interested to deliver the variants files might create a dummy library that will do only that.
For the very beginners, it's easier to explain how to add the needed library (via the PIO's Libraries -> Add to project option), instead of manual change in the platformio.ini.

There's a support for variants_dir in board.json. You helped me with the code addition in #238.
I use it in my development environment, but that will not a good user experience to copy directories with files.

I see there's $PROJECT_CORE_DIR directive, so I could try:
"variants_dir" : "$PROJECT_CORE_DIR/.pio/libdeps/<board_name>/<my_library>/variants"
in the board.json

[valeros]

In my case the board require to use a specific library.

As I already mentioned, that's a problem, so it's highly unlikely that a board with such workarounds will be added to this repository.

For the very beginners, it's easier to explain how to add the needed library (via the PIO's Libraries -> Add to project option), instead

So maybe it's also possible to explain them how to install a fork of this dev-platform with your variant files? The workflow is basically the same as for the library.

[4project-co-il]

As I already mentioned, that's a problem, so it's highly unlikely that a board with such workarounds will be added to this repository.

You mean a PR with a boar JSON that refer external directory will not be added to the repository?

So maybe it's also possible to explain them how to install a fork of this dev-platform with your variant files? The workflow is basically the same as for the library.

What do you mean by dev-platform? Like the atmelsam?
That's using an RPG to kill a fly...
Even

I understand the limits, especially when you repeat them all the time :)

I will try to emphasize that the real thing what is missing is the ability to add something to the project's platformio.ini based on the selected board and probably a library as well.

Selected board dictates the platform, board and framework.
Adding a library adds a lib_deps line.
And that's all the control we have.

That's not only the variants path.
A more generic solution that will allow adding directives to the platformio.ini will allow many problem solving.
For example: compilation flags, serial monitor parameters...