Skip to content
This repository has been archived by the owner on Jun 27, 2019. It is now read-only.

CI and QA

Jump to bottom
Bruno Dilly edited this page Jun 16, 2016 · 5 revisions

Continuous Integration and Quality Assurance are important development practices and we take them very seriously on Soletta™ Framework's development.

To help us to keep code quality we're using a few processes, tools and web services.

Code review

Every single commit should be reviewed before integration to Soletta project's master branch.

It's done on github, following these instructions.

Semaphore

For each pull request a semaphoreci build will be done, checking if Soletta framework is building fine for 3 different build configurations:

  • alldefconfig
  • modconfig
  • allyesconfig

For each of those builds many automated tests will be done:

  • check - runs numerous unit tests to make sure Soletta framework's internals are in good shape (tests at src/tests/)
  • check-fbp - guarantees that all flow tests are correctly parsed by sol-fbp-runner and behave as expected (tests at src/tests-fbp/)
  • check-fbp-bin - binaries are generated for all flow tests and executed to guarantee that code generated with sol-fbp-generator works as expected.
  • samples - tests if all samples build correctly (samples at src/samples/)
  • check-stub - checks if our tool that creates stubs for node types wasn't broken

Some extra checks are done on submitted code:

  • check-style - makes sure new code is following the project's coding style
  • doxygen - checks if no error is raised when creating C API documentation with doxygen (that should follow some standards)
  • doxy-coverage - checks if no exposed C API is lacking documentation

Also every time a PR lands on master branch, it's built and all the previous steps are done (for 32 and 64 bits architectures).

If any issue is detected on PRs, it must be fixed before integration to 'master' branch.

If any issue is found on 'master' branch, what should be a rare situation given all PRs are tested first, Soletta project developers should fix it immediately.

Coverity

Even with many flags to show errors in the compiler, running hundreds of tests and doing manual code review, some issues may pass and end up on 'master' branch.

Once a day 'master' branch is scanned by Coverity, a static analysis tool. When issues are found this service notifies Soletta project developers, that promptly fix them.

If you need access to it, please let us know.

Bug reporting

We ask to any QA person, developer or user that finds any issue or behavior that doesn't seem correct to fill an issue on github.

There are some steps that may be taken to help Soletta project's team to solve them as fast as possible.

Fuzzing

Some engineers run fuzz tests to guarantee Soletta framework's inputs are well validated.

We're working on making them more open to public, so stay tuned for more about this subject in a near future.

Clone this wiki locally