top-level index for firmware documentation

m-mcgowan · m-mcgowan · commit c211bb205b35 · 2015-07-18T23:04:43.000+02:00
diff --git a/README.md b/README.md
@@ -1,147 +1,64 @@
 <!---
 -->
 
-# Particle Firmware for the Core and Photon 
+# Particle Firmware for the Core and Photon
 
 This is the main source code repository of the Particle firmware libraries.
 
-1. [Download and Install Dependencies](docs/dependencies.md#1-download-and-install-dependencies)
-2. [Download and Build Repositories](#2-download-and-build-repositories)
-3. [Edit and Rebuild](#3-edit-and-rebuild)
-4. [Flash It!](#4-flash-it)
- 
-## 2. Download and Build Repositories
+# Getting Started
 
-The entire Particle firmware is contained in this repository.
+To get started building firmware locally, see [Getting Started](docs/gettingstarted.md.)
 
-There are two ways to download
-- through the git command line interface
-- download the zipped file from the github website
 
-We recommend the first approach, since it makes keeping up to date with new releases
-much simpler.
+# Resources
 
-**Method 1:** Through the git command line interface.
+- [Latest Release - 0.4.3](releases/tag/0.4.3-rc2)
+- [Changelog](CHANGELOG.md)
 
-Open up a terminal window, navigate to your destination directory and type the following commands:
 
-(Make sure you have git installed on your machine!)
+## Build System
 
-* `git clone https://github.com/spark/firmware.git`  
+- [Requirements/Dependencies](docs/dependencies.md)
 
-**Method 2:** Download the zipped files directly from the GitHub website
+## Application Firmware Development
 
-* [firmware](https://github.com/spark/firmware/archive/master.zip)
+- [Debugging support](debugging.md)
+- [make command syntax](docs/build.md)
+- [Firmware API](http://docs.particle.io/photon/firmware/)
 
-#### How do we *build* these repositories?
+## System Firmware Development
 
-Make sure you have downloaded and installed all the required dependencies as mentioned [previously.](docs/dependencies.md#1-download-and-install-dependencies). 
-Note, if you've downloaded or cloned these previously, you'll want to `git pull` or redownload all of them before proceeding.
+- [System Flags](system/system-flags.md)
+- [Continuous Integration](ci/README.md)
+- [Module Descriptor linking and retrieval](dynalib/src/readme.md)
+- [Photon SoftAP Protocol](hal/src/photon/soft-ap.md)
+- [WiFi Tester Firmware](user/applications/wifitester/readme.md)
+- [Testing](user/tests/readme.md)
+- [API Changes Checklist](wiki/Firmware-API-Changes-Checklist)
+- [Firmware Release Checklist](wiki/Firmware-Release-Checklist)
+- [System describe message](https://github.com/spark/firmware/wiki/Module-descriptor-format)
+- [build test suite](build/test/readme.md)
 
-Open up a terminal window, navigate to the `main` folder under firmware
-(i.e. `cd firmware/main`) and type:
+### Modules
 
-    make
+- Bootloader [overview](bootloader/README.md) and [internals](bootloader/documentation.md)
+- [Cloud Communications](communication/README.md)
 
-This will build your main application (`firmware/user/src/application.cpp`) and required dependencies.
+### Platforms
 
-*For example:* `D:\Spark\firmware\main [master]> make`
+- [Virtual Device](hal/src/gcc/readme.md)
+- [Starting a New Platform Hardware Abstraction Layer](hal/src/newhal/readme.md)
+- [Installing the USB Driver on Windows](misc/driver/windows/readme.md)
 
-You won't see any compiler output for about 30 seconds or so since verbose output is disabled by default and can be enabled with the `v=1` switch.
 
-*For example:* `D:\Spark\firmware\main [master]> make v=1`
 
-The [makefile documentation](docs/build.md) describes the build options supported and how to targeting platforms other than the Core.
-
-##### Common Errors
-
-* `arm-none-eabi-gcc` and other required gcc/arm binaries not in the PATH.
-  Solution: Add the /bin folder to your $PATH (i.e. `export PATH="$PATH:<SOME_GCC_ARM_DIR>/bin`).
-  Google "Add binary to PATH" for more details.
-
-* You get `make: *** No targets specified and no makefile found.  Stop.`
-  Solution: `cd firmware/main`
-
-Please issue a pull request if you come across similar issues/fixes that trip you up.
-
-### Navigating the code base
-
-All of the top-level directories are sub divided into functional folders that are
-the various libraries that make up the firmware.
-
-- platform: bare-metal services, the lowest layer in the system
-- bootloader: the bootloader, with sources for each platform
-- hal: the Hardware Abstraction Layer interface and an implementation for each supported platform
-- services: platform neutral services and macros (LED control, debug macros, static assertions)
-- communication: implements the protocol between the device and the cloud
-- dynalib: framework for producing dynamically linked libraries
-- system: the system firmware (Networking, firmware updates.)
-- wiring: the Wiring API
-- user: contains the default application code (Tinker) and your own applications
-- main: top-level project to build the firmware for a device
-- modules: dynamically linked modules for the Photon/P0/P1
-
-Within each library, the structure is 
-
-1. `/src` holds all the source code files
-2. `/inc` holds all the header files
-
-The compiled `.bin` and `.hex` files are output to a subdirectory of `build/target/`.
-The exact location is given in the final compiler output. (It depends upon the platform and on what is being built.)
-
-## 3. Edit and Rebuild
-
-Now that you have your hands on the firmware, its time to start hacking!
-
-### What to edit and what not to edit?
-
-The main user code sits in the application.cpp file under firmware/user/src/ folder. Unless you know what you are doing, refrain yourself from making changes to any other files.
-
-After you are done editing the files, you can rebuild the repository by running the `make` command in the `firmware/main/` directory. 
-If you have made changes to any of the other directories, make automatically determines which files need to be rebuilt and builds them for you.
-
-## 4. Flash It!
-
-Its now time to transfer your code to your Particle device! You can always do this using the Over the Air update feature or, if you like wires, do it over the USB.
-
-*Make sure you have the `dfu-util` command installed and available through the command line*
-
-#### Steps:
-1. Put your device into the DFU mode by holding down the mode/setup button on the device and then tapping on the RESET button once. Release the MODE/setup button after you start to see the RGB LED flashing in yellow. 
-It's easy to get this one wrong: Make sure you don't let go of the MODE/SETUP button until you see flashing yellow, about 3 seconds after you release the RESET button. 
-A flash of white then flashing green can happen when you get this wrong. You want flashing yellow.
-
-2. Open up a terminal window on your computer and type this command to find out if the device indeed being detected correctly. 
-
-   `dfu-util -l`   
-   you should get something similar to this in return:
-   ```
-   Found DFU: [1d50:607f] devnum=0, cfg=1, intf=0, alt=0, name="@Internal Flash  /0x08000000/20*001Ka,108*001Kg" 
-   Found DFU: [1d50:607f] devnum=0, cfg=1, intf=0, alt=1, name="@SPI Flash : SST25x/0x00000000/512*04Kg"
-   ```
-
-   (Windows users will need to use the Zatig utility to replace the USB driver as described earlier)
-
-3. Now, from the `main/` folder in your firmware repository and use the following command to transfer the *.bin* file into the device.
-   ```
-   make program-dfu
-   ```
-
-Upon successful transfer, the device will automatically reset and start the running the program.
-
-##### Common Errors
-* As of 12/4/13, you will likely see `Error during download get_status` as the last line from 
-the `dfu-util` command. You can ignore this message for now.  We're not sure what this error is all about.
-
-* If you are having trouble with dfu-util, (like invalid dfuse address), try a newer version of dfu-util. v0.7 works well.
-
-**Still having troubles?** Checkout our [resources page](https://www.spark.io/resources), hit us up on IRC, etc.
 
 ### CREDITS AND ATTRIBUTIONS
 
-The Particle application team: Zachary Crockett, Satish Nair, Zach Supalla, David Middlecamp and Mohit Bhoite.
+The firmware uses the GNU GCC toolchain for ARM Cortex-M processors, ARM's CMSIS libraries, STM32 standard peripheral libraries and Arduino's implementation of Wiring.
 
-The firmware uses the GNU GCC toolchain for ARM Cortex-M processors, ARM's CMSIS libraries, TI's CC3000 host driver libraries, STM32 standard peripheral libraries and Arduino's implementation of Wiring.
+On the Core: TI's CC3000 host driver libraries,
+On the photon: Broadcom's WICED WiFi SDK.
 
 ### LICENSE
 
diff --git a/bootloader/README.md b/bootloader/README.md
@@ -5,18 +5,20 @@
   * Really make sure you have a working programmer shield and an ST-link programmer.
   * Is this a brand new core from scratch (that you made? nice!) goto 1, or is it this a core that came from Spark - goto "unlocking your bootloader"
 
+Please note this documentation is for the Core only. The photon automaatically updates the bootloader as needed.
+
 ### Unlocking your bootloader
 
 Welcome! The bootloader on the stm32 is protected by a "write protect" flag that helps prevent accidental "bricking" of a core for those without a programmer shield / st-link programmer. But that's not you, you want to fully erase your core, or re-write your bootloader, and you're not scared of "bricking" anything.
 
  * Grab the "unlocker" from here: (https://github.com/spark/firmware/tree/master/bootloader/tools (use "raw")
  * dfu that with a:  `dfu-util -d 1d50:607f -a 0 -s 0x08005000:leave -D unlocker-firmware.bin`
- * Your core should restart, flash some lights, and end with a 'green' flash before wiping saved wifi profiles and resetting   
+ * Your core should restart, flash some lights, and end with a 'green' flash before wiping saved wifi profiles and resetting
  * Your bootloader is unlocked!
 
 **1. Make sure you have st-flash installed, or have a copy of the ST-Link Utility software.**
 
- * You can find the software on their site if you're using windows: http://www.st.com/web/catalog/tools/FM146/CL1984/SC724/SS1677/PF251168 
+ * You can find the software on their site if you're using windows: http://www.st.com/web/catalog/tools/FM146/CL1984/SC724/SS1677/PF251168
 
  * If you're on mac / linux grab "st-flash" you can grab and build a tool yourself:
     ```
@@ -27,7 +29,7 @@ Welcome! The bootloader on the stm32 is protected by a "write protect" flag that
     make
     make install
     ```
-    
+
 **2. Grab a copy of the latest bootloader from here:**
 
   https://github.com/spark/firmware/blob/master/bootloader/build/bootloader.bin?raw=true5.) With your core in your programmer shield / st-link connected, lets re-flash some stuff!
@@ -54,35 +56,35 @@ Welcome! The bootloader on the stm32 is protected by a "write protect" flag that
 ### It's time to write some stuff to external flash!
 
  **1. Lets make some keys (optional step if your keys have been fine):**
- 
+
 ```
     openssl genrsa -out core.pem 1024
     openssl rsa -in core.pem -pubout -out core_public.pem
     openssl rsa -in core.pem -outform DER -out core_private.der
 ```
-    
- **2. Flash your new private key:** 
- 
+
+ **2. Flash your new private key:**
+
     `dfu-util -d 1d50:607f -a 1 -s 0x00002000 -v -D core_private.der`
-    
- **3. Grab the server public key, and flash it:**  
+
+ **3. Grab the server public key, and flash it:**
     https://s3.amazonaws.com/spark-website/cloud_public.der
-    
+
     `dfu-util -d 1d50:607f -a 1 -s 0x00001000 -v -D cloud_public.der`
 
- **4. Grab the latest firmware and flash it**  
- 
+ **4. Grab the latest firmware and flash it**
+
     https://github.com/spark/firmware/blob/master/main/build/core-firmware.bin?raw=true
-    
+
 ```
-    # to the factory reset spot 
+    # to the factory reset spot
     dfu-util -a 1 -s 0x00020000 -v -D core-firmware.bin
 
     ## to the main firmware spot
     dfu-util -a 0 -s 0x08005000:leave -D core-firmware.bin
 ```
-    
+
  **5. If you made new keys, be sure to send someone at Spark your new public key, along with your core id, otherwise your core won't connect to the cloud again!**
- 
- *(Thanks to David for putting this together)*  
+
+ *(Thanks to David for putting this together)*
  Ref: https://community.spark.io/t/st-link-programmer-shield-only-so-youve-decided-you-want-to-update-your-bootloader-and-everything/2355
diff --git a/build/test/readme.md b/build/test/readme.md
@@ -1,15 +1,16 @@
 # Integration Tests for the makefile build
 
-# Setup
+## Setup
 
 Install BATS.
 
-# Running
+## Running
 
 ```
 cd build/test
 bats build.bats
 ```
 
+That's it. :-)
 
 
diff --git a/dynalib/src/readme.md b/dynalib/src/readme.md
@@ -1,21 +1,21 @@
 # Module Info
 
-The `module_info.h` file describes the structure of the module into that is 
-inserted into modules supporting self-describing placement and CRC checks.
+The `module_info.h` file describes the structure of the block of metadata (module info) that is
+inserted into modules supporting self-describing memory location and CRC checks.
 
-The structure is initialized via `module_info.c` and link at a specific 
-location by the linker script. The linker script provides the locations for the very
-start of the module and the very end of the module.
+The structure is initialized by the code in `module_info.c` and linked at a specific
+location by the linker script. The linker script also provides the addresses of the
+start of the module and the end of the module.
 
 ## Locating the module info struct
 
 The module info struct is located like this:
 
-- examine the first 4 address at the start of the module. 
+- examine the first 4 address at the start of the module.
 
-- If this is a module with an interrupt vector table (VTOR) the first address 
+- If this is a module with an interrupt vector table (VTOR) the first address
 will contain the top of the stack in ram (0x20xxxxxx). In this case the info table
-will be placed immediately after the VTOR table. For STM32F205, the VTOR table 
+will be placed immediately after the VTOR table. For STM32F205, the VTOR table
 comprises 97 interrupt vectors, giving a total size of 388 (0x184) bytes.
 
 - The other case is a module without the VTOR, where the module info appears at
@@ -35,3 +35,85 @@ can be cast and assigned to a `const module_info_t*`
     const module_info_t* module_info = (const module_info*)address;
 ```
 
+The binaries consumed in the Particle ecosystem should include "module info" descriptors that appear at fixed locations in the binary. There are 2 module info structures:
+
+- start info: appears at the start of the binary, this defines the platform, version, module type, module dependencies
+- end info: appears at the end of the binary. Defines product ID and version, as set in user code and provides storage for the SHA256 and CRC32 hashes that are added after the binary is built.
+
+
+# Compiling Module Info
+
+The module info structures are defined in `dynalib/inc/module_info.h`.  The structures are instantiated in `dynalib/inc/module_info.inc` which defines the default values for all platforms.
+
+The `main` project compiles the module info structure instances into a `module_info.o` object file - `main/src/module_info.c` simply includes `module_info.inc` from dynalib so the instance declarations become static data.
+
+## Linking Module Info
+
+Your project's linker script describes how the various object files are placed into the final firmware image. The module infos need to be placed at specific locations, and so we must use linker scripting to instruct the linker to place these at the desired locations.
+
+We provide some linker scripts to make this process simple (and avoid copy/pasting code.) All the linker scripts are stored in `build/arm/linker`. If not already done, you should add this directory to the linker search path with
+
+```
+LDFLAGS += -L$(COMMON_BUILD)/arm/linker
+```
+
+Add this alongside the other linker flags defines in your HAL `include.mk` file.
+
+
+All the linker script changes add data to the program memory region - this is where the main executable code is stored. To reuse our linker scripts that inject the module info, this section should be named `APP_FLASH`. For example,
+
+```
+.somesection :
+{
+  // .. declarations
+} > APP_FLASH
+
+.text :
+{
+  // .. more declarations
+} > APP_FLASH
+```
+
+
+### Module Start Symbol
+
+A symbol for the module start is used to compute various offsets.
+
+Add this include before any other APP_FLASH section.
+```
+INCLUDE 'module_start.ld'
+```
+
+In the example, this would come before `.somesection`.
+
+### Module Info Start
+
+After any mandatory sections in program memory, such as ISR vector tables, add
+
+```
+INCLUDE 'module_info.ld'
+```
+
+In the example, this would come after `.somesection` if that section must remain at the start of the binary image. If `.somesection` can appear anywhere, the include would be placed before it so that the module info is placed at the very beginning of the module.
+
+
+### Module Info End
+
+After all other `APP_FLASH` sections, add
+
+```
+INCLUDE 'module_end.ld'
+```
+
+In the example, this would be placed after the `.text` section.
+
+
+
+## Troubleshooting
+
+Without the module info end structures in place, building `main` will fail since the module info end data isn't present. This includes default SHA256 and CRC32 values that are built into the binary, and the buidl will fail when those do not match the values expected.  This is a sanity check in the build that it's overwriting the correct data when writing the SHA256 and CRC32 hashes to offsets from the end of the file.
+
+
+
+
+
