Skip to content

Commit

Permalink
Pub sub sample split (#296)
Browse files Browse the repository at this point in the history
Splits the PubSub sample down into smaller connect samples that show how just how to make a connection.

* Simplified command line parsing
* Fixed tests not working correctly after simplified command line parsing
* Added action support to command line parser
* Added additional command groups
* Split PubSub sample into smaller connect samples
* Fixed codebuild tests to work with sample split
* Adjusted code based on code review:
* Added PubSub sample back to codebuild tests
* Simplified connection samples by using utility functions in command_line_utils
* Removed PKCS11 PubSub sample (since it's replaced by PKCS11 connect sample)
* Added PKCS11 connection builder function to command_line_utils
* Converted Windows Cert PubSub to Windows Cert Connect
* Adjusted README to use consistent command format
* Fixed diff being included in fleet provision sample
  • Loading branch information
TwistedTwigleg authored Apr 1, 2022
1 parent d5480f1 commit e56d3e8
Show file tree
Hide file tree
Showing 17 changed files with 661 additions and 539 deletions.
Binary file added codebuild/.DS_Store
Binary file not shown.
17 changes: 17 additions & 0 deletions codebuild/samples/connect-linux.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
#!/bin/bash

set -e

env

pushd $CODEBUILD_SRC_DIR/samples/

ENDPOINT=$(aws secretsmanager get-secret-value --secret-id "unit-test/endpoint" --query "SecretString" | cut -f2 -d":" | sed -e 's/[\\\"\}]//g')

echo "Mqtt Direct test"
python3 basic_connect.py --endpoint $ENDPOINT --key /tmp/privatekey.pem --cert /tmp/certificate.pem

echo "Websocket test"
python3 websocket_connect.py --endpoint $ENDPOINT --signing_region us-east-1

popd
3 changes: 2 additions & 1 deletion codebuild/samples/linux-smoke-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,8 @@ phases:
commands:
- echo Build started on `date`
- $CODEBUILD_SRC_DIR/codebuild/samples/setup-linux.sh
- $CODEBUILD_SRC_DIR/codebuild/samples/connect-linux.sh
- $CODEBUILD_SRC_DIR/codebuild/samples/pubsub-linux.sh
post_build:
commands:
- echo Build completed on `date`
- echo Build completed on `date`
5 changes: 1 addition & 4 deletions codebuild/samples/pubsub-linux.sh
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,7 @@ pushd $CODEBUILD_SRC_DIR/samples/

ENDPOINT=$(aws secretsmanager get-secret-value --secret-id "unit-test/endpoint" --query "SecretString" | cut -f2 -d":" | sed -e 's/[\\\"\}]//g')

echo "Mqtt Direct test"
echo "PubSub test"
python3 pubsub.py --endpoint $ENDPOINT --key /tmp/privatekey.pem --cert /tmp/certificate.pem

echo "Websocket test"
python3 pubsub.py --endpoint $ENDPOINT --use_websocket --signing_region us-east-1

popd
192 changes: 165 additions & 27 deletions samples/README.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,25 @@
# Sample apps for the AWS IoT Device SDK v2 for Python

* [PubSub](#pubsub)
* [PKCS#11 PubSub](#pkcs11-pubsub)
* [Windows Certificate PubSub](#windows-certificate-pubsub)
* [Basic Connect](#basic-connect)
* [Websocket Connect](#websocket-connect)
* [PKCS#11 Connect](#pkcs11-connect)
* [Windows Certificate Connect](#windows-certificate-connect)
* [Shadow](#shadow)
* [Jobs](#jobs)
* [Fleet Provisioning](#fleet-provisioning)
* [Greengrass Discovery](#greengrass-discovery)

## Build instructions

First, install the aws-iot-devices-sdk-python-v2 with following the instructions from [Installation](../README.md#Installation).

Then change into the samples directory to run the Python commands to execute the samples. You can view the commands of a sample like this:

``` sh
python3 pubsub.py --help
```

## PubSub

This sample uses the
Expand All @@ -21,11 +33,6 @@ Status updates are continually printed to the console.

Source: `samples/pubsub.py`

Run the sample like this:
``` sh
python3 pubsub.py --endpoint <endpoint> --ca_file <file> --cert <file> --key <file>
```

Your Thing's
[Policy](https://docs.aws.amazon.com/iot/latest/developerguide/iot-policies.html)
must provide privileges for this sample to connect, subscribe, publish,
Expand Down Expand Up @@ -70,14 +77,120 @@ and receive.
</pre>
</details>

## PKCS#11 PubSub
Run the sample like this:
``` sh
python3 pubsub.py --endpoint <endpoint> --ca_file <file> --cert <file> --key <file>
```

## Basic Connect

This sample makes an MQTT connection using a certificate and key file. On startup, the device connects to the server using the certificate and key files, and then disconnects.
This sample is for reference on connecting via certificate and key files.

This sample is similar to the basic [PubSub](#pubsub),
Source: `samples/basic_connect.py`

Your Thing's
[Policy](https://docs.aws.amazon.com/iot/latest/developerguide/iot-policies.html)
must provide privileges for this sample to connect, subscribe, publish,
and receive.

<details>
<summary>(see sample policy)</summary>
<pre>
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"iot:Connect"
],
"Resource": [
"arn:aws:iot:<b>region</b>:<b>account</b>:client/test-*"
]
}
]
}
</pre>
</details>

Run the sample like this:
``` sh
python3 basic_connect.py --endpoint <endpoint> --ca_file <file> --cert <file> --key <file>
```

## Websocket Connect

This sample makes an MQTT connection via websockets and then disconnects. On startup, the device connects to the server via websockets and then disconnects.
This sample is for reference on connecting via websockets.

Source: `samples/websocket_connect.py`

Your Thing's
[Policy](https://docs.aws.amazon.com/iot/latest/developerguide/iot-policies.html)
must provide privileges for this sample to connect, subscribe, publish,
and receive.

<details>
<summary>(see sample policy)</summary>
<pre>
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"iot:Connect"
],
"Resource": [
"arn:aws:iot:<b>region</b>:<b>account</b>:client/test-*"
]
}
]
}
</pre>
</details>

Run the sample like this:
``` sh
python3 websocket_connect.py --endpoint <endpoint> --ca_file <file> --signing_region <signing region>
```

Note that using Websockets will attempt to fetch the AWS credentials from your enviornment variables or local files. See the [authorizing direct AWS](https://docs.aws.amazon.com/iot/latest/developerguide/authorizing-direct-aws.html) page for documentation on how to get the AWS credentials, which then you can set to the `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS`, and `AWS_SESSION_TOKEN` environment variables.

## PKCS#11 Connect

This sample is similar to the [Basic Connect](#basic-connect),
but the private key for mutual TLS is stored on a PKCS#11 compatible smart card or Hardware Security Module (HSM)

WARNING: Unix only. Currently, TLS integration with PKCS#11 is only available on Unix devices.

source: `samples/pkcs11_pubsub.py`
source: `samples/pkcs11_connect.py`

Your Thing's
[Policy](https://docs.aws.amazon.com/iot/latest/developerguide/iot-policies.html)
must provide privileges for this sample to connect, subscribe, publish,
and receive.

<details>
<summary>(see sample policy)</summary>
<pre>
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"iot:Connect"
],
"Resource": [
"arn:aws:iot:<b>region</b>:<b>account</b>:client/test-*"
]
}
]
}
</pre>
</details>

To run this sample using [SoftHSM2](https://www.opendnssec.org/softhsm/) as the PKCS#11 device:

Expand Down Expand Up @@ -120,14 +233,14 @@ To run this sample using [SoftHSM2](https://www.opendnssec.org/softhsm/) as the
5) Now you can run the sample:
```sh
python3 pkcs11_pubsub.py --endpoint <xxxx-ats.iot.xxxx.amazonaws.com> --ca_file <AmazonRootCA1.pem> --cert <certificate.pem.crt> --pkcs11_lib <libsofthsm2.so> --pin <user-pin> --token_label <token-label> --key_label <key-label>
python3 pkcs11_connect.py --endpoint <endpoint> --ca_file <path to root CA> --cert <path to certificate> --pkcs11_lib <path to PKCS11 lib> --pin <user-pin> --token_label <token-label> --key_label <key-label>
```
## Windows Certificate PubSub
## Windows Certificate Connect
WARNING: Windows only
This sample is similar to the basic [PubSub](#pubsub),
This sample is similar to the basic [Connect](#basic-connect),
but your certificate and private key are in a
[Windows certificate store](https://docs.microsoft.com/en-us/windows-hardware/drivers/install/certificate-stores),
rather than simply being files on disk.
Expand All @@ -141,7 +254,32 @@ If your certificate and private key are in a
[TPM](https://docs.microsoft.com/en-us/windows/security/information-protection/tpm/trusted-platform-module-overview),,
you would use them by passing their certificate store path.

source: `samples/windows_cert_pubsub.py`
source: `samples/windows_cert_connect.py`

Your Thing's
[Policy](https://docs.aws.amazon.com/iot/latest/developerguide/iot-policies.html)
must provide privileges for this sample to connect, subscribe, publish,
and receive.
<details>
<summary>(see sample policy)</summary>
<pre>
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"iot:Connect"
],
"Resource": [
"arn:aws:iot:<b>region</b>:<b>account</b>:client/test-*"
]
}
]
}
</pre>
</details>
To run this sample with a basic certificate from AWS IoT Core:
Expand Down Expand Up @@ -183,7 +321,7 @@ To run this sample with a basic certificate from AWS IoT Core:
4) Now you can run the sample:
```sh
python3 windows_cert_pubsub.py --endpoint xxxx-ats.iot.xxxx.amazonaws.com --root-ca AmazonRootCA.pem --cert CurrentUser\My\A11F8A9B5DF5B98BA3508FBCA575D09570E0D2C6
python3 windows_cert_connect.py --endpoint <endpoint> --ca_file <path to root CA> --cert <path to certificate>
```
## Shadow
Expand Down Expand Up @@ -488,11 +626,11 @@ and `--key` appropriately:
``` sh
python3 fleetprovisioning.py \
--endpoint [your endpoint]-ats.iot.[region].amazonaws.com \
--ca_file [pathToRootCA] \
--cert /tmp/provision.cert.pem \
--key /tmp/provision.private.key \
--template_name [TemplateName] \
--endpoint <endpoint> \
--ca_file <path to root CA> \
--cert <path to certificate> \
--key <path to private key> \
--template_name <template name> \
--template_parameters "{\"SerialNumber\":\"1\",\"DeviceLocation\":\"Seattle\"}"
```
Expand Down Expand Up @@ -528,13 +666,13 @@ using a permanent certificate set, replace the paths specified in the `--cert` a

``` sh
python3 fleetprovisioning.py \
--endpoint [your endpoint]-ats.iot.[region].amazonaws.com \
--ca_file [pathToRootCA] \
--cert /tmp/provision.cert.pem \
--key /tmp/provision.private.key \
--template_name [TemplateName] \
--endpoint <endpoint> \
--ca_file <path to root CA> \
--cert <path to certificate> \
--key <path to key> \
--template_name <template name> \
--template_parameters "{\"SerialNumber\":\"1\",\"DeviceLocation\":\"Seattle\"}" \
--csr /tmp/deviceCert.csr
--csr <path to csr file>
```

## Greengrass Discovery
Expand Down
55 changes: 55 additions & 0 deletions samples/basic_connect.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: Apache-2.0.

from uuid import uuid4

# This sample shows how to create a MQTT connection using a certificate file and key file.
# This sample is intended to be used as a reference for making MQTT connections.

# Parse arguments
import command_line_utils
cmdUtils = command_line_utils.CommandLineUtils("Basic Connect - Make a MQTT connection.")
cmdUtils.add_common_mqtt_commands()
cmdUtils.add_common_proxy_commands()
cmdUtils.add_common_logging_commands()
cmdUtils.register_command("key", "<path>", "Path to your key in PEM format.", True, str)
cmdUtils.register_command("cert", "<path>", "Path to your client certificate in PEM format.", True, str)
cmdUtils.register_command("port", "<int>",
"Connection port for direct connection. " +
"AWS IoT supports 433 and 8883 (optional, default=8883).",
False, int)
cmdUtils.register_command("client_id", "<str>",
"Client ID to use for MQTT connection (optional, default='test-*').",
default="test-" + str(uuid4()))
# Needs to be called so the command utils parse the commands
cmdUtils.get_args()

# Callback when connection is accidentally lost.
def on_connection_interrupted(connection, error, **kwargs):
print("Connection interrupted. error: {}".format(error))

# Callback when an interrupted connection is re-established.
def on_connection_resumed(connection, return_code, session_present, **kwargs):
print("Connection resumed. return_code: {} session_present: {}".format(return_code, session_present))


if __name__ == '__main__':
# Create a connection using a certificate and key.
# Note: The data for the connection is gotten from cmdUtils.
# (see build_direct_mqtt_connection for implementation)
mqtt_connection = cmdUtils.build_direct_mqtt_connection(on_connection_interrupted, on_connection_resumed)

print("Connecting to {} with client ID '{}'...".format(
cmdUtils.get_command(cmdUtils.m_cmd_endpoint), cmdUtils.get_command("client_id")))

connect_future = mqtt_connection.connect()

# Future.result() waits until a result is available
connect_future.result()
print("Connected!")

# Disconnect
print("Disconnecting...")
disconnect_future = mqtt_connection.disconnect()
disconnect_future.result()
print("Disconnected!")
Loading

0 comments on commit e56d3e8

Please sign in to comment.