Skip to content

Commit

Permalink
Support PKCS#11 for mutual TLS on Unix platforms (#259)
Browse files Browse the repository at this point in the history
- Update to latest `aws-crt-python`, which exposes PKCS#11 functionality (see awslabs/aws-crt-python#323)
- Add `pkcs11_pubsub.py` sample, demonstrating an MQTT connection where the private key is stored in PKCS#11 token.
  - Add docs for sample
  • Loading branch information
graebm authored Jan 8, 2022
1 parent 39606e2 commit 03eb38e
Show file tree
Hide file tree
Showing 4 changed files with 281 additions and 17 deletions.
55 changes: 55 additions & 0 deletions awsiot/mqtt_connection_builder.py
Original file line number Diff line number Diff line change
Expand Up @@ -249,6 +249,61 @@ def mtls_from_bytes(cert_bytes, pri_key_bytes, **kwargs) -> awscrt.mqtt.Connecti
return _builder(tls_ctx_options, **kwargs)


def mtls_with_pkcs11(*,
pkcs11_lib: awscrt.io.Pkcs11Lib,
user_pin: str,
slot_id: int = None,
token_label: str = None,
private_key_label: str = None,
cert_filepath: str = None,
cert_bytes=None,
**kwargs) -> awscrt.mqtt.Connection:
"""
This builder creates an :class:`awscrt.mqtt.Connection`, configured for an mTLS MQTT connection to AWS IoT,
using a PKCS#11 library for private key operations.
This function takes all :mod:`common arguments<awsiot.mqtt_connection_builder>`
described at the top of this doc, as well as...
Keyword Args:
pkcs11_lib (awscrt.io.Pkcs11Lib): Use this PKCS#11 library
user_pin (Optional[str]): User PIN, for logging into the PKCS#11 token.
Pass `None` to log into a token with a "protected authentication path".
slot_id (Optional[int]): ID of slot containing PKCS#11 token.
If not specified, the token will be chosen based on other criteria (such as token label).
token_label (Optional[str]): Label of the PKCS#11 token to use.
If not specified, the token will be chosen based on other criteria (such as slot ID).
private_key_label (Optional[str]): Label of private key object on PKCS#11 token.
If not specified, the key will be chosen based on other criteria
(such as being the only available private key on the token).
cert_filepath (Optional[str]): Use this X.509 certificate (file on disk).
The certificate must be PEM-formatted. The certificate may be
specified by other means instead (ex: `cert_file_contents`)
cert_bytes (Optional[bytes-like object]):
Use this X.509 certificate (contents in memory).
The certificate must be PEM-formatted. The certificate may be
specified by other means instead (ex: `cert_file_path`)
"""
_check_required_kwargs(**kwargs)

tls_ctx_options = awscrt.io.TlsContextOptions.create_client_with_mtls_pkcs11(
pkcs11_lib=pkcs11_lib,
user_pin=user_pin,
slot_id=slot_id,
token_label=token_label,
private_key_label=private_key_label,
cert_file_path=cert_filepath,
cert_file_contents=cert_bytes)

return _builder(tls_ctx_options, **kwargs)


def websockets_with_default_aws_signing(
region,
credentials_provider,
Expand Down
87 changes: 71 additions & 16 deletions samples/README.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,13 @@
# Sample apps for the AWS IoT Device SDK v2 for Python

* [pubsub](#pubsub)
* [shadow](#shadow)
* [fleet provisioning](#fleet-provisioning)
* [PubSub](#pubsub)
* [PKCS#11 PubSub](#pkcs11-pubsub)
* [Shadow](#shadow)
* [Jobs](#jobs)
* [Fleet Provisioning](#fleet-provisioning)
* [Greengrass Discovery](#greengrass-discovery)

## Pubsub
## PubSub

This sample uses the
[Message Broker](https://docs.aws.amazon.com/iot/latest/developerguide/iot-message-broker.html)
Expand Down Expand Up @@ -67,6 +69,59 @@ and receive.
</pre>
</details>

## PKCS#11 PubSub

This sample is similar to the [Pub-Sub](#pubsub),
but the private key for mutual TLS is stored on a PKCS#11 compatible smart card or Hardware Security Module (HSM)

WARNING: Unix only. Currently, TLS integration with PKCS#11 is only available on Unix devices.

source: `samples/pkcs11_pubsub.py`

To run this sample using [SoftHSM2](https://www.opendnssec.org/softhsm/) as the PKCS#11 device:

1) Create an IoT Thing with a certificate and key if you haven't already.

2) Convert the private key into PKCS#8 format
```sh
openssl pkcs8 -topk8 -in <private.pem.key> -out <private.p8.key> -nocrypt
```

3) Install [SoftHSM2](https://www.opendnssec.org/softhsm/):
```sh
sudo apt install softhsm
```

Check that it's working:
```sh
softhsm2-util --show-slots
```
If this spits out an error message, create a config file:
* Default location: `~/.config/softhsm2/softhsm2.conf`
* This file must specify token dir, default value is:
```
directories.tokendir = /usr/local/var/lib/softhsm/tokens/
```
4) Create token and import private key.
You can use any values for the labels, PINs, etc
```sh
softhsm2-util --init-token --free --label <token-label> --pin <user-pin> --so-pin <so-pin>
```
Note which slot the token ended up in
```sh
softhsm2-util --import <private.p8.key> --slot <slot-with-token> --label <key-label> --id <hex-chars> --pin <user-pin>
```
5) Now you can run the sample:
```sh
python3 pkcs11_pubsub.py --endpoint <xxxx-ats.iot.xxxx.amazonaws.com> --root-ca <AmazonRootCA1.pem> --cert <certificate.pem.crt> --pkcs11-lib <libsofthsm2.so> --pin <user-pin> --token-label <token-label> --key-label <key-label>
## Shadow
This sample uses the AWS IoT
Expand Down Expand Up @@ -306,14 +361,14 @@ and receive.

### Fleet Provisioning Detailed Instructions

#### Aws Resource Setup
#### AWS Resource Setup

Fleet provisioning requires some additional AWS resources be set up first. This section documents the steps you need to take to
get the sample up and running. These steps assume you have the AWS CLI installed and the default user/credentials has
sufficient permission to perform all of the listed operations. These steps are based on provisioning setup steps
that can be found at [Embedded C SDK Setup](https://docs.aws.amazon.com/freertos/latest/lib-ref/c-sdk/provisioning/provisioning_tests.html#provisioning_system_tests_setup).

First, create the IAM role that will be needed by the fleet provisioning template. Replace `RoleName` with a name of the role you want to create.
First, create the IAM role that will be needed by the fleet provisioning template. Replace `RoleName` with a name of the role you want to create.
``` sh
aws iam create-role \
--role-name [RoleName] \
Expand All @@ -325,17 +380,17 @@ aws iam attach-role-policy \
--role-name [RoleName] \
--policy-arn arn:aws:iam::aws:policy/service-role/AWSIoTThingsRegistration
```
Finally, create the template resource which will be used for provisioning by the demo application. This needs to be done only
once. To create a template, the following AWS CLI command may be used. Replace `TemplateName` with the name of the fleet
provisioning template you want to create. Replace `RoleName` with the name of the role you created previously. Replace
`TemplateJSON` with the template body as a JSON string (containing escape characters). Replace `account` with your AWS
account number.
Finally, create the template resource which will be used for provisioning by the demo application. This needs to be done only
once. To create a template, the following AWS CLI command may be used. Replace `TemplateName` with the name of the fleet
provisioning template you want to create. Replace `RoleName` with the name of the role you created previously. Replace
`TemplateJSON` with the template body as a JSON string (containing escape characters). Replace `account` with your AWS
account number.
``` sh
aws iot create-provisioning-template \
--template-name [TemplateName] \
--provisioning-role-arn arn:aws:iam::[account]:role/[RoleName] \
--template-body "[TemplateJSON]" \
--enabled
--enabled
```
The rest of the instructions assume you have used the following for the template body:
``` sh
Expand All @@ -345,13 +400,13 @@ If you use a different body, you may need to pass in different template paramete

#### Running the sample and provisioning using a certificate-key set from a provisioning claim

To run the provisioning sample, you'll need a certificate and key set with sufficient permissions. Provisioning certificates are normally
To run the provisioning sample, you'll need a certificate and key set with sufficient permissions. Provisioning certificates are normally
created ahead of time and placed on your device, but for this sample, we will just create them on the fly. You can also
use any certificate set you've already created if it has sufficient IoT permissions and in doing so, you can skip the step
that calls `create-provisioning-claim`.

We've included a script in the utils folder that creates certificate and key files from the response of calling
`create-provisioning-claim`. These dynamically sourced certificates are only valid for five minutes. When running the command,
`create-provisioning-claim`. These dynamically sourced certificates are only valid for five minutes. When running the command,
you'll need to substitute the name of the template you previously created, and on Windows, replace the paths with something appropriate.

(Optional) Create a temporary provisioning claim certificate set:
Expand All @@ -364,7 +419,7 @@ aws iot create-provisioning-claim \
```

The provisioning claim's cert and key set have been written to `/tmp/provision*`. Now you can use these temporary keys
to perform the actual provisioning. If you are not using the temporary provisioning certificate, replace the paths for `--cert`
to perform the actual provisioning. If you are not using the temporary provisioning certificate, replace the paths for `--cert`
and `--key` appropriately:
``` sh
Expand Down Expand Up @@ -415,7 +470,7 @@ python3 fleetprovisioning.py \
--key /tmp/provision.private.key \
--templateName [TemplateName] \
--templateParameters "{\"SerialNumber\":\"1\",\"DeviceLocation\":\"Seattle\"}" \
--csr /tmp/deviceCert.csr
--csr /tmp/deviceCert.csr
```

## Greengrass Discovery
Expand Down
154 changes: 154 additions & 0 deletions samples/pkcs11_pubsub.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,154 @@
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: Apache-2.0.

import argparse
from awscrt import io, mqtt
from awsiot import mqtt_connection_builder
import sys
import threading
import time
from uuid import uuid4
import json

# This sample is similar to `samples/pubsub.py` but the private key
# for mutual TLS is stored on a PKCS#11 compatible smart card or
# Hardware Security Module (HSM).
#
# See `samples/README.md` for instructions on setting up your PKCS#11 device
# to run this sample.
#
# WARNING: Unix only. Currently, TLS integration with PKCS#11 is only available on Unix devices.

parser = argparse.ArgumentParser(description="Send and receive messages through and MQTT connection.")
parser.add_argument('--endpoint', required=True, help="Your AWS IoT custom endpoint, not including a port. " +
"Ex: \"abcd123456wxyz-ats.iot.us-east-1.amazonaws.com\"")
parser.add_argument('--port', type=int, help="Specify port. AWS IoT supports 443 and 8883. (default: auto)")
parser.add_argument('--cert', required=True, help="File path to your client certificate, in PEM format.")
parser.add_argument('--pkcs11-lib', required=True, help="Path to PKCS#11 library.")
parser.add_argument('--pin', required=True, help="User PIN for logging into PKCS#11 token.")
parser.add_argument('--token-label', help="Label of PKCS#11 token to use. (default: None) ")
parser.add_argument('--slot-id', help="Slot ID containing PKCS#11 token to use. (default: None)")
parser.add_argument('--key-label', help="Label of private key on the PKCS#11 token. (default: None)")
parser.add_argument('--root-ca', help="File path to root certificate authority, in PEM format. (default: None)")
parser.add_argument('--client-id', default="test-" + str(uuid4()),
help="Client ID for MQTT connection. (default: 'test-*')")
parser.add_argument('--topic', default="test/topic",
help="Topic to subscribe to, and publish messages to. (default: 'test/topic')")
parser.add_argument('--message', default="Hello World!",
help="Message to publish. Specify empty string to publish nothing. (default: 'Hello World!')")
parser.add_argument('--count', default=10, type=int, help="Number of messages to publish/receive before exiting. " +
"Specify 0 to run forever. (default: 10)")
parser.add_argument('--verbosity', choices=[x.name for x in io.LogLevel], default=io.LogLevel.NoLogs.name,
help="Logging level. (default: 'NoLogs')")

# Using globals to simplify sample code
args = parser.parse_args()

io.init_logging(getattr(io.LogLevel, args.verbosity), 'stderr')

received_count = 0
received_all_event = threading.Event()


def on_connection_interrupted(connection, error, **kwargs):
# Callback when connection is accidentally lost.
print("Connection interrupted. error: {}".format(error))


def on_connection_resumed(connection, return_code, session_present, **kwargs):
# Callback when an interrupted connection is re-established.
print("Connection resumed. return_code: {} session_present: {}".format(return_code, session_present))


# Callback when the subscribed topic receives a message
def on_message_received(topic, payload, dup, qos, retain, **kwargs):
print("Received message from topic '{}': {}".format(topic, payload))
global received_count
received_count += 1
if received_count == args.count:
received_all_event.set()


if __name__ == '__main__':
# Spin up resources
event_loop_group = io.EventLoopGroup(1)
host_resolver = io.DefaultHostResolver(event_loop_group)
client_bootstrap = io.ClientBootstrap(event_loop_group, host_resolver)

print(f"Loading PKCS#11 library '{args.pkcs11_lib}' ...")
pkcs11_lib = io.Pkcs11Lib(
file=args.pkcs11_lib,
behavior=io.Pkcs11Lib.InitializeFinalizeBehavior.STRICT)
print("Loaded!")

# Create MQTT connection
mqtt_connection = mqtt_connection_builder.mtls_with_pkcs11(
pkcs11_lib=pkcs11_lib,
user_pin=args.pin,
slot_id=int(args.slot_id) if args.slot_id else None,
token_label=args.token_label,
private_key_label=args.key_label,
cert_filepath=args.cert,
endpoint=args.endpoint,
port=args.port,
client_bootstrap=client_bootstrap,
ca_filepath=args.root_ca,
on_connection_interrupted=on_connection_interrupted,
on_connection_resumed=on_connection_resumed,
client_id=args.client_id,
clean_session=False,
keep_alive_secs=30)

print("Connecting to {} with client ID '{}'...".format(
args.endpoint, args.client_id))

connect_future = mqtt_connection.connect()

# Future.result() waits until a result is available
connect_future.result()
print("Connected!")

# Subscribe
print("Subscribing to topic '{}'...".format(args.topic))
subscribe_future, packet_id = mqtt_connection.subscribe(
topic=args.topic,
qos=mqtt.QoS.AT_LEAST_ONCE,
callback=on_message_received)

subscribe_result = subscribe_future.result()
print("Subscribed with {}".format(str(subscribe_result['qos'])))

# Publish message to server desired number of times.
# This step is skipped if message is blank.
# This step loops forever if count was set to 0.
if args.message:
if args.count == 0:
print("Sending messages until program killed")
else:
print("Sending {} message(s)".format(args.count))

publish_count = 1
while (publish_count <= args.count) or (args.count == 0):
message = "{} [{}]".format(args.message, publish_count)
print("Publishing message to topic '{}': {}".format(args.topic, message))
message_json = json.dumps(message)
mqtt_connection.publish(
topic=args.topic,
payload=message_json,
qos=mqtt.QoS.AT_LEAST_ONCE)
time.sleep(1)
publish_count += 1

# Wait for all messages to be received.
# This waits forever if count was set to 0.
if args.count != 0 and not received_all_event.is_set():
print("Waiting for all messages to be received...")

received_all_event.wait()
print("{} message(s) received.".format(received_count))

# Disconnect
print("Disconnecting...")
disconnect_future = mqtt_connection.disconnect()
disconnect_future.result()
print("Disconnected!")
2 changes: 1 addition & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ def _load_version():
"Operating System :: OS Independent",
],
install_requires=[
'awscrt==0.12.1',
'awscrt==0.13.0',
],
python_requires='>=3.6',
)

0 comments on commit 03eb38e

Please sign in to comment.