How to differentiate between which hardware the experiment was run on?

[sgbaird]

If the data gets dumped into a single MongoDB collection, there's no way to distinguish which device the experiment was run on, and there's no curation as to whether fake data is being uploaded. There are a few options:

• I could use the Pico ID, but then it's sort of like making the default password "password" in a way that's accessible by anyone when the hardware is connected to the internet (not a huge deal from a security perspective for this demo - someone starts blinking a light and reading light channel data from your device, but not scalable to solutions that require access restriction What does access restriction look like for a scaled up cloud-accessible lab? #91 EDIT: use of Pico ID wouldn't work, because then the Pico ID would be pubicly visible as soon as one data entry gets used - like putting a publicly viewable data entry with the password in it
  • This also leaves room for someone falsely putting data under a particular device when it wasn't actually generated from there. This leads to an access restriction model where only users with specific credentials can contribute to specific collections

In other words, the ID that defines the hardware has to be unique to a device and private to only the users of that device. It should also be separate from credentials that are used to control the hardware, so that the database manager can see the database ID but not have remote hardware access.

On the MongoDB side:

• A role is a named set of permissions that an end-user can have for a document when for a request to the app. Roles are evaluated in order from top to bottom using the apply_when expression.

• Filters modify an incoming MongoDB query to return only a subset of the results matched by the query. Filters will filter out data that is returned by MongoDB before roles are evaluated, making them useful for request performance improvements.

• Requires having a unique (private) key and a collection associated with each hardware instance, where the key and collection get generated by a database manager, or preferably, automatically via some request form that has a mild form of authentication (e.g. Google account)
• MongoDB also has an authentication tab under data access
• API Keys seem to be at the database level, not at the collection level. However, maybe the API key can be distributed freely, while the ID associated with each user is kept private and is part of a custom role or filter. However, registering users requires some setup. EDIT: multiple API keys can be created, each of which have their own IDs associated with them. In other words, API keys can be assigned on a per-person basis, a collection created for each ID, and then have the ID checked before adding to a particular database. The ID should be kept private in this case, but will be accessible by the database manager. The Pico ID doesn't have to be disclosed to the database manager, and it can be changed from the default.

[rekumar]

1. Agree that using using hardware ID's is a bad idea, if only because development boards (arduino's for sure, maybe rpi's) can reuse ID's. At least this is the case for arduino uno's I have purchased from third party resellers.

2. Maybe a public-private key encryption scheme (like RSA, python example) is appropriate here?
   Example: the private key lives on your hardware device, the public key is the device identifier within your database. Any data generated by the device is terminated with some signature, then encrypted using the private key. Your database API then decrypts the data using the public key and checks for the signature before accepting the new data.

As a bonus, you can use these keys to do device-side validation (using its private key) to block instructions from bad actors. Instructions sent to the device would be encrypted using its public key, then decrypted on the device side to check the signature. In this scenario, maybe you don't want to use the public key as the device's ID in the database... you could encrypt the device's ID using a server-side private key to hide this from external viewers of the database.

3. This is a very forward-thinking problem to solve for most of our self-driving labs! Practically speaking, most of these labs will be operating on a university/national lab LAN. In my SDL's, I've relied on 1) our IT professionals to protect our networks from malicious actors, and 2) good API's with type + value checking to make it exceedingly difficult for our internal users to mess things up!

[rekumar]

Agreed about requiring a wrapper API and a scheduling/queue system. As I started digging in, I realized there were a lot of nuances dealing with access restrictions, secure data transfer, etc.:

I struggled a lot with the idea of switching from a single board computer (e.g. RPi 4B) to a microcontroller (i.e. RPi Pico W) and hence MicroPython instead of full Python.

I understand your perspective a bit more now. In my mind all access control/input checking/etc should be handled at the application level. As long as your hardware only accepts instructions from your application, this should be fine. Not hard to achieve this isolation using an encrypted handshake or an internal (or even dedicated) LAN for communication. The minor risk is that a developer could change the input checking step, though this seems reckless and unlikely. I'd even argue that moving hardware limits to the application level can improve transparency + avoid duplication (ie repeat code at API and hardware levels). However the riskiest cases are probably easy to catch on hardware without something like pydantic (ie if temp>500 raise TooHotError) Additionally, you can imagine limits that involve multiple steps of the SDL (ie temperature limits change based on your solvent's boiling point), which have to be caught at the application level.

Anecdotally, all beamlines I've used as an external user rely on (often easily bypassed) software limits to avoid instrument collisions etc, and they seem to function well enough.

I would love to see a SDL of sufficient scale where these are important issues :)

[sgbaird]

Great points about de-duplication between hardware and API and how to deal with multiple steps. I like the idea of risky situation avoidance being hardcoded on the device. I also like the idea of the application throwing an error for invalid inputs before it starts communicating with external systems.

For SDLs with multiple steps (i.e. multiple pieces of hardware that require separate communication), I lean toward the idea of the user communicating with a central processor as a "firewall" of sorts (e.g., physical hardware or cloud-based server) rather than having the user directly communicate with the individual pieces of hardware. Something where the scope of users that can change code on the physical hardware, server, etc. is narrower than the scope of users that can send requests to it.

I would love to see a SDL of sufficient scale where these are important issues :)

Agreed! Hoping to help make that a reality :)

[sgbaird]

@rekumar, apologies for resurfacing such an old thread, but I wanted to follow up on something.

Agree that using using hardware ID's is a bad idea, if only because development boards (arduino's for sure, maybe rpi's) can reuse ID's. At least this is the case for arduino uno's I have purchased from third party resellers.

Rather than use the hardware ID as some kind of password for authentication, I'm now thinking about using them to make it easier to distinguish between devices when using MQTT communication. Specifically, I'm considering using them within the MQTT "topics" to help organize the messages being sent to various devices. The messages would also include task IDs that get passed back and forth to help prevent crossing wires when messages are being sent around. I guess there's still the risk of two devices happening to have identical APIs and identical hardware IDs..

For context, I've been collecting my thoughts about a "naming schema" in #237.

[sgbaird]

@rekumar i.e., some sort of hardware ID/serial number would be used in the communication protocol while being

For example: pioreactor/ek193gai and pioreactor/fir385sjd would be communication topics for two separate devices. The orchestrator and each device would have a sort of handshake via the orchestrator sending an experiment/task UUID, the device returning that in the response (or in a corresponding database entry), and the orchestrator checking that the it matches the original.

Would you still recommend against using a hardware ID in this case, and instead generating a UUID? This would no longer be for privacy/security (which is handled by a private, secure broker), but rather as a convenient way to organize communication across devices, especially from the orchestrator's perspective.

[rekumar]

I think it's fine as long as you cover the scenario where two devices can have the same hwid and taskid. In many SDLs this just won't happen, but I can also see cases like two robot arms with the same hwid from the supplier, and you need both in a single task (like telling one arm to stay out of the way while the other picks a sample from a shared work envelope).

A few solutions:

• easy but still not perfect: use a combination of device IDs (like vendor id + product id + others), you can hash multiple values to still have a short id for convenience
• most robust, like for a modular lab: you have a device discovery layer in the middle. As devices come online, they ping this layer and get a unique id. When they sign off, that id is "offline". You can implement your own logic to determine this ID, which could involve hwid. In the past I've predefined unique names for the devices in their drivers, and just used the name as the unique ID -- it's a lot easier to read logs and stuff this way too. This discovery layer can also handle things like tracking device state, advertising the capabilities of each device, etc.
• middle ground -- use hwid as device id. when a device "signs on", it sends a message to an mqtt topic to ensure no other device is using this hwid. if there is already one, throw an error. You could do this without a device registry/discovery layer by programming your base device driver to reply to messages on its own topic like "{hwid}-ping". Upon connection, you try to ping on that topic and wait for like a 5s timeout to call it good.

[sgbaird]

Summary of access restrictions (minus the ability to assign private collections):

privilege	database manager	device owner	device user	general public	solution
read public data	✔️	✔️	✔️	✔️	public API key, MongoDB filters to only public collections
write to database	✔️	✔️	❌	❌	one MongoDB database API key per device owner
assign new database API keys	✔️	❌	❌	❌	via MongoDB online GUI
access map from database API keys to users	✔️	❌	❌	❌	via MongoDB online GUI
send commands to hardware	❌	✔️	✔️	❌	via user-specific authentication
assign new device users	❌	✔️	❌	❌	manually generate new authentication info

The database manager and the device owner each require exclusive, unique authentication. The database manager gets this through MongoDB login credentials. The device owner gets this by having physical access to the hardware and an optional, external server (e.g., pythonanywhere). The user needs to communicate privately with the device which can be handled through manual encryption #133. The device needs to communicate privately with the database, which MongoDB handles.

Related:

• https://crypto.stackexchange.com/questions/9723/vulnerabilities-if-encrypting-the-same-data-with-2-different-keys
• https://www.hackster.io/sandeep-mistry/connect-your-raspberry-pi-pico-w-to-aws-iot-core-8868b7
• https://forum.micropython.org/viewtopic.php?t=11906&p=64760
• https://www.venafi.com/blog/what-difference-between-public-key-and-private-key
• How to differenciate devices/users on mosquitto? eclipse-mosquitto/mosquitto#1645
• https://stackoverflow.com/questions/73000035/trying-to-use-mosquitto-broker-with-tls-using-paho-python

This is closely related to setups for materials Informatics data platforms like Materials Project and NOMAD. Citrination distinguishes between public vs. private collections.

[sgbaird]

MongoDB Docs: Define Permissions for Specific API Keys. Perhaps one collection per device via collection-specific permissions.

User ID of each API key comes from:

Permissions get set as follows:

This seems to be functioning as intended.

I'm still stumped on how to use https://test.mosquitto.org as the host while ensuring that only "authorized" clients get to run experiments on the device. It seems like I need to use SSL with certificates and an encrypted port (which interestingly seems to be commented out on test.mosquitto.org)

• Pico W: Issues with ussl / usocket with encrypted MQTT micropython/micropython#9259
• ussl module for Raspberry Pico W micropython/micropython#9222

EDIT: I think I figured out the secure handshake by using a free instance of HiveMQ and the directions at https://community.hivemq.com/t/getting-started-raspberry-pi-pico-w/1316/21?page=2. PR to follow soon.

[sgbaird]

For provenance, here is a thread from the Eclipse IoT Slack Group:

I want to send instructions to control LEDs from a Python client (we'll call this "Tim") to a MicroPython device (we'll call this "Jill") via the test.mosquitto.org host. Anyone can send instructions to Jill as long as they know the topic, but I want it so that Jill only carries out instructions that were given by Tim.
If I use port 1883, then any messages that are sent are unencrypted and unauthenticated, meaning that a set of instructions can be easily be copied and resent, such that Jill won't be able to tell if the new message is actually from Tim vs. someone pretending to be Tim.
I think one solution is to use encryption (e.g. port 8883) and include a private key that's known only to Tim and Jill. Does this seem right?

Reply from @NorbertHeusser::

Using port 8883 will not really in your case. The TLS layer will only help encrypt the traffic between each of the clients and the broker. It will not be encrypted inside the server. So using an encrypted socket between your client and the broker will only help to avoid anyone will steal you identity. Speaking in you terms anyone between Tim's computer and the server may steal Tims identity (username/password) and kick out the real Tim any time, if you don't use TLS connection.
But still everybody with wildcard subscription can easily see the messages send from Tim to Jill.
To avoid this you would either need to setup you own server where you have full control over the server. And this way nobody may connect to your broker and pickup the data. Or you book a hosted mosquitto somewhere. If you want to test how all of this works you may use one of our (I work for cedalo.com) free trial instances for a limited time.
If you want to stay with one of the open servers like test.mosquitto.org you would need to implement you own end-to-end encryption. Easiest way would be to use a private share secret. So some random secret know by Tim and Jill. Now Tim may use a symmetric encryption algorithm to encode the message and send it encrypted (e.g. as a bas64 encoded string). Now Jill knowing the secret and the algorithm used will be able to decode it. Or you "sign" you message by based on the secret so Jill is able to verify the message is really from Tim by checking the signature.

If you don't have a save way to transfer a secret from Jim to Jill upfront it will get more complicated ;-)

Thank you for the thorough answer! A private server with TLS seems like the way to go. Symmetric encryption alone would (mostly) prevent fake Tim from knowing what the message actually means, but it wouldn't stop fake Tim from copying that same message verbatim and spamming it to Jill, correct?

Reply from @NorbertHeusser:

Yes, you are right. Fake Tim would be able to replay. You could add an incrementing Id with each request and drop requests with older ids. But yes, it get more and more complicated.

EDIT: However, see note from post above about HiveMQ, which solved the issue for me.