Events over webhooks example app #12

bowenwr · 2024-04-04T14:54:53Z

Adds an example app demonstrating receiving a data event from Benchling and writing the results to an external PostgreSQL database.

Copies many of the files and patterns (e.g., localtunnel) from examples/chem-sync-local-flask.

bowenwr · 2024-04-04T14:57:46Z

examples/event-webhooks-local-flask/README.md

+
+![image info](./docs/demo-full.gif)
+
+> ℹ️ Events delivered by webhooks is in early access and unavailable by default. To use this example, please reach out to Benchling support to enable webhook delivery for events on your tenant.


@Psterne @dgervais-benchling I didn't see any public docs on getting setup with Events over Webhooks. I also used "Events delivered by webhooks" as I feel like "events over webhooks" is our internal language and not necessarily intuitive to users.

Please suggest any alternative copy you'd like here or elsewhere in the README!

@bowenwr Updated docs for Events (via Webhooks and Eventbridge) are WIP. We can link to it from here once those are live.

I agree that "Events delivered by webhooks" is better. At some point I'm hoping we can just call it "Webhooks"!

bowenwr · 2024-04-04T14:59:21Z

examples/event-webhooks-local-flask/README.md

+A database password can be created now. It can be any valid PostgreSQL password. One way to generate a random password for local development purposes on *nix systems might be:
+
+```bash
+date +%s | sha256sum | base64 | head -c 16 | cat > .database_password


@jasonbenchling or @stash-benchling for sec eng (Jason previously reviewed the last example App). Wanted to be sure this was reasonable guidance.

I'd actually almost prefer to disable passwords for postgres for this local example only, but I figured it was better to:

Have the user set some password, even if it's weak

Store it securely (e.g., via Docker secrets)

I would not use the current time for a source of entropy on it's own. Stakes are low here, this is a test/sample app, but we don't want people YOLOing that pattern into production 😅

Since we assume the user has python available (after all, it's a flask app), we can use a python one liner to simplify getting secure randomness. There's some bash gymnastics that need to be done to read directly from /dev/urandom and get usable output, so better to take advantage of python's wrapper IMO.

Something like:
python -c "import secrets; print(secrets.token_hex(64))" > .database_password would be preferable

@jasonbenchling Thanks for the quick response. The Flask App runs its own container where Python is installed. I tried this in my own local terminal (where python is not installed globally) and it failed within my virtual environment 😅

I'm hoping to reduce the number of steps / requirements for developers, so I think the safest thing to do is just remove this "random" password generation and tell the user to edit in their own?

@bowenwr That seems reasonable -- tell the end user to "generate and store a secure random password" or something along those lines.

openssl is another probably pretty common option to generate randomness, but that also might not always be present. Some of the internet suggested /dev/urandom based methods that work on linux boxes don't work on my mac, so that's probably not a great portable option either.

bowenwr · 2024-04-04T15:00:12Z

examples/event-webhooks-local-flask/README.md

+### Benchling Prerequisites
+1. Access to a Benchling tenant, like `https://my-tenant.benchling.com`
+2. Ensure you've been granted access to the [Benchling Developer Platform Capability](https://help.benchling.com/hc/en-us/articles/9714802977805-Access-the-Benchling-Developer-Platform).
+3. This example also requires Events delivered by Webhooks to be enabled on your tenant. Reach out to Benchling support to find out more about participating in early access.


@Psterne @dgervais-benchling similar to above, let me know if you want different copy to express this requirement.

Looks good. Is there a step 4 that's missing? Should Global Apps be step 4?

The number skipped 4 and went to 5. Fixed now, thanks for catching!

bowenwr · 2024-04-04T15:01:35Z

examples/event-webhooks-local-flask/local_app/benchling_app/handler.py

+    if isinstance(webhook.message, EventCreatedWebhookV0Beta):
+        # Since we'll be receiving ALL entity registrations, add a filter to only
+        # work with events that meet our criteria (e.g., specific schema)
+        if _is_target_event(app, webhook.message):


@edwin-benchling This filtering logic is one of the main differences from the other example App that would merit closer review.

bowenwr · 2024-04-04T15:02:03Z

examples/event-webhooks-local-flask/local_app/benchling_app/sync_data.py

+from local_app.lib.postgresql import write_data
+
+
+def sync_event_data(app: App, entity_registered: V2EntityRegisteredEventBeta) -> None:


@edwin-benchling This is the second file that's "new" and could use closer review.

bowenwr · 2024-04-04T15:02:43Z

examples/event-webhooks-local-flask/local_app/lib/postgresql.py

+logger = get_logger()
+
+
+def write_data(entity_name: str, benchling_api_id: str, field_value: str | None) -> str:


@edwin-benchling This is the third and final file that is new and merits closer review.

Although I don't wanna focus too hard on teaching users how to connect to PostgreSQL in Python and intentionally kept this light.

bowenwr · 2024-04-04T15:03:49Z

FYI @benchling/app-ecosystem for broad awareness.

I tagged @edwin-benchling as DRI for engineering review, but feedback is welcome from everyone.

bowenwr · 2024-04-04T17:21:19Z

examples/event-webhooks-local-flask/README.md

+echo.> .database_password
+```
+
+A database password can be created now. It can be any valid PostgreSQL password. Generate and store a secure password in `.database_password`. For local development, you could open the file and manually enter any valid password.


@jasonbenchling Removed the "random generation" bash example and converted to general guidance.

jasonbenchling

Nit re: pbpaste, but other than that, looks good

jasonbenchling · 2024-04-04T17:51:00Z

examples/event-webhooks-local-flask/README.md

+
+```bash
+touch .client_secret
+pbpaste > .client_secret


pbpaste is just a MacOS thing, afaik

bowenwr · 2024-04-04T18:58:29Z

Nit re: pbpaste, but other than that, looks good

Ugh, we also have it in examples/chem-sync-local-flask. Nobody complained during the live workshop so I wonder if people just rolled with it. 😅

bowenwr · 2024-04-05T14:25:47Z

FYI @benchling/solutions-teams for an App code example using the new beta Events over Webhooks functionality.

edwin-benchling · 2024-08-02T20:57:25Z

examples/event-webhooks-local-flask/local_app/lib/postgresql.py

+
+
+def _connection() -> psycopg2.extensions.connection:
+    return psycopg2.connect(


I feel like it'd be easier to build a Repository pattern and implement it with an in-memory database and leave it as an exercise to the reader to provide their own implementation of a durable database, especially if the point of the example code is not really about persistence...

but since this is already done, I don't feel as strongly.

edwin-benchling · 2024-08-02T20:58:34Z

examples/event-webhooks-local-flask/tests/unit/local_app/lib/test_postgresql.py

+
+
+class TestPostgreSql:
+    # postgresql fixture provided by pytest-postgresql


Note: I'm not sure why, but this fixture did not kick in when running tests locally. I'd just get errors like

> raise ExecutableMissingException( f"Could not found {self.executable}. Is PostgreSQL server installed? " f"Alternatively pg_config installed might be from different " f"version that postgresql-server." ) from ex E pytest_postgresql.exceptions.ExecutableMissingException: Could not found /usr/lib/postgresql/14/bin/pg_ctl. Is PostgreSQL server installed? Alternatively pg_config installed might be from different version that postgresql-server.

Events over webhooks example app

12e0517

bowenwr marked this pull request as ready for review April 4, 2024 14:56

bowenwr requested review from edwin-benchling and Psterne April 4, 2024 14:56

bowenwr commented Apr 4, 2024

View reviewed changes

Code review - update security guidance for password generation

0c57737

bowenwr requested a review from jasonbenchling April 4, 2024 17:20

bowenwr commented Apr 4, 2024

View reviewed changes

jasonbenchling approved these changes Apr 4, 2024

View reviewed changes

bowenwr added 2 commits April 4, 2024 15:19

Add note about scaling

acf1980

Upgrade Benchling SDK to 1.12.0

4ebac47

edwin-benchling added 5 commits August 2, 2024 11:11

update refs from v0 to v2

a0f257b

ruff

f1b1ca3

mypy and unit test

d244939

fix ut

8428fbd

lint

216c7d4

edwin-benchling reviewed Aug 2, 2024

View reviewed changes

edwin-benchling approved these changes Aug 2, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Events over webhooks example app #12

Events over webhooks example app #12

bowenwr commented Apr 4, 2024

bowenwr Apr 4, 2024

Psterne Apr 4, 2024

bowenwr Apr 4, 2024

jasonbenchling Apr 4, 2024

bowenwr Apr 4, 2024

jasonbenchling Apr 4, 2024

bowenwr Apr 4, 2024

Psterne Apr 4, 2024

bowenwr Apr 4, 2024

bowenwr Apr 4, 2024

bowenwr Apr 4, 2024

bowenwr Apr 4, 2024

bowenwr commented Apr 4, 2024

bowenwr Apr 4, 2024

jasonbenchling left a comment

jasonbenchling Apr 4, 2024

bowenwr commented Apr 4, 2024

bowenwr commented Apr 5, 2024

edwin-benchling Aug 2, 2024

edwin-benchling Aug 2, 2024


		![image info](./docs/demo-full.gif)

		> ℹ️ Events delivered by webhooks is in early access and unavailable by default. To use this example, please reach out to Benchling support to enable webhook delivery for events on your tenant.

		from local_app.lib.postgresql import write_data


		def sync_event_data(app: App, entity_registered: V2EntityRegisteredEventBeta) -> None:

		logger = get_logger()


		def write_data(entity_name: str, benchling_api_id: str, field_value: str \| None) -> str:



		def _connection() -> psycopg2.extensions.connection:
		return psycopg2.connect(



		class TestPostgreSql:
		# postgresql fixture provided by pytest-postgresql

Events over webhooks example app #12

Are you sure you want to change the base?

Events over webhooks example app #12

Conversation

bowenwr commented Apr 4, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bowenwr commented Apr 4, 2024

Choose a reason for hiding this comment

jasonbenchling left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bowenwr commented Apr 4, 2024

bowenwr commented Apr 5, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment