blobs.contentType can be NULL

[alxndrsn]

• there is no NOT NULL constraint on the blobs.contentType column, i.e. column is NULLABLE
• NULL values are turning up in production instances
• null values for Blob frames throw in s3 code: TypeError: response header response-content-type should be of type "string"

Related

Issues

• Form attachment content type should be determined server-side #489

PRs

• submissions/attachments: demonstrate current null Content-Type handling #1352
• s3: fix handling of NULL blob.contentType values #1353
• test/e2e/api: include response status text in errors if no response body #1354
• blobs: prevent NULL contentType in db #1355
• blobs: derive missing contentType from file extension #1356

Potential fixes

1. ✅ continue to allow NULL values, but handle these in s3 code (s3: fix handling of NULL blob.contentType values #1353)
2. set a default value for blobs."contentType" in the db, e.g. application/octet-stream; try to guess content type from file extension if no contentType is provided on upload
3. reject null contentType values when trying to insert values in the db
4. set a default contentType value for blobs with NULL values in db

[lognaturel]

We believe these nulls are created in two ways: when someone uploads a file using the API instead of a browser and when someone uses a browser that does not recognize geojson files (#489). The nulls are causing production servers that use S3 to crash.

Fix 1 feels sufficient to me for a patch release.

I think 3 is unacceptable because users may not have a way to fix the issue and the contenttype information is not that important.

I'd like to propose doing 2 and also setting the mime types for files with geojson and cvs extensions based on extension. These are the two cases we know of where nulls have been introduced. There are suggestions for stronger validations and type determination at #489 (comment) and below but they don't feel worth the effort to me at this time.

I don't feel like we need to do anything for blobs that already nave null contenttypes in the db beyond the fix 1 handling.

[ktuite]

Fix 1 sounds good as a patch release to me, too. We don't have to get into the messy details of blobs and content types, but it does stop breaking things.

I was able to upload blobs with null content types through the API, and in doing so, noticed we don't have a lot of documentation for uploading attachments. I was actually struggling to explicitly set a content type on my attachment uploads.

Forms don't seem to have anything about sending attachments: https://docs.getodk.org/central-api-form-management/#listing-form-attachments
Submissions do have some details: https://docs.getodk.org/central-api-submission-management/#uploading-an-attachment

[alxndrsn]

I'd like to propose doing 2 and also setting the mime types for files with geojson and cvs extensions based on extension.

👍 I've added this to the description of #2