Provide script that calls Fastly CLI to rewrite dictionary with current contents of CSV

[pnorman]

We need to start updating the attribution_blocked_domains dictionary with a shell script to avoid manual work. This script will need to call the fastly CLI.

As far as I can tell, this needs to be done by

1. calling fastly dictionary describe to get the dictionary ID
2. Get items with fastly dictionary-entry list
3. Diff the items against the CSV file
4. Add any missing, and remove any that have been removed.

It is important that it does not rewrite the entire dictionary contents, as that will remove the date added information which is useful.

See also https://developer.fastly.com/learning/concepts/edge-state/dynamic-config/#edge-dictionaries

[Firefishy]

Rather please move to using terraform-fastly (private repo). Terraform can read the CSV file direct from github if neccessary.

[Firefishy]

https://registry.terraform.io/providers/fastly/fastly/latest/docs/resources/service_dictionary_items

[Firefishy]

Example:

resource "fastly_service_dictionary_items" "attribution_blocked_domains" {
  dictionary_id = "1EMhjtZi0Mwq5ZcSElM5Gs"
  items = {
    "saverudata.info"        = "LWG email of 2022-08-01"
    "www.impresaitalia.info" = "LWG email of 2022-08-01"
    "www.topozone.com"       = "board request via grischard of 2023-06-29"
  }
  manage_items = true
  service_id   = "3vqMSxvFYqyVOoTY8ti37P"
}

You can literally start just with that in the terraform.

[pnorman]

We don't have terraform ready to go, and no ETA on when it will be ready.

[Firefishy]

We don't have terraform ready to go, and no ETA on when it will be ready.

I've literally posted the only terraform code that is required above. I've now even made the job easier by publishing JSON which can be used directly by terraform HCL: https://raw.githubusercontent.com/openstreetmap/tile-attribution/main/tile-attribution.json

[Firefishy]

Fully working code:

terraform {

  required_version = "~> 1.5.6"

  required_providers {
    fastly = {
      source  = "fastly/fastly"
      version = ">= 5.4.0"
    }
  }
}

# Configure the Fastly Provider
# Use export FASTLY_API_KEY="afastlyapikey" terraform plan
provider "fastly" {}

data "http" "tile_attribution_data" {
  url = "https://raw.githubusercontent.com/openstreetmap/tile-attribution/main/tile-attribution.json"
}

import {
  id = "3vqMSxvFYqyVOoTY8ti37P/1EMhjtZi0Mwq5ZcSElM5Gs"
  to = fastly_service_dictionary_items.attribution_blocked_domains
}

resource "fastly_service_dictionary_items" "attribution_blocked_domains" {
  dictionary_id = "1EMhjtZi0Mwq5ZcSElM5Gs"
  items         = jsondecode(data.http.tile_attribution_data.response_body)
  manage_items  = true
  service_id    = "3vqMSxvFYqyVOoTY8ti37P"
}

[pnorman]

If it's ready to go, that's good. Closing, and stopping manual updates.