More informative error message when request too long

[djvanderlaan]

When there are too many filter statements, the request URL to the odata portal becomes too long and the request fails. Currently, the resulting error message is very uninformative. Most people probably don't know how to address this error.

Example:

library(cbsodataR)

tbl <- "70072ned"
meta <- cbs_get_meta(tbl)

gemeentes <- meta$RegioS$Key
gemeentes <- gemeentes[grep("^GM", gemeentes)]
jaren <- meta$Perioden$Key

onderwerpen <- c("TotaleBevolking_1",
  "JongerDan5Jaar_4",
  "k_5Tot10Jaar_5",
  "k_10Tot15Jaar_6",
  "k_15Tot20Jaar_7",
  "k_20Tot25Jaar_8",
  "k_25Tot45Jaar_9",
  "k_45Tot65Jaar_10",
  "k_65Tot80Jaar_11",
  "k_80JaarOfOuder_12",
  "Bevolkingsdichtheid_57",
  "VestigingUitAndereGemeente_69",
  "VertrekNaarAndereGemeente_70",
  "BinnenlandsMigratiesaldo_71",
  "BinnenlandsMigratiesaldoRelatief_72",
  "VerhuismobiliteitRelatief_73",
  "Bevolkingsgroei_79",
  "TotaalAantalParticuliereHuishoudens_82",
  "VoorraadOp1Januari_90",
  "Nieuwbouwwoningen_91",
  "Woningen_97",
  "GemiddeldeWoningwaarde_99",
  "TotaleOppervlakte_187")

dta <- cbs_get_data(id = tbl, Perioden = jaren, 
  RegioS = gemeentes, 
  select = c("Perioden", "RegioS", onderwerpen))

This result in either the following error message:

Error in curl::curl_fetch_memory(url, handle = handle) : 
  OpenSSL SSL_read: SSL_ERROR_SYSCALL, errno 104

Or sometimes the following:

Error in get_json(url, verbose = verbose) : 
  Request-URI Too Long (HTTP 414). Failed to Client error: (414) Request-URI Too Long.

Perhaps add the following lines to cbs_download_data after url <- URLencode(url):

if (nchar(url) > 2000L) 
  warning(paste0(c("The request URL is longer than 2000 characters. ", 
    "This could cause the request to fail on some platforms. ", 
    "If so, try to reduce the number of filter statements and filter the data afterwards.")))

Or, catch the error: something like:

res <- get_json(url, verbose = verbose)
tryCatch({
  res <- get_json(url, verbose = verbose),
}, error = function(e) {
    warning <- if (nchar(url) < 2000L) "" else 
      paste0(c("\n\nThe request URL is longer than 2000 characters. ", 
        "This could cause the request to fail on some platforms. ", 
        "Try to reduce the number of filter statements and ",
        "filter the data afterwards."))
    stop("Request failed with the following message:\n", e$message, warning)
})