Bugs in Backblaze B2 path handling

[logiclrd]

This issue is not a bug in this code, but it is a bug and it affects code that is using this library.

Based on hints in the documentation, it is suggested that the Backblaze B2 API was initially created as an exclusively HTTP POST based system, with parameters in JSON bodies or HTTP headers. More recently, a push was made to transition to idiomatic HTTP GET for get-like endpoints, which put file paths into the request URL. I have encountered problems with this, with certain characters that are explicitly allowed in filenames (per the documentation) not working with common endpoints like b2_download_file_by_name.

After some investigation, the work-around that I have come to is to transition any operation where a problematic filename is causing problems to the corresponding one using fileId. This means that an additional operation is needed to determine the fileId value that corresponds to the filename, but this operation can be done in a way that does not have these problems with common characters in filenames.

Characters that I have encountered causing this problem include:

• , (comma)
• [ (open square bracket)
• ] (close square bracket)
• & (ampersand)
• _ (underscore)

One possible course forward would be to codify the workaround in the implementation of DownloadFileByName so that that type of request just works even if an actual b2_download_file_by_name would have failed.

Another possibility is merely to document the problem so that developers know it exists.

This is, in essence, the function I wrote to facilitate translating a request from a filename to a fileId-based request:

    async Task<string?> GetFileIdByName(string remoteStorageBucketId, string fileName, bool throwIfNotFound = true)
    {
      var request = new ListFileVersionRequest(remoteStorageBucketId);

      request.StartFileName = fileName;
      request.MaxFileCount = 1;

      var response = await _b2Client.Files.ListVersionsAsync(request, cacheTTL: TimeSpan.FromSeconds(10));

      response.EnsureSuccessStatusCode();
    
      var file = response.Response.Files.SingleOrDefault(file => file.FileName == fileName);

      if (file == null)
      {
        if (throwIfNotFound)
          throw new FileNotFoundException();
        else
          return null;
      }

      return file.FileId;
    }

This function can then be used like this:

      Task<IApiResults<DownloadFileResponse>> task;

      if (fileName.IndexOfAny(B2ProblematicFileNameCharacters) < 0)
      {
        // Fast path: b2_download_file_by_name
        var request = new DownloadFileByNameRequest(FindAndCacheBucketName(), fileName);

        task = _b2Client.DownloadAsync(request, destinationStream, default, cancellationToken);
      }
      else
      {
        // Workaround: b2_list_file_versions -> b2_download_file_by_id
        var fileID = GetFileIdByName(fileName);

        var request = new DownloadFileByIdRequest(fileID);

        task = _b2Client.DownloadByIdAsync(request, destinationStream, default, cancellationToken);
      }

      var result = await task;

      if (!result.IsSuccessStatusCode)
        throw new Exception("The operation did not complete successfully.");

(NB, these functions might have some minor errors and may require a bit of work, because I had to modify the actual code from my project to remove unnecessary complexity, so this is not exactly the state of the code in my codebase.)

Hopefully this information can be useful to somebody :-)