Learnings _ Gotchas for API v2

JSONAPI::Resources Gem

Classes for Resources

There are three main classes you will need to consider making sub-classes for to support a resource. Each has a specific purpose.

• Controllers
• Resources
• Operation Processors

Controllers

Extending JSONAPI::ResourceController, these are the entry point for requests to the JSON:API system. They can mostly be left empty, but may require specific overrides under certain conditions. For an example of this, check out QcAssaysController which overrides the behaviour for the create method.

Resources

Extending JSONAPI:Resource via BaseResource, these form the bulk of the implementation for how a resource behaves. There are a number of keywords the gem provides to be able to define aspects of the resource that fall broadly into two categories. More specific detail about these can be found in the documentation on resources.

How a resource behaves

• immutable specified anywhere in the resource class indicates that the resource can only be fetched and there is no way to create, update or delete the resource via the API.
• model_hint allows you to specify which model name for a resource. This can be useful when handling polymorphism as it lets you to coerce the gem into declaring the "type" for the resource which may not match the underlying model. This is used for declaring the type for Purpose, Plate, Tube and Request sub-classes, as seen here. This allows their Rails model to be associated with the given resource type.
• filter allows you to define which filters can be used by the API. An example of these can be found in TransferTemplateResource where the block applies the filter to the resource. As shown in the YARD documentation there, the API call to apply the filter passes a query parameter, in this case ?filter[uuid]=the-uuid-to-filter-by. Note that the response will still be an array of resources, even if only one matches.

What a resource provides (fields)

Commonly referred to as fields by the library, these can be defined as follows. Some specific details about these are found in later parts of this document as well.

• attribute defines a simple value type on the underlying record model that will be (de)serialised from/into JSON. Unless specifying a delegate for the attribute, the name of the attribute defines which value on the model to return.
• has_one defines a related resource for the given key. In this case, a singular resource. This could, for example, be a tube_rack for a TubeResource. There is no concept of belongs_to in the API, so you always use has_one.
  • Note that the resource being linked to should have its own Resource class defined and that the name of the resource is inferred from the type in the underlying model relationship. Although not strictly needed for the TubeResource to provide a JSON:API response, the TubeRackResource wouldn't need a controller or a route setting up for it, but if these are not provided, attempts to retrieve the resource directly will fail, so they should be provided.
• has_many defines an array of related resources, such as tubes on a TubeRackResource. The same rules apply as for the has_one relationships above.

For the has_one and has_many relationships above, it might be that you'd like to define the class name for the resource. This might be necessary if you're dealing with polymorphism, but shouldn't be necessary in many cases. If this is the case, the class_name parameter can be passed with a string indicating the record's model name in it's singular form.

It's worth noting that, as a user of the API, requesting a resource without using the include query parameter, only provides links to the related resources without including any of their own field data. More about the use of include can be found in the JSON:API format documentation.

For all of the field declarations, you might want to control how the value is obtained from the model or updated on the model. This is done by defining a getter and/or setter method on the field. Where these are not provided, a default implementation is provided by the library. An example of providing custom methods can be seen on PlatePurposeResource where the asset shape is an attribute on the resource with the name of the shape and gets converted to and from an actual AssetShape record on the model. Although not demonstrated in that resource, it's worth noting that a lot of the time, using UUIDs and names to identify actual records is not the correct design choice for the API, and it's better to use the built in mechanisms of JSON:API to identify related records by their ID with a proper has_one or has_many relationship. You can combine these with the attributes where legacy support is required, like where Limber requires UUIDs of related records to be passed to resources.

Operation Processors

Extending JSONAPI:Processor, these classes are optional for a resource, but allow pre-parsing of requests before they get processed by the core library functions. Overriding these is not yet a documented feature, and is instead listed as "TBD" in the documentation. However doing so is necessary in some edge cases. This not only allows you to manipulate specific types of operation on requests before they are executed, but also lets you generate useful errors for the user of the API where their request is invalid in a nuanced way. Examples of this manipulation and why it was needed are as follows.

• Transfers are created from TransferTemplates and cannot be created otherwise. The TransferProcessor in the same file as the TransfersController checks that a UUID for a template has been given and finds the template record. It contains the specific type of Transfer to be created and adds its transfers to the attributes of the request. These are used by the resource to create the correct model record object and ensures the transfers are populated on it.
• TagLayouts can be created by providing all the required fields, while optionally also including a TagLayoutTemplate. If the template UUID has been provided, the TagLayoutProcessor uses it to fill in empty fields on the TagLayout with those found on the template. It generates descriptive errors if the fields are specified in both the template and also in the request being made to create the new TagLayout resource.
• QcFiles use a library to store the file contents, and that library expects the file to exist on disk with information about the file's location and stored filename to be in the record at the time create is called on it. Therefore, the QcFileProcessor investigates the request's attributes to find file contents and a filename. These are used to generate a Tempfile and to ensure the data about its location and the intended filename are included in the request's attributes.

Magic Naming

The following are examples of the definitions for JSONAPI::Resources sub-classes you might want to create for a couple of resources named Comment and Person:

Type	Names	Comments
Controller	CommentsController / PeopleController	Plural name.
Processor	CommentProcessor / PersonProcessor	Singular name.
Resource	CommentResource / PersonResource	Singular name.

Plural names are sometimes derived automatically by Rails, but special cases like Labware pluralising to Labware must be declared manually in config/initializers/inflections.rb. Person pluralises to people, and this would also need to be declared if this were a model we were using in Sequencescape.

When setting up the routing for JSON:API resources, use the jsonapi_resources keyword in the routes.rb file, specifying the name of the resource as a symbol in the plural form, like was used for the controller above.

Permissions

These are definable at the resource/endpoint level as well as at the field level. The way these are implemented are quite different however.

Permissions at the Resource Level

immutable on a resource does what it sounds like. It does this by not creating POST, PATCH or DELETE method handlers for the resource during creation of endpoints via the jsonapi_resources entry in the routes for Sequencescape.

There are times where we want to remove only certain actions on resources, such as limiting a resource to creation and not allowing updating, which is quite common. To do this, jsonapi_resources also supports removing particular methods via parameters such as exccept: %i[update] in the routes declarations. This stops the PATCH method from being created.

Other options are as usual in Rails, including %i[create destroy index show update]. These are the methods on the resource's JSONAPI:ResourceController sub-class which perform these operations. As many of the controller's state, we do not need to implement these methods as they are done so by the library. It is very unusual to have to override these.

Permissions at the Field Level

When requests are handled by their controller class, they use the equivalently named Resource class to identify what to accept in a DELETE, GET, PATCH or POST request and what to return from those. What should be acceptable is defined by self.creatable_fields(context), self.updatable_fields(context) and fetchable_fields methods for POST, PATCH and GET requests, respectively.

BaseResource provides parameters which can be applied to fields: readonly, write_once or writeonly. These populate the above methods for you automatically and are documented in that class. For convenience, their meanings are as follows.

Property	Creatable	Updatable	Readable	Comment
default	✅	✅	✅	No permission parameters assigned to the field.
readonly: true	❌	❌	✅	e.g. fields that are autogenerated from other data.
writeonly: true	✅	✅	❌	e.g. fields used to create or update a resource, but not persisted.
write_once: true	✅	❌	✅	e.g. fields that need to be applied during creation, but do not make sense to update later.

With hindsight, it might have been cleaner to create parameters called creatable, updatable and readable so that all 8 combinations could be accounted for, but there was a pre-existing pattern in place. If we switched to this suggestion, the defaults for these parameters would be true where it could be changed to false by the inclusion of the parameter.

Marking Relationships with always_include_linkage_data: true

Some relationships have the parameter always_include_linkage_data: true which changes the way the response is formatted when you haven’t explicitly used the include query parameters for related resources. Instead of only being given URLs to the related resources, a small snippet is given which includes the type of the resources and their IDs, like { "data" { "type": "tubes", "id": 12 } }. This can be useful to get the ids of these resources without having to make additional requests or pull back all the metadata for the resources, but unfortunately the gem used by Limber does not support this response well. It uses the small snippet to generate an instance of the resource without any properties or relationships associated with it.

In the case that always_include_linkage_data is enabled for a relationship, the client must explicitly ask for the resources to be included in order for the full set of field data to be accessible. When the relationships are not explicitly included, the ID for the related resource is accessible on the by accessing relationships.{related_name}.dig(:data, :id). The benefits of optimising efficiency by using always_include_linkage_data feel like they're vastly outweighed by the cognitive overhead of trying to debug the resulting responses, so should be avoided.

Custom Methods for Resources

Some of the endpoints on v1 supported things like calling a preview method on bait_library_layout. The library in Sequencescape doesn’t provide simple support for such methods using JSON:API compliant request handling and response rendering. But you can use standard Rails nested endpoints to create methods beneath the JSON:API endpoints.

Define the Route for the Custom Method

• Create a block for the parent jsonapi_resources in the routes.rb file and use either a collection or member block to define endpoints that either apply to all the resources of that type or for specific resource records.
• Specify the type of method you want to receive (e.g. post or get) and give the name of the method as a symbol.

An example of this setup is shown for BaitLibraryLayoutResource here.

Implement the Method on the Controller

• Generate a method on the controller that handles that resource type and implement it like any controller method might be implemented, finally calling render json: hash_or_array, status: ok or handling any errors encountered.
• Unfortunately, the JSONAPI::Resources gem does not help with handling these requests.
• However, fortunately, the JSON API Client gem does accept the response if it is formatted to look like JSON:API and makes the result easy to work with.

An example of a controller implementing a custom method is on the BaitLibraryLayoutsController.

Documentation

Documentation of the API is being done via YARD documentation. These should be very detailed for the Resource classes as they will form the main touchpoint for users of the API. As such, the standard set of information for the resource should be:

• Class documentation which includes:
  • The purpose of the resource and any nuances about it.
  • Notes about the available endpoints for the resource.
  • @examples of requests you can make to do things with the resource.
  • A reference to the JSON:API format documentation and the JSONAPI::Resources package.
• Documentation of each attribute.
  • Defining the access permissions for the attribute in square brackets.
  • The return type of the attribute.
  • A description of the attribute and any nuances about it. The headline should appear as a description of the return value.
  • An @note if the attribute is required when creating a resource.
  • An @see and an @deprecated if the attribute is also accessible as a relationship.
• Documentation of each has_one and has_many.
  • Defining the access permissions for the relationship in square brackets.
  • The return type of the relationship, as the Resource class.
  • A description of the attribute any any nuances about it. The headline should appear as a description of the return value.
  • An @note about the behaviour if this relationship is also specified via an attribute in the same request.
• Documentation of filters
  • Documenting it as a method called x_filter where x is the name of the filter.
  • A description of the field being used to filter.
  • An @example of the filter being applied in a request's URL.

A well documented example of a resource is the QcFileResource which contains documentation for the resource, attributes, relationships and filters.

Testing

Tests for resources are broken up into two main categories: resources and requests.

Tests for Resources

These are found in ./spec/resources/api/v2/ and test the properties of the resource by first creating a stub of the resource using a factory and then using a variety of method to check that resource itself and all of its fields and filter declarations.

• A test of the Rails model name assigned to the resource; e.g. it { is_expected.to have_model_name 'Tube' }.
• A series of tests around the attributes on the resource; e.g. it { is_expected.to have_readonly_attribute :uuid }. Available methods include:
  • have_readonly_attribute
  • have_readwrite_attribute
  • have_writeonly_attribute
  • have_write_once_attribute
• A series of tests around relationships on the resource; e.g. it { is_expected.to have_a_write_once_has_one(:labware).with_class_name('Labware') }. Available methods include:
  • have_a_readonly_has_one
  • have_a_writable_has_one — for when it can be written to and updated.
  • have_a_write_once_has_one — for when it can only be written to on creation of the resource.
  • have_a_readonly_has_many
  • have_a_writable_has_many — for when it can be written to and updated.
  • have_a_write_once_has_many — for when it can only be written to on creation of the resource.
• A series of tests declaring that the filters have been defined; e.g. it { is_expected.to filter(:uuid) }

The above methods are defined as resource matchers in ./spec/support/api_v2_resource_matchers.rb if they need to be modified in future.

Tests for Requests

These are found in ./spec/requests/api/v2/ and test how the resource is handled as different types of request come in. These can get quite complicated depending on how the resource should behave. There are shared examples available for some of the tests. At a minimum, they should test the following.

• Given a number of resources in the databases:
  • They are all returned when accessing the endpoint without a specific ID.
  • All filters for the resource will return the correct list of resources.
• Selecting a single resource by ID:
  • The response contains all the expected fields.
  • The response does not contain write-only fields.
  • The related resources are not included in a default request.
  • When including related resources, data is bundled in the response (see shared examples).
• Patching an existing resource, either:
  • Fails to find the endpoint if we've turned off updating via immutable or by removing update from the available routes; or
  • Updates the resource correctly and returns the updated values.
  • Handles malformed payloads.
• Deleting an existing resource:
  • Fails to find the endpoint, except in carefully selected cases which there are none at this time.
• Creating a new resource, either:
  • Fails to find the endpoint for an immutable resource; or
  • Creates the resource with the given values.
  • Responds with the data for the created resource.
  • Handles malformed payloads.
  • Any other behaviour specific to this resource, such as providing a relationship and an attribute for the same model property.

JSON API Client Gem

Defining Attributes and Relationships

JSON API Client requires you create a model object for endpoints you wish to call from Sequencescape. Attributes do not have to be listed in the model, but you can use the property method to define them if you wish. Relationships will not work correctly unless you specifying has_one, belongs_to, has_many, etc.

Specifying includes for a Request

An example of specifying the includes for a Plate are captured in the following request Plate.includes('wells.aliquots.request.submission,wells.aliquots.request.request_type').find(uuid:).first.

Calling Custom Methods

Supporting custom methods on the server is super simple. First declare the endpoint in the resource's JsonApiClient::Resource sub-class.

class Sequencescape::Api::V2::BaitLibraryLayout < Sequencescape::Api::V2::Base
  custom_endpoint :preview, on: :collection, request_method: :post
end

This can then be called using syntax such as:

Sequencescape::Api::V2::BaitLibraryLayout
    .preview(plate_uuid: plate_uuid, user_uuid: user_uuid)
    .first

This calls the endpoint with a JSON:API compliant request body and the result will be coerced into resource objects like any other JSON:API call.