Batch job processing

[benjie]

Description

Replaces #99.

The PR gives the ability to configure Graphile Worker such that getJob, completeJob and failJob can be batched:

• Pools are now responsible for getting jobs and will grab the number of jobs that you specify up front, and distribute these to workers on demand; each time their queue is exhausted they'll fetch another batch
• Pools are also now responsible for completing and failing jobs; they will wait the specified number of milliseconds after a completeJob or failJob call and see if any other calls are made in the interrim; all of these results will be sent to the database at the same time reducing the total number of transactions.

Note that enabling these features changes the behavior of Worker in a few ways:

• Since pools grab a number of jobs up front they represent a snapshot at that time and newer higher priority jobs will not be evaluated until the queue is exhausted
• Since pools grab a number of jobs up front jobs may not be as evenly distributed across workers
• Since pools grab a number of jobs up front, jobs may not start until a later time than they previously did, increasing latency (but also increasing throughput)

Performance impact

If not enabled, impact is minimal.

If enabled, throughput improvement at the cost of potential latency increases

Security impact

Not known.

Checklist

• My code matches the project's code style and yarn lint:fix passes.
• I've added tests for the new feature, and yarn test passes.
• I have detailed the new feature in the relevant documentation.
• I have added this feature to 'Pending' in the RELEASE_NOTES.md file (if one exists).
• If this is a breaking change I've explained why.

[benjie]

Closing in favour of #474