feat(sortKey): allow sortKey or blockHeight to be provided via qu…

[dtfiedler]

…ery param

This allows us to evaluate a contract state up to a specified sortKey or blockHeight.

Block Height example:

curl http://localhost:3000/v1/contract/bLAgYxAdX2Ry-nt6aH2ixgvJXbpsEYm28NgJgyqfs-U/interactions?blockHeight=1307087

{
    "contractTxId": "bLAgYxAdX2Ry-nt6aH2ixgvJXbpsEYm28NgJgyqfs-U",
    "interactions": [
        {
            "height": 1307087,
            "timestamp": 1700594511,
            "input": {
                "function": "buyRecord",
                "name": "developer",
                "contractTxId": "atomic",
                "type": "lease",
                "years": 2,
                "auction": false
            },
            "owner": "J-x6qHeWPYuzbi2pt0HtPw5KxitNwRU2I3hdUWJ9DAo",
            "sortKey": "000001307087,0000000000000,6c094a6ae0b62b50afc204dc91c2d8663ffe61904097c09efde8cc0ca1df4e78",
            "valid": false,
            "error": "Caller balance not high enough to purchase this name for 1837.5 token(s)!",
            "id": "IHvhOecLEtgVaOFfKJW9Ig4GQzfxnPnLuzuYawNaCm0"
        },
        {
            "height": 1307087,
            "timestamp": 1700594511,
            "input": {
                "function": "buyRecord",
                "name": "deepanshu",
                "contractTxId": "atomic",
                "type": "lease",
                "years": 2,
                "auction": false
            },
            "owner": "J-x6qHeWPYuzbi2pt0HtPw5KxitNwRU2I3hdUWJ9DAo",
            "sortKey": "000001307087,0000000000000,42845e0d850895e791f498c13cb7850405d9c3a7f631af7f0ca2cef28348a8ea",
            "valid": true,
            "id": "tLUDNRR82EXEZYqCDsAdnibdWtNWFzfZ2Rhdcp0W7S8"
        }
        ...
    ],
    "sortKey": "000001307087,0000000000000,6c094a6ae0b62b50afc204dc91c2d8663ffe61904097c09efde8cc0ca1df4e78",
    "evaluationOptions": { "maxInteractionEvaluationTimeSeconds": 3600 }
}

Sort Key example:

curl http://localhost:3000/v1/contract/bLAgYxAdX2Ry-nt6aH2ixgvJXbpsEYm28NgJgyqfs-U?sortKey=000001307087,0000000000000,0cbe0824cdf7ba421c49d08ebf6cd0957d0396da8ebe78107a37baa31ed5bbe4

Result:

{
  "contractTxId": "bLAgYxAdX2Ry-nt6aH2ixgvJXbpsEYm28NgJgyqfs-U",
  "state": {
    "auctions": {
      "alacazam": {
        "contractTxId": "PP2uMO_Ig7bGCYswrhDWfaPR27qKl0GikgVJyRTS0Qs",
        "endHeight": 1317112,
        "floorPrice": 3000,
        "initiator": "7waR8v4STuwPnTck1zFVkQqJh5K9q9Zik4Y5-5dV7nk",
        "settings": {
          "auctionDuration": 10080,
          "exponentialDecayRate": 0.000002,
          "floorPriceMultiplier": 1,
          "scalingExponent": 190,
          "startPriceMultiplier": 50
        },
        "startHeight": 1307032,
        "startPrice": 150000,
        "type": "permabuy"
      },
      // other auctions
    },
    "balances": {
      "-4xgjroXENKYhTWqrBo57HQwvDL51mMvSxJy6Y2Z_sA": 5000,
      "-7vXsQZQDk8TMDlpiSLy3CnLi5PDPlAaN2DaynORpck": 5000,
      "-9JU3W8g9nOAB1OrJQ8FxkaWCpv5slBET2HppTItbmk": 5000,
      "-9ip1RdBZ64owkHXgY7dvRddgm39K_0uikAJX05YkHw": 5000,
      "-C5AiYcMmrOFHrnQwhJg1Xtzhz23T0zNB0ACge1otHs": 89.0187421874997,
      // other balances
    },
    // other state data
    "ticker": "tIO"
  },
  "sortKey": "000001307087,0000000000000,6c094a6ae0b62b50afc204dc91c2d8663ffe61904097c09efde8cc0ca1df4e78",
  "evaluationOptions": { "maxInteractionEvaluationTimeSeconds": 3600 }
}

[atticusofsparta]

new RegExp('^[0-9]{12},[0-9]{13},[0-9a-fA-F]{64}$')

[dtfiedler]

added with tests in 26ee31b

[arielmelendez]

The hexidecimal string for the last sortkey segment (last set of characters after a comma) produced by warp's LexicographicalInteractionSorter only produces lowercase hex strings so you can remove A-F from the final range set in regex.

[dtfiedler]

a0889df

[fedellen]

looking good!

[arielmelendez]

Nice. Too bad OpenAPI doesn't make it easier to specify mutual exclusivity among query parameters.

[dtfiedler]

+1

[arielmelendez]

Something to think about - this list will eventually become untenably large and we may have to start thinking about how to handle these types of requests in a streaming fashion (e.g. some combination of generator functions, streamed responses, and pages of data at a time).

[dtfiedler]

we have a ticket to add pagination, but there is more work to do for stream responses/generators - will create a ticket

[arielmelendez]

Pagination is great start. There isn't yet a clear use case for a streaming version and the complexity would be way higher. You're on the right track already.

[arielmelendez]

What's the motivation for using a hash of these inputs for the cache key rather than literal strings? What are the tradeoffs? e.g. does one provide better cache key collision protection?

[dtfiedler]

eh, probably a bit of overkill but primarily to create fixed-sized/length strings for the cache keys and it matches our pattern for hashing down our evaluation options. given these are user-provided inputs they have high cardinality and can also be undefined.

sortKey: undefined
blockHeight: undefined
resulting hash: `465039ec2625fb7eace80f9a3e9bd63dd04b0c55d971f6af1894f4b8c1b3c0ad`

vs.
undefined-undefined (without adding some special logic to handle empties/undefineds)

as for performance - that's a good call out. i'd be interested to see what type of overhead this generates with a lot of keys created/accessed but given it's such a small amount of data being hashed I don't see a ton of waste?

[arielmelendez]

That essentially confirms my assumptions. Part of me wonders if the hashing layer is worth it though given that:

1. Under the hood, PromiseCache, which is built on the simple-cache library's EphemeralCache, has at its base at Map object which is benefiting from all the niceties getting the performance tradeoffs you want from customized hash tables. So, at a base level of running the cache optimally, we're doing redundant work and it may actually be hampering some of what the base implementation is trying to optimize for.
2. We may be making debugging harder for ourselves if hash keys we want to inspect are then hashed again, losing some of their original context during the process of the redundant hashing.

I think what you have is ok to ship, but this might be something you want to revisit later.