Skip to content
This repository was archived by the owner on Sep 16, 2024. It is now read-only.

feat(sortKey): allow sortKey or blockHeight to be provided via qu… #76

Merged
merged 21 commits into from
Dec 5, 2023
Merged

feat(sortKey): allow sortKey or blockHeight to be provided via qu… #76

Show file tree
Hide file tree
Changes from 15 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
826f29a
feat(sortKey): allow `sortKey` or `blockHeight` to be provided via qu…
Nov 28, 2023
08c691c
fix(interactions): add support for maxBlockheight on interactions end…
Nov 28, 2023
b9875c6
chore(tests): add integration tests for blockHeight and sortKeys
Nov 28, 2023
44fa04b
chore(tests): update docker compose env variables
Nov 28, 2023
fc3759e
fix(gql): fix query param to allow 0 to be provided as blockHeight
Nov 28, 2023
b504466
fix(query): separate sort key and block height into separate state at…
Nov 28, 2023
305a9e6
fix(grapqhl): update graphql to use sorter from warp
Nov 30, 2023
1da0d01
fix(query): separate sort key and block height into separate state at…
Nov 28, 2023
f621eeb
fix(gql): sort interactions in descending sortKey order
Nov 30, 2023
97169f6
fix(interactions): allow sort keys on interactions endpoint
Nov 30, 2023
754d20b
chore(sortKey): add sortKey to every contract evaluation endpoint, up…
Nov 30, 2023
a454ff4
chore(tests): add sortKey checks in all evaluation tests
Nov 30, 2023
5382946
chore(tests): add more tests to validate use of `sortKey` and `blockH…
Nov 30, 2023
9d0126b
chore(tests): fix record test
Nov 30, 2023
062314b
chore: remove unintended console statement
Nov 30, 2023
6e458d1
fix(sortKey): move the sortKey filter out of GQL and do it after all …
Nov 30, 2023
8dcbd08
chore(swagger): update swagger docs
Nov 30, 2023
cf27d27
chore(swagger): add components for response types
Nov 30, 2023
26ee31b
feat(sortKey): add regex to validate sortKeys, add tests
Nov 30, 2023
98d2433
fix(recs): make sortKey non-optional, update variable references to i…
Dec 4, 2023
a0889df
fix(warp): remove A-F in regex for sort keys
Dec 4, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docker-compose.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,9 +29,9 @@ services:
environment:
HOST: arns-service
PORT: 3000
GATEWAY_HOST: arlocal
GATEWAY_PORT: 1984
GATEWAY_PROTOCOL: http
GATEWAY_HOST: ${GATEWAY_HOST:-arlocal}
GATEWAY_PORT: ${GATEWAY_PORT:-1984}
GATEWAY_PROTOCOL: ${GATEWAY_PROTOCOL:-http}
depends_on:
- arlocal
- arns-service
119 changes: 108 additions & 11 deletions docs/openapi.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,7 @@ openapi: 3.0.0
info:
title: ArNS Microservice
version: 0.1.0
description: A koa microservice that provides API interface for fetching and retrieving ArNS related Smartweave contracts.

description: A koa microservice that provides API interface for fetching and retrieving ArNS related Smartweave contracts.
servers:
- url: '/v1'

Expand Down Expand Up @@ -39,29 +38,21 @@ components:
properties:
sourceType:
type: string
enum: ['arweave', 'otherSource']
description: The source type of the evaluation.
enum: ['arweave', 'other']
internalWrites:
type: boolean
description: Flag to enable or disable internal writes.
useKVStorage:
type: boolean
description: Indicates if key-value storage is used.
remoteStateSyncEnabled:
type: boolean
description: Flag for enabling remote state synchronization.
waitForConfirmation:
type: boolean
description: Whether to wait for confirmation before proceeding.
updateCacheForEachInteraction:
type: boolean
description: Specifies if the cache should be updated for each interaction.
maxInteractionEvaluationTimeSeconds:
type: integer
description: The maximum time in seconds for interaction evaluation.
throwOnInternalWriteError:
type: boolean
description: Indicates if an error should be thrown on internal write failure.

ContractInteraction:
type: array
Expand Down Expand Up @@ -93,6 +84,18 @@ paths:
schema:
type: string
example: 'bLAgYxAdX2Ry-nt6aH2ixgvJXbpsEYm28NgJgyqfs-U'
- in: query
name: blockHeight
required: false
description: Evaluate the contract up to a specific block height. Only applicable if sortKey is not provided.
schema:
type: number
- in: query
name: sortKey
required: false
description: Evaluate the contract at up to a specific sort key. Only applicable if blockHeight is not provided.
schema:
type: string
responses:
'200':
description: OK
Expand All @@ -111,6 +114,8 @@ paths:
example: '000001301946,0000000000000,d2efe5278648460ed160e1d8a28fb86ab686e36cf14a3321d0a2b10c6851ea99'
evaluationOptions:
$ref: '#/components/schemas/EvaluationOptions'
'400':
description: Bad request if query parameters are invalid.
'404':
description: Contract not found.
'503':
Expand Down Expand Up @@ -250,6 +255,18 @@ paths:
schema:
type: string
example: 'owner'
- in: query
name: blockHeight
required: false
description: Evaluate the contract up to a specific block height. Only applicable if sortKey is not provided.
schema:
type: number
- in: query
name: sortKey
required: false
description: Evaluate the contract at up to a specific sort key. Only applicable if blockHeight is not provided.
schema:
type: string
responses:
'200':
description: OK
Expand All @@ -264,6 +281,9 @@ paths:
field:
type: any
example: {}
sortKey:
type: string,
example: '000001301946,0000000000000,d2efe5278648460ed160e1d8a28fb86ab686e36cf14a3321d0a2b10c6851ea99'
evaluationOptions:
$ref: '#/components/schemas/EvaluationOptions'
'404':
Expand All @@ -289,6 +309,18 @@ paths:
schema:
type: string
example: 'gh673M0Koh941OIITVXl9hKabRaYWABQUedZxW-swIA'
- in: query
name: blockHeight
required: false
description: Evaluate the contract up to a specific block height. Only applicable if sortKey is not provided.
schema:
type: number
- in: query
name: sortKey
required: false
description: Evaluate the contract at up to a specific sort key. Only applicable if blockHeight is not provided.
schema:
type: string
responses:
'200':
description: OK
Expand Down Expand Up @@ -321,6 +353,9 @@ paths:
'endTimestamp': 1714581732,
},
]
sortKey:
type: string,
example: '000001301946,0000000000000,d2efe5278648460ed160e1d8a28fb86ab686e36cf14a3321d0a2b10c6851ea99'
evaluationOptions:
$ref: '#/components/schemas/EvaluationOptions'
'404':
Expand All @@ -346,6 +381,18 @@ paths:
schema:
type: string
example: 'ario'
- in: query
name: blockHeight
required: false
description: Evaluate the contract up to a specific block height. Only applicable if sortKey is not provided.
schema:
type: number
- in: query
name: sortKey
required: false
description: Evaluate the contract at up to a specific sort key. Only applicable if blockHeight is not provided.
schema:
type: string
responses:
'200':
description: OK
Expand Down Expand Up @@ -381,6 +428,9 @@ paths:
owner:
type: string
example: 'QGWqtJdLLgm2ehFWiiPzMaoFLD50CnGuzZIPEdoDRGQ'
sortKey:
type: string
example: '000001301946,0000000000000,d2efe5278648460ed160e1d8a28fb86ab686e36cf14a3321d0a2b10c6851ea99'
evaluationOptions:
$ref: '#/components/schemas/EvaluationOptions'
'404':
Expand All @@ -406,6 +456,18 @@ paths:
schema:
type: string
example: 'QGWqtJdLLgm2ehFWiiPzMaoFLD50CnGuzZIPEdoDRGQ'
- in: query
name: blockHeight
required: false
description: Evaluate the contract up to a specific block height. Only applicable if sortKey is not provided.
schema:
type: number
- in: query
name: sortKey
required: false
description: Evaluate the contract at up to a specific sort key. Only applicable if blockHeight is not provided.
schema:
type: string
responses:
'200':
description: OK
Expand All @@ -423,6 +485,9 @@ paths:
balance:
type: number
example: 994963650
sortKey:
type: string
example: '000001301946,0000000000000,d2efe5278648460ed160e1d8a28fb86ab686e36cf14a3321d0a2b10c6851ea99'
evaluationOptions:
$ref: '#/components/schemas/EvaluationOptions'
'404':
Expand All @@ -442,6 +507,18 @@ paths:
schema:
type: string
example: 'bLAgYxAdX2Ry-nt6aH2ixgvJXbpsEYm28NgJgyqfs-U'
- in: query
name: blockHeight
required: false
description: Evaluate the contract up to a specific block height. Only applicable if sortKey is not provided.
schema:
type: number
- in: query
name: sortKey
required: false
description: Evaluate the contract at up to a specific sort key. Only applicable if blockHeight is not provided.
schema:
type: string
responses:
'200':
description: OK
Expand All @@ -455,8 +532,13 @@ paths:
example: 'bLAgYxAdX2Ry-nt6aH2ixgvJXbpsEYm28NgJgyqfs-U'
interactions:
$ref: '#/components/schemas/ContractInteraction'
sortKey:
type: string
example: '000001301946,0000000000000,d2efe5278648460ed160e1d8a28fb86ab686e36cf14a3321d0a2b10c6851ea99'
evaluationOptions:
$ref: '#/components/schemas/EvaluationOptions'
'400':
description: Bad request if query parameters are invalid.
'404':
description: Contract not found.
'503':
Expand All @@ -480,6 +562,18 @@ paths:
schema:
type: string
example: 'QGWqtJdLLgm2ehFWiiPzMaoFLD50CnGuzZIPEdoDRGQ'
- in: query
name: blockHeight
required: false
description: Evaluate the contract up to a specific block height. Only applicable if sortKey is not provided.
schema:
type: number
- in: query
name: sortKey
required: false
description: Evaluate the contract at up to a specific sort key. Only applicable if blockHeight is not provided.
schema:
type: string
responses:
'200':
description: OK
Expand Down Expand Up @@ -510,6 +604,9 @@ paths:
'id': '2wszuZi_rwoOFjowdH7GLbgdeIZBaGbMLXiOuIV-6_0',
},
]
sortKey:
type: string
example: '000001301946,0000000000000,d2efe5278648460ed160e1d8a28fb86ab686e36cf14a3321d0a2b10c6851ea99'
evaluationOptions:
$ref: '#/components/schemas/EvaluationOptions'
'404':
Expand Down
72 changes: 48 additions & 24 deletions src/api/graphql.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,11 @@
*/
import Arweave from 'arweave';
import { ArNSInteraction } from '../types.js';
import { LexicographicalInteractionsSorter, TagsParser } from 'warp-contracts';
import {
GQLResultInterface,
LexicographicalInteractionsSorter,
TagsParser,
} from 'warp-contracts';
import logger from '../logger';

export const MAX_REQUEST_SIZE = 100;
Expand Down Expand Up @@ -101,7 +105,12 @@ export async function getDeployedContractsByWallet(

export async function getWalletInteractionsForContract(
arweave: Arweave,
params: { address?: string; contractTxId: string },
params: {
address?: string | undefined;
contractTxId: string;
sortKey: string | undefined;
blockHeight: number | undefined;
},
): Promise<{
interactions: Map<
string,
Expand All @@ -110,7 +119,12 @@ export async function getWalletInteractionsForContract(
}> {
const parser = new TagsParser();
const interactionSorter = new LexicographicalInteractionsSorter(arweave);
const { address, contractTxId } = params;
const {
address,
contractTxId,
blockHeight: blockHeightFilter,
sortKey: sortKeyFilter,
} = params;
let hasNextPage = false;
let cursor: string | undefined;
const interactions = new Map<
Expand All @@ -130,7 +144,10 @@ export async function getWalletInteractionsForContract(
values:["${contractTxId}"]
}
],
sort: HEIGHT_DESC,
block: {
min: 0,
max: ${blockHeightFilter ?? null}
},
first: ${MAX_REQUEST_SIZE},
bundledIn: null,
${cursor ? `after: "${cursor}"` : ''}
Expand Down Expand Up @@ -160,27 +177,35 @@ export async function getWalletInteractionsForContract(
}`,
};

const { status, ...response } = await arweave.api.post(
const { status, data } = await arweave.api.post<GQLResultInterface>(
'/graphql',
queryObject,
);

if (status !== 200) {
throw Error(
`Failed to fetch contracts for wallet. Status code: ${status}`,
);
}

if (!response.data.data?.transactions?.edges?.length) {
if (!data.data.transactions?.edges?.length) {
continue;
}
for (const e of response.data.data.transactions.edges) {

// remove interactions without block data
const validInteractions = data.data.transactions.edges.filter(
(i) => i.node.block && i.node.block.height && i.node.block.id,
);
// sort them using warps sort logic and adds sort keys
const sortedInteractions = await interactionSorter.sort(validInteractions);
for (const i of sortedInteractions) {
// basic validation for smartweave tags
const inputTag = parser.getInputTag(e.node, contractTxId);
const contractTag = parser.getContractTag(e.node);
const inputTag = parser.getInputTag(i.node, contractTxId);
const contractTag = parser.getContractTag(i.node);
if (!inputTag || !contractTag) {
logger.debug('Invalid tags for interaction via GQL, ignoring...', {
contractTxId,
interactionId: e.node.id,
interactionId: i.node.id,
inputTag,
contractTag,
});
Expand All @@ -189,25 +214,24 @@ export async function getWalletInteractionsForContract(
const parsedInput = inputTag?.value
? JSON.parse(inputTag.value)
: undefined;
const sortKey = await interactionSorter.createSortKey(
e.node.block.id,
e.node.id,
e.node.block.height,
);
interactions.set(e.node.id, {
height: e.node.block.height,
timestamp: e.node.block.timestamp,
sortKey,
interactions.set(i.node.id, {
height: i.node.block.height,
timestamp: i.node.block.timestamp,
input: parsedInput,
owner: e.node.owner.address,
owner: i.node.owner.address,
sortKey: i.node.sortKey,
});

// if we have a sort key filter, we can stop here
if (i.node.sortKey === sortKeyFilter) {
break;
}
}
cursor =
response.data.data.transactions.edges[MAX_REQUEST_SIZE - 1]?.cursor ??
undefined;
hasNextPage =
response.data.data.transactions.pageInfo?.hasNextPage ?? false;
data.data.transactions.edges[MAX_REQUEST_SIZE - 1]?.cursor ?? undefined;
hasNextPage = data.data.transactions.pageInfo?.hasNextPage ?? false;
} while (hasNextPage);

return {
interactions,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something to think about - this list will eventually become untenably large and we may have to start thinking about how to handle these types of requests in a streaming fashion (e.g. some combination of generator functions, streamed responses, and pages of data at a time).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we have a ticket to add pagination, but there is more work to do for stream responses/generators - will create a ticket

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pagination is great start. There isn't yet a clear use case for a streaming version and the complexity would be way higher. You're on the right track already.

};
Expand Down
Loading