Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add and use Accounts API for Account balance calls #4781

Merged
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
7cee69b
feat: add multichain accounts API services
Prithpal-Sooriya Oct 10, 2024
6e4171a
docs: scaffold and comment sections of TokenDetectionController
Prithpal-Sooriya Oct 10, 2024
b52827e
Merge branch 'main' of github.com:MetaMask/core into NOTIFY-1179/add-…
Prithpal-Sooriya Oct 11, 2024
d201250
feat: add accounts API support networks endpoint
Prithpal-Sooriya Oct 11, 2024
816ca78
refactor: rename fetchSupportedNetworks function
Prithpal-Sooriya Oct 11, 2024
d4c37ef
feat: add token detection via API method and connect to detection con…
Prithpal-Sooriya Oct 11, 2024
9f5724d
refactor: add barrel file
Prithpal-Sooriya Oct 11, 2024
9adebed
test: add test coverage for tokenDetectionController
Prithpal-Sooriya Oct 11, 2024
4e08cba
Merge branch 'main' of github.com:MetaMask/core into NOTIFY-1179/add-…
Prithpal-Sooriya Oct 14, 2024
0911070
docs: update test util jsdoc comment
Prithpal-Sooriya Oct 14, 2024
8664211
docs: remove todo/note comments
Prithpal-Sooriya Oct 14, 2024
228515c
test: add ignore comments for untestable statements
Prithpal-Sooriya Oct 14, 2024
200beb7
Merge branch 'main' of github.com:MetaMask/core into NOTIFY-1179/add-…
Prithpal-Sooriya Oct 15, 2024
4a464ab
Merge branch 'main' into NOTIFY-1179/add-account-api-to-token-detecti…
Prithpal-Sooriya Oct 15, 2024
87c4155
Merge branch 'main' of github.com:MetaMask/core into NOTIFY-1179/add-…
Prithpal-Sooriya Oct 16, 2024
cc53352
refactor: use existing enums and constants
Prithpal-Sooriya Oct 16, 2024
7157e10
Merge branch 'NOTIFY-1179/add-account-api-to-token-detection-controll…
Prithpal-Sooriya Oct 16, 2024
9bd0f70
Merge branch 'main' into NOTIFY-1179/add-account-api-to-token-detecti…
Prithpal-Sooriya Oct 16, 2024
56e242c
Merge branch 'main' into NOTIFY-1179/add-account-api-to-token-detecti…
Prithpal-Sooriya Oct 17, 2024
909fde8
Merge branch 'main' into NOTIFY-1179/add-account-api-to-token-detecti…
Prithpal-Sooriya Oct 18, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
160 changes: 158 additions & 2 deletions packages/assets-controllers/src/TokenDetectionController.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,14 @@ import type {
PreferencesControllerStateChangeEvent,
} from '@metamask/preferences-controller';
import type { Hex } from '@metamask/utils';
import { hexToNumber } from '@metamask/utils';

import type { AssetsContractController } from './AssetsContractController';
import { isTokenDetectionSupportedForNetwork } from './assetsUtil';
import {
fetchMultiChainBalances,
fetchSupportedNetworks,
} from './multi-chain-accounts-service/multi-chain-accounts';
import type {
GetTokenListState,
TokenListMap,
Expand Down Expand Up @@ -191,6 +196,46 @@ export class TokenDetectionController extends StaticIntervalPollingController<To
};
}) => void;

#accountsAPI = {
isAccountsAPIEnabled: true,
supportedNetworksCache: null as number[] | null,
async getSupportedNetworks() {
if (!this.isAccountsAPIEnabled) {
throw new Error('Accounts API Feature Switch is disabled');
}

if (this.supportedNetworksCache) {
return this.supportedNetworksCache;
}

const result = await fetchSupportedNetworks().catch(() => null);
this.supportedNetworksCache = result;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the cache and dynamic lookup of support. Since its in memory rather than state, it'll re run when extension is reloaded or browser close/open. But not each time you open the popup. Which seems like a reasonable interval.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep only on browser close/open. We can think of more sophisticating caches if need be... later though 😄

return result;
},

async getMultiChainBalances(address: string, chainId: Hex) {
if (!this.isAccountsAPIEnabled) {
throw new Error('Accounts API Feature Switch is disabled');
}

const chainIdNumber = hexToNumber(chainId);
const supportedNetworks = await this.getSupportedNetworks();

if (!supportedNetworks || !supportedNetworks.includes(chainIdNumber)) {
const supportedNetworksErrStr = (supportedNetworks ?? []).toString();
throw new Error(
`Unsupported Network: supported networks ${supportedNetworksErrStr}, network: ${chainIdNumber}`,
);
}

const result = await fetchMultiChainBalances(address, {
networks: [chainIdNumber],
});

return result.balances;
},
};

/**
* Creates a TokenDetectionController instance.
*
Expand All @@ -200,13 +245,15 @@ export class TokenDetectionController extends StaticIntervalPollingController<To
* @param options.interval - Polling interval used to fetch new token rates
* @param options.getBalancesInSingleCall - Gets the balances of a list of tokens for the given address.
* @param options.trackMetaMetricsEvent - Sets options for MetaMetrics event tracking.
* @param options.useAccountsAPI - Feature Switch for using the accounts API when detecting tokens (default: true)
*/
constructor({
interval = DEFAULT_INTERVAL,
disabled = true,
getBalancesInSingleCall,
trackMetaMetricsEvent,
messenger,
useAccountsAPI = true,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Big proponent of feature switching (in case this PR gets too large, or we need to turn this feature off).

Happy to remove otherwise.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is enabled by default

}: {
interval?: number;
disabled?: boolean;
Expand All @@ -225,6 +272,7 @@ export class TokenDetectionController extends StaticIntervalPollingController<To
};
}) => void;
messenger: TokenDetectionControllerMessenger;
useAccountsAPI?: boolean;
}) {
super({
name: controllerName,
Expand Down Expand Up @@ -257,6 +305,8 @@ export class TokenDetectionController extends StaticIntervalPollingController<To
);
this.#isUnlocked = isUnlocked;

this.#accountsAPI.isAccountsAPIEnabled = useAccountsAPI;

this.#registerEventListeners();
}

Expand Down Expand Up @@ -518,10 +568,23 @@ export class TokenDetectionController extends StaticIntervalPollingController<To
? STATIC_MAINNET_TOKEN_LIST
: tokensChainsCache[chainIdAgainstWhichToDetect]?.data ?? {};

const tokenDetectionPromises = this.#getSlicesOfTokensToDetect({
const tokenCandidateSlices = this.#getSlicesOfTokensToDetect({
chainId: chainIdAgainstWhichToDetect,
selectedAddress: addressAgainstWhichToDetect,
});

// Attempt Accounts API Detection
const accountAPIResult = await this.#addDetectedTokensViaAPI({
chainId: chainIdAgainstWhichToDetect,
selectedAddress: addressAgainstWhichToDetect,
}).map((tokensSlice) =>
tokenCandidateSlices,
});
if (accountAPIResult?.result === 'success') {
return;
}

// Attempt RPC Detection
const tokenDetectionPromises = tokenCandidateSlices.map((tokensSlice) =>
this.#addDetectedTokens({
tokensSlice,
selectedAddress: addressAgainstWhichToDetect,
Expand Down Expand Up @@ -578,6 +641,99 @@ export class TokenDetectionController extends StaticIntervalPollingController<To
return slicesOfTokensToDetect;
}

/**
* This adds detected tokens from the Accounts API, avoiding the multi-call RPC calls for balances
* @param options - method arguments
* @param options.selectedAddress - address to check against
* @param options.chainId - chainId to check tokens for
* @param options.tokenCandidateSlices - these are tokens we know a user does not have (by checking the tokens controller).
* We will use these these token candidates to determine if a token found from the API is valid to be added on the users wallet.
* It will also prevent us to adding tokens a user already has
* @returns a success or failed object
*/
async #addDetectedTokensViaAPI({
Prithpal-Sooriya marked this conversation as resolved.
Show resolved Hide resolved
selectedAddress,
chainId,
tokenCandidateSlices,
}: {
selectedAddress: string;
chainId: Hex;
tokenCandidateSlices: string[][];
}) {
return await safelyExecute(async () => {
const tokenBalances = await this.#accountsAPI
.getMultiChainBalances(selectedAddress, chainId)
.catch(() => null);

if (!tokenBalances || tokenBalances.length === 0) {
return { result: 'failed' } as const;
}

const tokensWithBalance: Token[] = [];
const eventTokensDetails: string[] = [];

const tokenCandidateSet = new Set<string>(tokenCandidateSlices.flat());

tokenBalances.forEach((token) => {
const tokenAddress = token.address;

// Make sure that the token to add is in our candidate list
// Ensures we don't add tokens we already own
if (!tokenCandidateSet.has(token.address)) {
return;
}

// We need specific data from tokenList to correctly create a token
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are the fields we get from the token list the icon url and aggregators?

We'll eventually stop storing icon urls in state since its always calculable as "https://static.cx.metamask.io/api/v1/tokenIcons/${chainId}/${address}.png".

Aggregators are shown in the UI but we could just query them from the big token list rather than duplicating on each imported token. So I think we keep this for now but could remove the dependency later.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep 100% agree. How expensive would it be for the accounts API to serve this additional information?

Copy link
Contributor Author

@Prithpal-Sooriya Prithpal-Sooriya Oct 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aha saw a relevant comment you've made before, great callout.
https://github.com/MetaMask/mobile-planning/issues/1935#issuecomment-2384061477

// So even if we have a token that was detected correctly by the API, if its missing data we cannot safely add it.
if (!this.#tokenList[token.address]) {
return;
}

// NOTE: some of this data is also available from the `token` balance,
// however it is missing the `aggregators` and `iconUrl` properties
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These 2 fields that are missing from the Account API are technically optional. We could leave them out.

However I do not know the unintended effects if we were to do this.

const { decimals, symbol, aggregators, iconUrl, name } =
this.#tokenList[token.address];
eventTokensDetails.push(`${symbol} - ${tokenAddress}`);
tokensWithBalance.push({
address: tokenAddress,
decimals,
symbol,
aggregators,
image: iconUrl,
isERC721: false,
name,
});
});

if (tokensWithBalance.length) {
this.#trackMetaMetricsEvent({
Prithpal-Sooriya marked this conversation as resolved.
Show resolved Hide resolved
event: 'Token Detected',
category: 'Wallet',
properties: {
tokens: eventTokensDetails,
// TODO: Either fix this lint violation or explain why it's necessary to ignore.
// eslint-disable-next-line @typescript-eslint/naming-convention
token_standard: 'ERC20',
Prithpal-Sooriya marked this conversation as resolved.
Show resolved Hide resolved
// TODO: Either fix this lint violation or explain why it's necessary to ignore.
// eslint-disable-next-line @typescript-eslint/naming-convention
asset_type: 'TOKEN',
},
});

await this.messagingSystem.call(
'TokensController:addDetectedTokens',
tokensWithBalance,
{
selectedAddress,
chainId,
},
);
}

return { result: 'success' } as const;
});
}

async #addDetectedTokens({
tokensSlice,
selectedAddress,
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
import type { GetBalancesResponse } from '../types';

export const MOCK_GET_BALANCES_RESPONSE: GetBalancesResponse = {
Prithpal-Sooriya marked this conversation as resolved.
Show resolved Hide resolved
count: 6,
balances: [
{
object: 'token',
address: '0x0000000000000000000000000000000000000000',
symbol: 'ETH',
name: 'Ether',
type: 'native',
timestamp: '2015-07-30T03:26:13.000Z',
decimals: 18,
chainId: 1,
balance: '0.026380882267770930',
},
{
object: 'token',
address: '0x4200000000000000000000000000000000000042',
name: 'Optimism',
symbol: 'OP',
decimals: 18,
balance: '5.250000000000000000',
chainId: 10,
},
{
object: 'token',
address: '0x2791bca1f2de4661ed88a30c99a7a9449aa84174',
name: 'USD Coin (PoS)',
symbol: 'USDC',
decimals: 6,
balance: '22.484688',
chainId: 137,
},
{
object: 'token',
address: '0x0000000000000000000000000000000000000000',
symbol: 'MATIC',
name: 'MATIC',
type: 'native',
timestamp: '2020-05-30T07:47:16.000Z',
decimals: 18,
chainId: 137,
balance: '2.873547261071381088',
},
{
object: 'token',
address: '0x912ce59144191c1204e64559fe8253a0e49e6548',
name: 'Arbitrum',
symbol: 'ARB',
decimals: 18,
balance: '14.640000000000000000',
chainId: 42161,
},
{
object: 'token',
address: '0xd83af4fbd77f3ab65c3b1dc4b38d7e67aecf599a',
name: 'Linea Voyage XP',
symbol: 'LXP',
decimals: 18,
balance: '100.000000000000000000',
chainId: 59144,
},
],
unprocessedNetworks: [],
};
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
import type { GetSupportedNetworksResponse } from '../types';

export const MOCK_GET_SUPPORTED_NETWORKS_RESPONSE: GetSupportedNetworksResponse =
{
fullSupport: [1, 137, 56, 59144, 8453, 10, 42161, 534352],
partialSupport: {
balances: [59141, 42220, 43114],
},
};
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
import nock from 'nock';

import { MOCK_GET_BALANCES_RESPONSE } from './mocks/mock-get-balances';
import { MOCK_GET_SUPPORTED_NETWORKS_RESPONSE } from './mocks/mock-get-supported-networks';
import {
fetchMultiChainBalances,
fetchSupportedNetworks,
} from './multi-chain-accounts';

const MOCK_ADDRESS = '0xd8dA6BF26964aF9D7eEd9e03E53415D37aA96045';

describe('fetchSupportedNetworks()', () => {
const createMockAPI = () =>
nock('https://accounts.api.cx.metamask.io').get('/v1/supportedNetworks');

it('should successfully return supported networks array', async () => {
const mockAPI = createMockAPI().reply(
200,
MOCK_GET_SUPPORTED_NETWORKS_RESPONSE,
);

const result = await fetchSupportedNetworks();
expect(result).toStrictEqual(
MOCK_GET_SUPPORTED_NETWORKS_RESPONSE.fullSupport,
);
expect(mockAPI.isDone()).toBe(true);
});

it('should throw error when fetch fails', async () => {
const mockAPI = createMockAPI().reply(500);

await expect(async () => await fetchSupportedNetworks()).rejects.toThrow(
expect.any(Error),
);
expect(mockAPI.isDone()).toBe(true);
});
});

describe('fetchMultiChainBalances()', () => {
const createMockAPI = () =>
nock('https://accounts.api.cx.metamask.io').get(
`/v2/accounts/${MOCK_ADDRESS}/balances`,
);

it('should successfully return balances response', async () => {
const mockAPI = createMockAPI().reply(200, MOCK_GET_BALANCES_RESPONSE);

const result = await fetchMultiChainBalances(MOCK_ADDRESS);
expect(result).toBeDefined();
expect(result).toStrictEqual(MOCK_GET_BALANCES_RESPONSE);
expect(mockAPI.isDone()).toBe(true);
});

it('should successfully return balances response with query params to refine search', async () => {
const mockAPI = createMockAPI()
.query({
networks: '1,10',
})
.reply(200, MOCK_GET_BALANCES_RESPONSE);

const result = await fetchMultiChainBalances(MOCK_ADDRESS, {
networks: [1, 10],
});
expect(result).toBeDefined();
expect(result).toStrictEqual(MOCK_GET_BALANCES_RESPONSE);
expect(mockAPI.isDone()).toBe(true);
});

const testMatrix = [
{ httpCode: 429, httpCodeName: 'Too Many Requests' }, // E.g. Rate Limit
{ httpCode: 422, httpCodeName: 'Unprocessable Content' }, // E.g. fails to fetch any balances from specified chains
{ httpCode: 500, httpCodeName: 'Internal Server Error' }, // E.g. Server Rekt
];

it.each(testMatrix)(
'should throw when $httpCode "$httpCodeName"',
async ({ httpCode }) => {
const mockAPI = createMockAPI().reply(httpCode);

await expect(
async () => await fetchMultiChainBalances(MOCK_ADDRESS),
).rejects.toThrow(expect.any(Error));
expect(mockAPI.isDone()).toBe(true);
},
);
});
Loading
Loading