Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Allow the HBar Rate Limiter to be disabled. #3252

Open
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

ebadiere
Copy link
Collaborator

This fix allows the HBar Rate Limiter to be disabled by setting the HBAR_RATE_LIMIT_TINYBAR to 0

Related issue(s):

Fixes #3251

Notes for reviewer:

Checklist

  • Documented (Code comments, README, etc.)
  • Tested (unit, integration, etc.)

@ebadiere ebadiere added the bug Something isn't working label Nov 12, 2024
@ebadiere ebadiere added this to the 0.61.0 milestone Nov 12, 2024
@ebadiere ebadiere self-assigned this Nov 12, 2024
Copy link

github-actions bot commented Nov 12, 2024

Test Results

    4 files  +    3    421 suites  +414   27s ⏱️ -3s
1 493 tests +1 488  1 492 ✅ +1 488  1 💤 +1  0 ❌  - 1 
1 502 runs  +1 466  1 501 ✅ +1 466  1 💤 +1  0 ❌  - 1 

Results for commit 1938973. ± Comparison against base commit ce43b76.

This pull request removes 5 and adds 1493 tests. Note that renamed tests count towards both.
"before all" hook in "@api-batch-2 RPC Server Acceptance Tests" ‑ RPC Server Acceptance Tests Acceptance tests @api-batch-2 RPC Server Acceptance Tests "before all" hook in "@api-batch-2 RPC Server Acceptance Tests"
@release-light, @release should execute "eth_getTransactionReceipt" for hash of 2930 transaction ‑ RPC Server Acceptance Tests Acceptance tests @api-batch-1 RPC Server Acceptance Tests RPC Server Acceptance Tests Transaction related RPC Calls @release-light, @release should execute "eth_getTransactionReceipt" for hash of 2930 transaction
@release-light, @release should execute "eth_getTransactionReceipt" for hash of London transaction ‑ RPC Server Acceptance Tests Acceptance tests @api-batch-1 RPC Server Acceptance Tests RPC Server Acceptance Tests Transaction related RPC Calls @release-light, @release should execute "eth_getTransactionReceipt" for hash of London transaction
@release-light, @release should execute "eth_getTransactionReceipt" for hash of legacy transaction ‑ RPC Server Acceptance Tests Acceptance tests @api-batch-1 RPC Server Acceptance Tests RPC Server Acceptance Tests Transaction related RPC Calls @release-light, @release should execute "eth_getTransactionReceipt" for hash of legacy transaction
@release-light, @release should execute "eth_sendRawTransaction" for legacy EIP 155 transactions ‑ RPC Server Acceptance Tests Acceptance tests @api-batch-1 RPC Server Acceptance Tests RPC Server Acceptance Tests Transaction related RPC Calls @release-light, @release should execute "eth_sendRawTransaction" for legacy EIP 155 transactions
"eth_blockNumber" return the latest block number on second try ‑ @ethGetBlockByNumber using MirrorNode "eth_blockNumber" return the latest block number on second try
"eth_blockNumber" should return the latest block number using cache ‑ @ethGetBlockByNumber using MirrorNode "eth_blockNumber" should return the latest block number using cache
"eth_blockNumber" should return the latest block number ‑ @ethGetBlockByNumber using MirrorNode "eth_blockNumber" should return the latest block number
"eth_blockNumber" should throw an error if no blocks are found after third try ‑ @ethGetBlockByNumber using MirrorNode "eth_blockNumber" should throw an error if no blocks are found after third try
"eth_blockNumber" should throw an error if no blocks are found ‑ @ethGetBlockByNumber using MirrorNode "eth_blockNumber" should throw an error if no blocks are found
Adds a revertReason field for receipts with errorMessage ‑ @ethGetTransactionReceipt eth_getTransactionReceipt tests Adds a revertReason field for receipts with errorMessage
BLOCK_HASH filter timeouts and throws the expected error ‑ @ethGetLogs using MirrorNode timeout BLOCK_HASH filter timeouts and throws the expected error
BLOCK_HASH filter ‑ @ethGetLogs using MirrorNode BLOCK_HASH filter
Can extract the account number out of an account pagination next link url ‑ MirrorNodeClient Can extract the account number out of an account pagination next link url
Can extract the evm address out of an account pagination next link url ‑ MirrorNodeClient Can extract the evm address out of an account pagination next link url
…

♻️ This comment has been updated with latest results.

Signed-off-by: Eric Badiere <[email protected]>
@ebadiere ebadiere marked this pull request as ready for review November 13, 2024 15:12
Copy link
Collaborator

@Nana-EC Nana-EC left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good approach.
Question on the config changes.
Also I provided a suggestion to be a bit more strict

@@ -145,7 +145,7 @@ export default {
// @ts-ignore
HBAR_RATE_LIMIT_DURATION: parseInt(ConfigService.get('HBAR_RATE_LIMIT_DURATION') || '86400000'), // 1 day
// @ts-ignore
HBAR_RATE_LIMIT_TOTAL: BigNumber(ConfigService.get('HBAR_RATE_LIMIT_TINYBAR') || '800000000000'), // 8000 HBARs
HBAR_RATE_LIMIT_TOTAL: BigNumber(ConfigService.get('HBAR_RATE_LIMIT_TINYBAR') ?? '800000000000'), // 8000 HBARs
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Q: why this change and does it affect all the other configs?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While working on the acceptance test I noticed that the zero value for the HBAR_RATE_LIMIT_TINYBAR was not getting picked up. Most likely because the logical OR operator || returns the first truthy value and 0 is falsy. The nullish coalescing operator ?? falls back to the default value when the left-hand operand is null or undefined, not when it's 0 or any other falsy value.

@@ -90,6 +96,10 @@ export class HbarLimitService implements IHbarLimitService {
this.reset = this.getResetTimestamp();
this.remainingBudget = this.totalBudget;

if (this.remainingBudget.toTinybars().isZero()) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: as a guard I'd say zero or less

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

);
it('should return false if the rate limiter is disabled by setting HBAR_RATE_LIMIT_TINYBAR to zero', async function () {
// hbarLimitServiceDisabled.disableRateLimiter();
const result = await hbarLimitServiceDisabled.shouldLimit(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This actually doesn't really test that the rate limiter is working.
For instance if a future code change was made and the return was false for some other reason that a 0 value this logic would not catch it.
I think we should reconsider the layout on the product side.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated test to include a condition that would normally trigger rate limiting, but verifies that shouldLimit returns false since rate limiting is not enabled.

@@ -181,6 +191,10 @@ export class HbarLimitService implements IHbarLimitService {
requestDetails: RequestDetails,
estimatedTxFee: number = 0,
): Promise<boolean> {
if (this.disableRateLimiter) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggest adding an isEnabled() function to IHbarLimitService.
Then this method can call it and it can be tested.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@@ -887,5 +914,77 @@ describe('@hbarlimiter HBAR Limiter Acceptance Tests', function () {
});
});
});

describe('@hbarlimiter-batch3 BASIC Tier', () => {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: not related to PR focus but thanks for adding more coverage

Copy link
Collaborator Author

@ebadiere ebadiere Nov 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had to add it to another batch because of the order of loading environment variables in the acceptance tests. The relay.ts reads and sets the HBAR_RATE_LIMIT_TINYBAR at boot time.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe he is addressing the scenario in which the limiter is turned off, allowing an account to make calls without being HBAR rate limited.

@ebadiere, if that's true and the suite is needed, I suggest renaming the test suite from "BASIC Tier" to something more descriptive, such as "Unlimited Limit" or "Limiter Off" or something like that. The term "BASIC Tier" is confusing, as we have another suite with the same name.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@quiet-node Done.

packages/server/tests/acceptance/hbarLimiter.spec.ts Outdated Show resolved Hide resolved
@@ -887,5 +914,77 @@ describe('@hbarlimiter HBAR Limiter Acceptance Tests', function () {
});
});
});

describe('@hbarlimiter-batch3 BASIC Tier', () => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe he is addressing the scenario in which the limiter is turned off, allowing an account to make calls without being HBAR rate limited.

@ebadiere, if that's true and the suite is needed, I suggest renaming the test suite from "BASIC Tier" to something more descriptive, such as "Unlimited Limit" or "Limiter Off" or something like that. The term "BASIC Tier" is confusing, as we have another suite with the same name.

packages/server/tests/acceptance/hbarLimiter.spec.ts Outdated Show resolved Hide resolved
packages/server/tests/acceptance/hbarLimiter.spec.ts Outdated Show resolved Hide resolved
package.json Show resolved Hide resolved
trigger rate limiting, and added isEnabled() method.

Signed-off-by: Eric Badiere <[email protected]>
Comment on lines 915 to 918
const basicPlans = await hbarSpendingPlanRepository.findAllActiveBySubscriptionTier(
[SubscriptionTier.BASIC],
requestDetails,
);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't seem like this is used anywhere

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, thanks. Removed.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed.

Comment on lines 904 to 913
const neededAccounts: number = 1;
accounts.push(
...(await Utils.createMultipleAliasAccounts(
mirrorNode,
initialAccount,
neededAccounts,
initialBalance,
requestDetails,
)),
);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like you created an extra account here but in the test below you use the initialAccount, accounts[0]. Guess we don't need to create extra account here

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. Removed and just using global.accounts[0]. Thanks.

Comment on lines +159 to +162
isEnabled(): boolean {
return !this.disableRateLimiter;
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like you attempted to add this for testing purposes, right? I guess we can remove it and can only use disableRateLimiter to determine if it should throw limit or not

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was recommended by @Nana-EC in one of the earlier comments. It makes sense to me.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I think it's cleaner interface wise also since the service isn't required to be on. I'm open to debate

Comment on lines +202 to +204
if (!this.isEnabled()) {
return false;
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (!this.isEnabled()) {
return false;
}
if (this.disableRateLimiter) {
return false;
}

@@ -37,6 +37,12 @@ export class HbarLimitService implements IHbarLimitService {
PRIVILEGED: Hbar.fromTinybars(constants.HBAR_RATE_LIMIT_PRIVILEGED),
};

/**
* Disables the rate limiter.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is quite misleading. It doesn't actually disables the limiter but it's more like a flag to turn the limiter on and off, right?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated comments to be more descriptive.

@@ -31,4 +31,5 @@ export interface IHbarLimitService {
estimatedTxFee?: number,
): Promise<boolean>;
addExpense(cost: number, ethAddress: string, requestDetails: RequestDetails, ipAddress?: string): Promise<void>;
isEnabled(): boolean;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

might not be needed

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was recommended by @Nana-EC

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah see #3252 (comment)

Signed-off-by: Eric Badiere <[email protected]>

fix: clean up of experimental code.

Signed-off-by: Eric Badiere <[email protected]>
@ebadiere ebadiere force-pushed the 3251-habr-rate-limiter-can-be-disabled branch from 240a06c to 1938973 Compare November 13, 2024 22:38
Copy link

sonarcloud bot commented Nov 13, 2024

Copy link

codecov bot commented Nov 13, 2024

Codecov Report

Attention: Patch coverage is 85.71429% with 1 line in your changes missing coverage. Please review.

Project coverage is 78.40%. Comparing base (264d2cb) to head (1938973).
Report is 6 commits behind head on main.

Files with missing lines Patch % Lines
packages/relay/src/lib/constants.ts 0.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3252      +/-   ##
==========================================
+ Coverage   77.85%   78.40%   +0.55%     
==========================================
  Files          66       69       +3     
  Lines        4453     4627     +174     
  Branches      993     1040      +47     
==========================================
+ Hits         3467     3628     +161     
- Misses        613      640      +27     
+ Partials      373      359      -14     
Flag Coverage Δ
config-service 98.14% <ø> (ø)
relay 78.61% <85.71%> (+0.02%) ⬆️
server 83.28% <ø> (ø)
ws-server 36.66% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...s/relay/src/lib/services/hbarLimitService/index.ts 89.62% <100.00%> (+6.68%) ⬆️
packages/relay/src/lib/constants.ts 90.19% <0.00%> (+3.92%) ⬆️

... and 19 files with indirect coverage changes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Status: Backlog
Development

Successfully merging this pull request may close these issues.

HBar Rate Limiter should have the ability to be disabled.
3 participants