Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test(perf): increase cpu threshold #6249

Merged
merged 3 commits into from
Dec 17, 2024

Conversation

Hweinstock
Copy link
Contributor

Problem

#6225

  • Didn't think 400% could be exceeded, but this one is especially volatile. Sometimes it barely spikes, and sometimes it goes crazy.

Solution

  • Increase to 500%

  • Treat all work as PUBLIC. Private feature/x branches will not be squash-merged at release time.
  • Your code changes must meet the guidelines in CONTRIBUTING.md.

License: I confirm that my contribution is made under the terms of the Apache 2.0 license.

Copy link
Contributor

@jpinkney-aws jpinkney-aws left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At this point are we even getting value from the performance tests? It feels like we're just going to increase the number indefinitely

@Hweinstock
Copy link
Contributor Author

I don't think the system stat thresholds are giving us much insight because of their flakiness. The thresholds on fs operations does give us some info, and would catch performance regressions where we somehow accidentally blow up the number of fs operations. However, I think its unlikely an engineer would do this, and performance issues affecting customers would likely be more nuanced than us making a few unneeded fs calls.

@Hweinstock
Copy link
Contributor Author

Hmmm, looks like these tests are very flaky

describe('full support', function () {
const fileExtLists = ['java', 'js', 'ts', 'py', 'tsx', 'jsx']
before(async function () {
this.timeout(60000)
})
beforeEach(async function () {
tempFolder = (await createTestWorkspaceFolder()).uri.fsPath
})
afterEach(async function () {
sinon.restore()
await closeAllEditors()
})
fileExtLists.forEach((fileExt) => {
it('should be non empty', async function () {
sinon.stub(FeatureConfigProvider.instance, 'getProjectContextGroup').returns('control')
const editor = await toTextEditor('content-1', `file-1.${fileExt}`, tempFolder)
await toTextEditor('content-2', `file-2.${fileExt}`, tempFolder, { preview: false })
await toTextEditor('content-3', `file-3.${fileExt}`, tempFolder, { preview: false })
await toTextEditor('content-4', `file-4.${fileExt}`, tempFolder, { preview: false })
await assertTabCount(4)
const actual = await crossFile.fetchSupplementalContextForSrc(editor, fakeCancellationToken)
assert.ok(actual && actual.supplementalContextItems.length !== 0)
})
})
})
.

@justinmk3
Copy link
Contributor

Hmmm, looks like these tests are very flaky

#6252

@Hweinstock Hweinstock marked this pull request as ready for review December 16, 2024 19:12
@Hweinstock Hweinstock requested a review from a team as a code owner December 16, 2024 19:12
@justinmk3
Copy link
Contributor

/runintegrationtests

@Hweinstock Hweinstock changed the title test(perf): increase cpu threshold. test(perf): increase cpu threshold Dec 17, 2024
@Hweinstock
Copy link
Contributor Author

/retryBuilds

@Hweinstock Hweinstock merged commit 32c7b76 into aws:master Dec 17, 2024
21 of 28 checks passed
@Hweinstock Hweinstock deleted the perf/changeThreshold branch December 17, 2024 16:47
karanA-aws pushed a commit to karanA-aws/aws-toolkit-vscode that referenced this pull request Jan 17, 2025
## Problem
aws#6225
- Didn't think 400% could be exceeded, but this one is especially
volatile. Sometimes it barely spikes, and sometimes it goes crazy.
## Solution
- Increase to 500%

---

- Treat all work as PUBLIC. Private `feature/x` branches will not be
squash-merged at release time.
- Your code changes must meet the guidelines in
[CONTRIBUTING.md](https://github.com/aws/aws-toolkit-vscode/blob/master/CONTRIBUTING.md#guidelines).

License: I confirm that my contribution is made under the terms of the
Apache 2.0 license.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants