Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compress on the fly #82

Merged
merged 9 commits into from
Oct 6, 2023
Merged

Compress on the fly #82

merged 9 commits into from
Oct 6, 2023

Conversation

vchuravy
Copy link
Member

Fixes #81

@vchuravy vchuravy requested a review from NHDaly April 28, 2023 03:50
Copy link
Member

@NHDaly NHDaly left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, huh. TIL! :) Thanks Valentin.

test/PProf.jl Outdated Show resolved Hide resolved
test/flamegraphs.jl Outdated Show resolved Hide resolved
test/Allocs.jl Outdated Show resolved Hide resolved
This is a breaking change, since the format of the files we write has changed, and the process to open them has differed.
@NHDaly
Copy link
Member

NHDaly commented Jul 21, 2023

🤔 hrm tests are failing now, even though we are decoding. i'm not sure why. Any idea?

@NHDaly
Copy link
Member

NHDaly commented Aug 14, 2023

@adnan-alhomssi: This seems like a good idea, and is related to your suggestion that we should be compressing the profiles and heapsnapshots before sending them back from ProfileEndpoints.jl.

Can you have a look and see if you can help us figure out why the tests aren't passing?

@NHDaly
Copy link
Member

NHDaly commented Sep 1, 2023

Request for review: @Drvi

test/PProf.jl Show resolved Hide resolved
@codecov-commenter
Copy link

codecov-commenter commented Oct 6, 2023

Codecov Report

Merging #82 (a59d315) into main (45399a8) will increase coverage by 0.06%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main      #82      +/-   ##
==========================================
+ Coverage   96.59%   96.66%   +0.06%     
==========================================
  Files           3        3              
  Lines         294      300       +6     
==========================================
+ Hits          284      290       +6     
  Misses         10       10              
Files Coverage Δ
src/Allocs.jl 92.40% <100.00%> (+0.19%) ⬆️
src/PProf.jl 99.19% <100.00%> (+0.01%) ⬆️
src/flamegraphs.jl 96.90% <100.00%> (+0.06%) ⬆️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@NHDaly NHDaly enabled auto-merge (squash) October 6, 2023 21:37
@NHDaly NHDaly merged commit 6e9f56c into main Oct 6, 2023
11 checks passed
@NHDaly NHDaly deleted the vc/compress branch October 6, 2023 21:52
@Drvi
Copy link
Contributor

Drvi commented Oct 12, 2023

Will this break reading profiles from older PProf.jl versions? Typically with text files, one can sniff the gzip "magic number", but with ProtoBuf I think we need to check first if there are possible collisions with the encoded message.

@NHDaly
Copy link
Member

NHDaly commented Oct 20, 2023

Thankfully, i don't think PProf.jl ever reads profiles, we just hand them over to pprof, which seems to be able to read them fine, in either version.

So i think it's okay? What concern did you have in mind?

@Drvi
Copy link
Contributor

Drvi commented Oct 20, 2023

Ah if we don't need to read it on Julia side, then it's fine:+1: sorry for the noise

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Actually compress the outputs
4 participants