Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DOC] update doc for 24.12 release [skip ci] #11841

Merged
merged 5 commits into from
Dec 13, 2024

Conversation

nvliyuan
Copy link
Collaborator

@nvliyuan nvliyuan commented Dec 9, 2024

This PR updates download docs for 24.12.0 release.
Signed-off-by: liyuan [email protected]

Signed-off-by: liyuan <[email protected]>
@nvliyuan nvliyuan requested review from sameerz, jlowe and viadea December 9, 2024 09:08
@nvliyuan nvliyuan self-assigned this Dec 9, 2024
@nvliyuan nvliyuan added the documentation Improvements or additions to documentation label Dec 9, 2024
Signed-off-by: liyuan <[email protected]>
docs/download.md Outdated
* Support ArrayJoin expression
* Optimize Expand and Aggregate expression performance
* Improve JSON related expressions
* Implement Kudo shuffle format to improve shuffle performance
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This disabled by default with an internal config. There's no documentation on how to enable this, so I'm not sure it's appropriate to call it out here. It also doesn't improve shuffle performance for small schemas and/or relatively small partition counts (e.g.: default 200 or less).

docs/download.md Outdated
* Optimize Expand and Aggregate expression performance
* Improve JSON related expressions
* Implement Kudo shuffle format to improve shuffle performance
* Add the ability to decompress snappy and ztsd compressed Parquet files via CPU
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is also an internal-only config with no documentation. Not sure this is appropriate to advertise.

docs/download.md Outdated
* Implement Kudo shuffle format to improve shuffle performance
* Add the ability to decompress snappy and ztsd compressed Parquet files via CPU
* Add repartition-based algorithm fallback in hash aggregate
* Support spark function months_between
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* Support spark function months_between
* Support Spark function months_between

docs/download.md Outdated
* Support asynchronous writing for Parquet files
* Add retry support to improve sub hash-join stability
* Improve JSON scan and from_json
* Support setting a limited number of tasks for build-in profiling tool
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same issue here, the self-profiling tool is still experimental and not publicly documented yet.

docs/download.md Outdated
* Add retry support to improve sub hash-join stability
* Improve JSON scan and from_json
* Support setting a limited number of tasks for build-in profiling tool
* Bugs fix for memory leak
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally this should be a bit more descriptive. Is it a host or device memory leak? What are the conditions for the leak, i.e.: what types of operations were affected?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Double check the PRs that we don't have a memory leak bug fix this release, will remove it.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have at least this one: 3449c8a

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and this one 96a58d1

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated, thx

docs/download.md Show resolved Hide resolved
Signed-off-by: liyuan <[email protected]>
@nvliyuan nvliyuan requested a review from jlowe December 10, 2024 03:02
jlowe
jlowe previously approved these changes Dec 10, 2024
@abellina
Copy link
Collaborator

Just double checking that this PR can go in after #11848. Or, should it be pulled into the merge to main branch? Probably doesn't matter too much.

Signed-off-by: liyuan <[email protected]>
@nvliyuan
Copy link
Collaborator Author

Just double checking that this PR can go in after #11848. Or, should it be pulled into the merge to main branch? Probably doesn't matter too much.

this pr need to go in before #11848 as we used to do

@nvliyuan nvliyuan requested review from abellina and jlowe and removed request for abellina December 11, 2024 02:38
@NvTimLiu
Copy link
Collaborator

Just double checking that this PR can go in after #11848. Or, should it be pulled into the merge to main branch? Probably doesn't matter too much.

this pr need to go in before #11848 as we used to do

Yes, we'll do the final up-merge for merge-to-main PR #11848, to include all PRs targeting to the branch-24.12, then merge them to main!

@NvTimLiu NvTimLiu self-requested a review December 11, 2024 05:13
NvTimLiu
NvTimLiu previously approved these changes Dec 11, 2024
Copy link
Collaborator

@NvTimLiu NvTimLiu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Signed-off-by: liyuan <[email protected]>
@nvliyuan
Copy link
Collaborator Author

Hi @jlowe @abellina , can you please help check again, thx

@nvliyuan
Copy link
Collaborator Author

build

@nvliyuan nvliyuan merged commit 4b9bb23 into NVIDIA:branch-24.12 Dec 13, 2024
48 of 49 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants