Sync a fork 10 #14
Open
Sync a fork 10 #14
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…nts from gravitino-env.sh (apache#6385) <!-- 1. Title: [#<issue>] <type>(<scope>): <subject> Examples: - "[apache#123] feat(operator): support xxx" - "[apache#233] fix: check null before access result in xxx" - "[MINOR] refactor: fix typo in variable name" - "[MINOR] docs: fix typo in README" - "[apache#255] test: fix flaky test NameOfTheTest" Reference: https://www.conventionalcommits.org/en/v1.0.0/ 2. If the PR is unfinished, please mark this PR as draft. --> ### What changes were proposed in this pull request? Update Gravitino JVM memory default. ### Why are the changes needed? Fix: apache#6384 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? No test
…ig files and open-file flag test cases (apache#6280) ### What changes were proposed in this pull request? Add gvfs-fuse integration tests for big files and open-file flag test cases ### Why are the changes needed? Fix: apache#6279 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? IT
…ete...limit..` clause. (apache#6393) ### What changes were proposed in this pull request? PostgreSQL does not support SQL sentences like `DELETE FROM xxxx_table where xxxx limit 10` , Cluase `limit xxx` is not allowed in the `Delete syntax` ### Why are the changes needed? it's a bug. Fix: apache#6380 ### Does this PR introduce _any_ user-facing change? N/A ### How was this patch tested? UT
…6360) ### What changes were proposed in this pull request? Add tag support for model in CLI. 1. `UntagEntity` 2. `TagEntity` 3. `ListEntityTags` The logic for handling models in these three methods has been added. need to add the processing logic to the `RemoveAllTags` method. ### Why are the changes needed? Fix: apache#6356 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? local test.
…ng (apache#6398) ### What changes were proposed in this pull request? Improve the gvfs to expose the nested exception for better understanding. ### Why are the changes needed? The exception message may not be enough to understand the problem why the fs initialization is failed. So we should expose the whole stack for better understanding. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Existing tests.
…void conflict with authorization module (apache#6376) ### What changes were proposed in this pull request? Remove jar `protobuf-java.jar` from the distribution package to avoid conflicts ### Why are the changes needed? To make authorization works for GCS fileset. Fix: apache#6375 ### Does this PR introduce _any_ user-facing change? N/A. ### How was this patch tested? N/A
…6.1 (apache#6374) ### What changes were proposed in this pull request? Upgrade Iceberg version from 1.5.2 to 1.6.1 In 1.6, the `token` endpoint is deprecated and there're no new endpoint is added. ### Why are the changes needed? Fix: apache#6394 ### Does this PR introduce _any_ user-facing change? no ### How was this patch tested? run pass existing ITs
…gvfs-fuse command line tools (apache#6321) ### What changes were proposed in this pull request? 1. Support mount and umount command 2. Make mount create a daemon process to run background ### Why are the changes needed? Fix: apache#6320 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? IT --------- Co-authored-by: Qiming Teng <[email protected]>
…esystem::stat() to improve the performance of open_dal_filesystem (apache#6358) ### What changes were proposed in this pull request? Passing file type argument in the Filesystem::stat() to improve the performance of open_dal_filesystem ### Why are the changes needed? Fix: apache#6357 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? UI --------- Co-authored-by: Qiming Teng <[email protected]>
### What changes were proposed in this pull request? Refactor to add a command context and simple wrappers on output to make it easy for "global" command like a "--quiet" option to be added. Note this in progress as this only has changed Metlake. Some duplicate code can be removed once everything is done. ### Why are the changes needed? For maintainability. Fix: apache#6326 ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Tested locally. --------- Co-authored-by: Shaofeng Shi <[email protected]>
…pache#6408) ### What changes were proposed in this pull request? Move the `Fileset` and `Model` related classes to the sub-package for better organized. ### Why are the changes needed? For better organized. Fix: apache#6407 ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Existing tests.
…pache#5980) ### What changes were proposed in this pull request? Implemented pre-event handling for all tag-related operations, including: - listTags - listTagsInfo - getTag - createTag - alterTag - deleteTag - listMetadataObjectsForTag - listTagsForMetadataObject - listTagsInfoForMetadataObject - associateTagsForMetadataObject - getTagForMetadataObject ### Why are the changes needed? Fix: apache#5900 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? add UT
…the return value of fileset catalog (apache#6410) ### What changes were proposed in this pull request? This PR changes the return value to `GenericFileset` for fileset operation. ### Why are the changes needed? Previously, it returns the plain fileset dto, so that it cannot support some operations like get credentials. Fix: apache#6409 ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Existing tests.
…ache#6359) ### What changes were proposed in this pull request? Fix the CLI delete the anonymous user ### Why are the changes needed? Fix: apache#5932 ### Does this PR introduce _any_ user-facing change? (Please list the user-facing changes introduced by your change, including 1. Change in user-facing APIs. 2. Addition or removal of property keys.) ### How was this patch tested? local test ```bash gcli user delete -m demo_metalake --user anonymous # Can't delete anonymous user. This will cause unexpected behavior. ``` <img width="536" alt="image" src="https://github.com/user-attachments/assets/479ec7eb-2ad3-42bd-b70a-a6f2ce05f794" />
### What changes were proposed in this pull request? Add table command context CLI. ### Why are the changes needed? Fix: apache#6419 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? local test.
) ### What changes were proposed in this pull request? Add Fileset command context CLI ### Why are the changes needed? Fix: apache#6415 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? local test
### What changes were proposed in this pull request? Add model command context CLI ### Why are the changes needed? Fix: apache#6417 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? local test
### What changes were proposed in this pull request? Add schema command context CLI ### Why are the changes needed? Fix: apache#6416 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? local test.
) ### What changes were proposed in this pull request? Add catalog command context CLI ### Why are the changes needed? Fix: apache#6414 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? local test.
### What changes were proposed in this pull request? Add stream command context CLI ### Why are the changes needed? Fix: apache#6418 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? local test.
### What changes were proposed in this pull request? Add column command context CLI ### Why are the changes needed? Fix: apache#6420 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? local test.
…iew.md` (apache#6439) ### What changes were proposed in this pull request? This PR fixes the description of the `Multi-engine support` section in `overview.md`, as Gravitino now supports connectors for Apache Spark and Apache Flink. ### Why are the changes needed? Gravitino now supports connectors for Apache Spark and Apache Flink. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? manual review
### What changes were proposed in this pull request? Add tags command context CLI ### Why are the changes needed? Fix: apache#6423 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? local test
…pache#6437) ### What changes were proposed in this pull request? Add user and group command context CLI ### Why are the changes needed? Fix: apache#6421 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? local test.
…pache#6438) ### What changes were proposed in this pull request? Add roles and owner command context CLI ### Why are the changes needed? Fix: apache#6422 ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? local test.
… simple commands. (apache#6440) ### What changes were proposed in this pull request? Refactor getURL in CLI and add context to simple commands. ### Why are the changes needed? Fix: apache#6424 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? local test.
### What changes were proposed in this pull request?
The PR updates incorrect log format codes from `%s` to `{}` in the
following lines:
- Line 420
- Line 483
- Line 504
- Line 525
The original code in these lines used the `%s` placeholder, which is
invalid for the logging framework used. It has been corrected to use
`{}` to match the proper syntax for log messages.
### Why are the changes needed?
This change is necessary to fix the incorrect log format codes in the
codebase. The `%s` placeholder is not valid and can lead to incorrect
log formatting or errors when the logs are generated. The correct
placeholder syntax for this logging framework is `{}`.
Fix: apache#6429
### Does this PR introduce _any_ user-facing change?
No user-facing changes are introduced. This is an internal fix for the
logging format.
### How was this patch tested?
The changes were verified manually by reviewing the code and ensuring
that all instances of `%s` were replaced with `{}` in the affected
lines. No new functionality was added, so no additional tests were
required.
…ate paimon table by flink (apache#6362) ### What changes were proposed in this pull request? Support specifying primary keys during create paimon table by flink ### Why are the changes needed? Fix: apache#6361 ### Does this PR introduce _any_ user-facing change? None ### How was this patch tested? Add testCreateTableWithPrimaryKey case in org.apache.gravitino.flink.connector.integration.test.FlinkCommonIT
### What changes were proposed in this pull request? multi catalog and view operation should shown in document side bars. ### Why are the changes needed? User could find multi catalog configuration easily ### Does this PR introduce _any_ user-facing change? no ### How was this patch tested? just document
### What changes were proposed in this pull request? Remove redundant code in CLI, and make context argument final in Command class. ### Why are the changes needed? The constructor should take only one argument, CommandContext. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? local test.
…pache#6447) ### What changes were proposed in this pull request? Add --quiet option to the Gravitino CLI ### Why are the changes needed? Fix: apache#6097 ### Does this PR introduce _any_ user-facing change? user can use --quiet option suppress the output. ### How was this patch tested? local test ```bash gcli tag create -m cli_demo --tag tagA tagB -i # Tags tagA,tagB created gcli tag create -m cli_demo --tag tagC tagD -i --quiet # no output ```
### What changes were proposed in this pull request? Refactor getAuth method in CLI. ### Why are the changes needed? Fix: apache#6449 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? local test.
### What changes were proposed in this pull request? authentication.type of catalog property should send correct request to server ### Why are the changes needed? N/A Fix: apache#6450 ### Does this PR introduce _any_ user-facing change? N/A ### How was this patch tested? manually
… roles is bound to many metadata. (apache#6455) ### What changes were proposed in this pull request? fix issue apache#6238 improve performance when a single role is bound to many metadata. ### Why are the changes needed? Use batch queries when getting role securable object full names instead of loop queries to get each securable object full name. Fix: apache#6238 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Unit tests and integration tests have all passed, this feature has been running internally at Xiaomi for two weeks. Co-authored-by: luoxin5 <[email protected]>
…o configure gvfs-fuse. (apache#6465) ### What changes were proposed in this pull request? Fix the bug of using a relative path to configure gvfs-fuse. ### Why are the changes needed? Fix: apache#6464 ### Does this PR introduce _any_ user-facing change? NO ### How was this patch tested? Manually test
…FSFileSystemProvider` problem (apache#6463) ### What changes were proposed in this pull request? In the current code base, we need to add `catalog-hadoop` to make GVFS client works or class `HDFSFileSystemProvider` and `LocalFileSystemProvider` can't be found. ### Why are the changes needed? It's a bug. Fix: apache#6462 ### Does this PR introduce _any_ user-facing change? N/A ### How was this patch tested? ITs, UTs and test locally.
### What changes were proposed in this pull request? This pr removes outdated tips from `web-ui.md` because, as indicated in the subsequent text, operations such as creating or modifying schemas, tables, or filesets can now be performed through the web UI. ### Why are the changes needed? Remove outdated tips from `web-ui.md` ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? manual review
…on parameters through environment variables (apache#6458) ### What changes were proposed in this pull request? Add the environment variables `GRAVITINO_JDBC_USER`, `GRAVITINO_JDBC_PASSWORD`, and `GRAVITINO_S3_ENDPOINT` to the `rewrite_config.py`, so database user and password can be set by the user in a container environment as well as a custom S3 endpoint. ### Why are the changes needed? Currently, JDBC user and PW are hardcoded to `iceberg` and cannot be changed when using the container images. One workaround is including the values in the JDBC connection string, but this neither elegant nor in line with how containers commonly handle this issue. Additionally, it wasn't possible to set the value of `gravitino.iceberg-rest.s3-endpoint` in a container environment, which can be useful in a variety of situations when using a provider other than AWS. ### Does this PR introduce _any_ user-facing change? Added the three aforementioned two environment variables that can be used by the user. ### How was this patch tested? Was tested locally only, since this is a minimal change.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
git fetch upstream
git merge upstream/main
// resolve conflict and then
git add bin/common.sh.template
git commit -m "resolve merge conflict"
git push origin sync-a-fork-10