Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Respect the pagination size for queries #117

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Respect the pagination size for queries #117

wants to merge 1 commit into from

Conversation

derjust
Copy link
Owner

@derjust derjust commented Jan 17, 2018

This change respects the pagination size as provided from a Pageable
object given to a findXXX method.
Mostly important in conjunction with lazy list processing via

DynamoDBMapperConfig.Builder builder = new DynamoDBMapperConfig.Builder();
builder.setPaginationLoadingStrategy(PaginationLoadingStrategy.ITERATION_ONLY);

as this loads the result set of a query page-by-page.
Also see the comments in
com.amazonaws.services.dynamodbv2.datamodeling.PaginatedQueryList<T> as
used by
org.socialsignin.spring.data.dynamodb.core.DynamoDBTemplate.query(Class<T>, QueryRequest)

This was referenced Jan 17, 2018
Closed
Open
derjust
derjust commented Jan 17, 2018
View reviewed changes
...va/org/socialsignin/spring/data/dynamodb/repository/query/AbstractDynamoDBQueryCriteria.java Outdated
return queryRequest;
}

private void applyPageableIfSpecified(QueryRequest queryRequest) {
if (pageable != null) {
Copy link
Owner Author

@derjust derjust Jan 17, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use Pageable#unpaged() instead

derjust
derjust commented Jan 17, 2018
View reviewed changes
src/main/java/org/socialsignin/spring/data/dynamodb/repository/query/DynamoDBQueryCreator.java Outdated
public DynamoDBQueryCreator(PartTree tree,
DynamoDBEntityInformation<T, ID> entityMetadata,
DynamoDBOperations dynamoDBOperations) {
super(tree, entityMetadata, dynamoDBOperations);
pageable = null;
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use Pageable#unpaged() instead

@derjust
Copy link
Owner Author

derjust commented Jan 17, 2018

And update for the Wiki/Documentation is required

@codecov-io
Copy link

codecov-io commented Jan 17, 2018 •
edited
Loading

Codecov Report

Merging #117 into master will decrease coverage by 15.34%.
The diff coverage is 66.66%.

Impacted file tree graph

@@              Coverage Diff              @@
##             master     #117       +/-   ##
=============================================
- Coverage     64.29%   48.95%   -15.35%     
+ Complexity      549      424      -125     
=============================================
  Files            71       70        -1     
  Lines          1882     1859       -23     
  Branches        354      352        -2     
=============================================
- Hits           1210      910      -300     
- Misses          543      828      +285     
+ Partials        129      121        -8
Impacted Files Coverage Δ Complexity Δ
...ynamodb/repository/query/DynamoDBQueryCreator.java 60% <66.66%> (-20%) 2 <0> (ø)
...epository/query/AbstractDynamoDBQueryCriteria.java 67.71% <66.66%> (-1.35%) 103 <2> (+5)
...ory/config/DynamoDBRepositoryNameSpaceHandler.java 0% <0%> (-100%) 0% <0%> (-2%)
...pository/config/DynamoDBRepositoriesRegistrar.java 0% <0%> (-100%) 0% <0%> (-3%)
.../dynamodb/mapping/event/AuditingEventListener.java 0% <0%> (-100%) 0% <0%> (-2%)
...ynamodb/mapping/DefaultDynamoDBDateMarshaller.java 0% <0%> (-100%) 0% <0%> (-1%)
...namodb/mapping/AbstractDynamoDBDateMarshaller.java 0% <0%> (-100%) 0% <0%> (-5%)
...ng/data/dynamodb/query/QueryRequestCountQuery.java 0% <0%> (-100%) 0% <0%> (-2%)
...data/dynamodb/query/QueryExpressionCountQuery.java 0% <0%> (-100%) 0% <0%> (-2%)
...ository/support/DynamoDBRepositoryFactoryBean.java 0% <0%> (-100%) 0% <0%> (-7%)
... and 51 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a8523f4...437d47a. Read the comment docs.

@derjust
Copy link
Owner Author

derjust commented Jan 17, 2018

Updated documentation at https://github.com/derjust/spring-data-dynamodb/wiki/Query-Size-Limits-and-Pageable

@derjust
Copy link
Owner Author

derjust commented Jan 17, 2018

Second thought on org.socialsignin.spring.data.dynamodb.repository.query.AbstractDynamoDBQuery.SlicedExecution.readPageOfResultsRestrictMaxResultsIfNecessary(Iterator<T>, int) required

@derjust
Respect the pagination size for queries
437d47a 
This change respects the pagination size as provided from a Pageable
object given to a findXXX method.
Mostly important in conjunction with lazy list processing via
```java
DynamoDBMapperConfig.Builder builder = new DynamoDBMapperConfig.Builder();
builder.setPaginationLoadingStrategy(PaginationLoadingStrategy.ITERATION_ONLY);
```
as this loads the result set of a query page-by-page.
Also see the comments in
`com.amazonaws.services.dynamodbv2.datamodeling.PaginatedQueryList<T>` as
used by
`org.socialsignin.spring.data.dynamodb.core.DynamoDBTemplate.query(Class<T>,
QueryRequest)`
@derjust derjust mentioned this pull request Mar 11, 2019
Open
@gmiro
Copy link

gmiro commented Feb 21, 2022

@derjust - Do you plan on merging this PR anytime soon? Trying to decide if I should use the DDB mapper directly instead of relying on spring-data-dynamodb. Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@derjust @codecov-io @gmiro