Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Re-prepare statement throttle #143

Draft
wants to merge 2 commits into
base: scylla-3.x
Choose a base branch
from
Draft

Re-prepare statement throttle #143

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
fd8b68b
Add re-prepare statement throttle.
Bouncheck Jun 12, 2022
ae13fd2
Formatting.
Bouncheck Jun 16, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions driver-core/src/main/java/com/datastax/driver/core/RequestHandler.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,8 @@ class RequestHandler {
private static final boolean HOST_METRICS_ENABLED =
Boolean.getBoolean("com.datastax.driver.HOST_METRICS_ENABLED");
private static final QueryLogger QUERY_LOGGER = QueryLogger.builder().build();

private static final int SPECULATIVE_EXECUTION_MAX_REPREPARES = 5;
static final String DISABLE_QUERY_WARNING_LOGS = "com.datastax.driver.DISABLE_QUERY_WARNING_LOGS";

final String id;
Expand Down Expand Up @@ -381,6 +383,10 @@ class SpeculativeExecution implements Connection.ResponseCallback {
// This is incremented by one writer at a time, so volatile is good enough.
private volatile int retriesByPolicy;

// In rare cases we can enter seemingly infinite loop of repreparing a statement.
// This counter is used for breaking such stalemates.
private volatile int retriesByUnprepared;

private volatile Connection.ResponseHandler connectionHandler;

SpeculativeExecution(Message.Request request, int position) {
Expand Down Expand Up @@ -825,13 +831,30 @@ public void onSet(
toPrepare.getQueryString()));
}

if (retriesByUnprepared > SPECULATIVE_EXECUTION_MAX_REPREPARES) {
connection.release();
String msg =
String.format(
"Statement %s (%s) is not prepared on %s and reprepare threshold (%d) has been reached for this execution. "
+ "This might have been caused by driver misuse or the cluster. Check cluster logs for the reason of possible prepared statement cache evictions.",
toPrepare.getQueryString(),
id,
toPrepare.getQueryKeyspace(),
SPECULATIVE_EXECUTION_MAX_REPREPARES);

logger.error(msg);
setFinalException(connection, new DriverInternalError(msg));
return;
}

logger.info(
"Query {} is not prepared on {}, preparing before retrying executing. "
+ "Seeing this message a few times is fine, but seeing it a lot may be source of performance problems",
toPrepare.getQueryString(),
toPrepare.getQueryKeyspace(),
connection.endPoint);

retriesByUnprepared++;
write(connection, prepareAndRetry(toPrepare.getQueryString()));
// we're done for now, the prepareAndRetry callback will handle the rest
return;
Expand Down