Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add TestRule for preserving logs #191

Open
wants to merge 1 commit into
base: scylla-4.x
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -286,6 +286,10 @@ If you use CCM, it **must** be with `CcmRule`.
For an example of a Simulacron-based parallelizable test, see `NodeTargetingIT`. For a CCM-based
test, see `DirectCompressionIT`.

Currently `CcmRule` based tests will not preserve CCM config directory upon failure. To have CCM
logs available in case of failure during automated testing combine it with `PreserveLogsRule` so
that logs can be uploaded.

##### Serial tests

These tests cannot run in parallel, in general because they require CCM clusters of different sizes,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,12 @@
import com.datastax.oss.driver.api.core.config.DriverConfigLoader;
import com.datastax.oss.driver.api.core.session.Session;
import com.datastax.oss.driver.api.testinfra.ccm.CcmRule;
import com.datastax.oss.driver.api.testinfra.ccm.PreserveLogsRule;
import com.datastax.oss.driver.api.testinfra.session.SessionRule;
import com.datastax.oss.driver.api.testinfra.session.SessionUtils;
import com.datastax.oss.driver.categories.ParallelizableTests;
import org.junit.ClassRule;
import org.junit.Rule;
import org.junit.Test;
import org.junit.experimental.categories.Category;
import org.junit.rules.RuleChain;
Expand All @@ -42,6 +44,8 @@ public class ConnectKeyspaceIT {
@ClassRule
public static final TestRule CHAIN = RuleChain.outerRule(CCM_RULE).around(SESSION_RULE);

@Rule public PreserveLogsRule LOGS_RULE = new PreserveLogsRule(CCM_RULE.getCcmBridge());

@Test
public void should_connect_to_existing_keyspace() {
CqlIdentifier keyspace = SESSION_RULE.keyspace();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ public abstract class BaseCcmRule extends CassandraResourceRule {
new Thread(
() -> {
try {
ccmBridge.remove();
ccmBridge.removeOrStop();
} catch (Exception e) {
// silently remove as may have already been removed.
}
Expand All @@ -57,7 +57,7 @@ protected void before() {

@Override
protected void after() {
ccmBridge.remove();
ccmBridge.removeOrStop();
}

private Statement buildErrorStatement(
Expand Down Expand Up @@ -232,4 +232,8 @@ public ProtocolVersion getHighestProtocolVersion() {
return DefaultProtocolVersion.V3;
}
}

public CcmBridge getCcmBridge() {
return ccmBridge;
}
Comment on lines +235 to +238

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this method moved here from CustomCcmRule?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could be an accident, but this was so long ago that I need to look at it again.

Copy link
Collaborator Author

@Bouncheck Bouncheck Aug 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it was to allow combining new rule with all other rules. New rule needs ccmBridge to modify its state.

}
Original file line number Diff line number Diff line change
Expand Up @@ -179,6 +179,8 @@ public class CcmBridge implements AutoCloseable {
private final List<String> dseWorkloads;
private final String jvmArgs;

private boolean keepLogs = false;

private CcmBridge(
Path configDirectory,
int[] nodes,
Expand Down Expand Up @@ -378,6 +380,14 @@ public void remove() {
execute("remove");
}

public void removeOrStop() {
if (keepLogs) {
stop();
} else {
remove();
}
}

public void pause(int n) {
execute("node" + n, "pause");
}
Expand Down Expand Up @@ -471,7 +481,11 @@ protected void processLine(String line, int logLevel) {

@Override
public void close() {
remove();
removeOrStop();
}

public void setKeepLogs(boolean keepLogs) {
this.keepLogs = keepLogs;
}

/**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -50,10 +50,6 @@ protected void after() {
CURRENT.compareAndSet(this, null);
}

public CcmBridge getCcmBridge() {
return ccmBridge;
}

public static Builder builder() {
return new Builder();
}
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
package com.datastax.oss.driver.api.testinfra.ccm;

import org.junit.rules.TestWatcher;
import org.junit.runner.Description;

public class PreserveLogsRule extends TestWatcher {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question - why not to make it configurable from environment variable ? So that it could be easily set for whole run.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC this was the simplest way to have the rule react only in case of failure. The goal was to skip logs of passing runs.
Global switch would be nice for running locally, but I'm not sure if we want CI to keep regularly uploading unnecessary log files.

Copy link
Collaborator

@dkropachev dkropachev Sep 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We definitely would want to store ccm logs on test failures on cicd.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's make it global flag that controls whether we keep ccm logs after the test or not.

private final CcmBridge ccmBridge;

public PreserveLogsRule(CcmBridge ccmBridge) {
this.ccmBridge = ccmBridge;
}

@Override
protected void failed(Throwable e, Description description) {
ccmBridge.setKeepLogs(true);
}
}