HBASE-29280: Fix bug in RawAsyncTableImpl#coprocessorServiceUntilComplete retry logic

[charlesconnell]

I noticed a bug that I introduced in HBASE-28770. I introduced the ability for coprocessor endpoints to send partial responses. When this happens, the client needs to send a new request to fetch remaining results, and may sleep before doing so. The logic for sleeping looks like:

if (waitInterval.isZero()) {
  AsyncConnectionImpl.RETRY_TIMER.newTimeout(
    (timeout) -> coprocessorServiceUntilComplete(stubMaker, updatedCallable, callback,
      locateFinished, unfinishedRequest, region, span),
    waitInterval.toMillis(), TimeUnit.MILLISECONDS);
} else {
  coprocessorServiceUntilComplete(stubMaker, updatedCallable, callback, locateFinished,
    unfinishedRequest, region, span);
}  

which has backwards logic. The first line should read

if (!waitInterval.isZero()) {}

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 28s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	hbaseanti	0m 0s		Patch does not have any anti-patterns.
			_ master Compile Tests _
+1 💚	mvninstall	3m 26s		master passed
+1 💚	compile	0m 50s		master passed
+1 💚	checkstyle	0m 16s		master passed
+1 💚	spotbugs	0m 47s		master passed
+1 💚	spotless	0m 48s		branch has no errors when running spotless:check.
			_ Patch Compile Tests _
+1 💚	mvninstall	3m 10s		the patch passed
+1 💚	compile	0m 50s		the patch passed
+1 💚	javac	0m 50s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 16s		the patch passed
+1 💚	spotbugs	0m 54s		the patch passed
+1 💚	hadoopcheck	13m 22s		Patch does not cause any errors with Hadoop 3.3.6 3.4.0.
+1 💚	spotless	0m 51s		patch has no errors when running spotless:check.
			_ Other Tests _
+1 💚	asflicense	0m 11s		The patch does not generate ASF License warnings.
		34m 24s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6952/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#6952
JIRA Issue	HBASE-29280
Optional Tests	dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
uname	Linux a2ed9d896e3b 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / e0c7f3f
Default Java	Eclipse Adoptium-17.0.11+9
Max. process+thread count	83 (vs. ulimit of 30000)
modules	C: hbase-client U: hbase-client
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6952/1/console
versions	git=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 29s		Docker mode activated.
-0 ⚠️	yetus	0m 3s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ master Compile Tests _
+1 💚	mvninstall	5m 24s		master passed
+1 💚	compile	0m 37s		master passed
+1 💚	javadoc	0m 32s		master passed
+1 💚	shadedjars	8m 55s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	5m 50s		the patch passed
+1 💚	compile	1m 9s		the patch passed
+1 💚	javac	1m 9s		the patch passed
+1 💚	javadoc	0m 54s		the patch passed
+1 💚	shadedjars	12m 15s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
+1 💚	unit	4m 17s		hbase-client in the patch passed.
		41m 46s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6952/1/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	#6952
JIRA Issue	HBASE-29280
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux a33bc7a407d1 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / e0c7f3f
Default Java	Eclipse Adoptium-17.0.11+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6952/1/testReport/
Max. process+thread count	328 (vs. ulimit of 30000)
modules	C: hbase-client U: hbase-client
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6952/1/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

[Apache9]

No UT can cover this?

[charlesconnell]

To be honest, I think it's pretty difficult to unit test this. We don't want to actually introduce a sleep in a unit test, because that's slow, so we can't verify behavior by timing. We also can't mock out AsyncConnectionImpl.RETRY_TIMER because it's final.