-
Notifications
You must be signed in to change notification settings - Fork 47
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weโll occasionally send you account related emails.
Already on GitHub? Sign in to your account
INTERNAL: make lop piped operations process synchronously #795
base: develop
Are you sure you want to change the base?
Conversation
9eb5496
to
076d02b
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
์ผ๋ถ ๋ฆฌ๋ทฐ
|| rv.getOperationStatus().getResponse() == CollectionResponse.CANCELED) { | ||
// countdown if this is last op | ||
latch.countDown(); | ||
} else if (!rv.getOperationStatus().isSuccess()) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
rv.getOperationStatus().isSuccess()
์กฐ๊ฑด์ผ๋ก next ์ฐ์ฐ ์ํ ์ฌ๋ถ๋ฅผ ํ๋จํ๊ธฐ ์ด๋ ต์ต๋๋ค.
์ด ๋ถ๋ถ์ ๋ํด offline ๋
ผ์ํด์ผ ํ ๊ฒ ๊ฐ์ต๋๋ค.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
๊ด๋ จ ์ฌํญ์ ์ข ๋ ์์ธํ๊ฒ ์ ์ผ๋ฉด ๋ค์๊ณผ ๊ฐ์ต๋๋ค.
- PIPE_ERROR ์๋ต์ด ์จ ๊ฒฝ์ฐ
- ํ์ฌ์ piped ์ฐ์ฐ์์ PIPE_ERROR๋ก ์ธํด ์์ ์ํ๋์ง item ์ฐ์ฐ์ด ์์ ์ ์์ต๋๋ค.
- ์ด ๊ฒฝ์ฐ, ํ์ฌ failed Result์ ๊ฒฐ๊ณผ๋ฅผ ๋ด์ง ์๊ณ ์์ผ๋ฏ๋ก, ์ด๋ฅผ ์ถ๊ฐํด์ผ ํฉ๋๋ค.
- PIPE_END ์๋ต์ด ์จ ๊ฒฝ์ฐ
- ๊ฐ๋ณ ์ฐ์ฐ์ ๋ํด NOT_FOUND, ELEMENT_EXISTS, OVERFLOWED, OUT_OF_RANGE, TYPE_MISMATCH, BKEY_MISMATCH ์ค๋ฅ๊ฐ ๋ฐ์ํ ์ ์์ต๋๋ค.
- B+Tree collection์ ๋ํด BKEY_MISMATCH ์ค๋ฅ๊ฐ ๋ฐ์ํ ๊ฒฝ์ฐ, ์ฃผ์ด์ง element๋ง insert ์คํจํ๊ณ ๋๋จธ์ง ์ฑ๊ณตํ ์ ์์ต๋๋ค. ์ด ๊ฒฝ์ฐ, next ์ฐ์ฐ์ ์ํํ๋ ๊ฒ์ด ๋ง์ง ์๋ ์ง ? ELEMENT_EXISTS ์ค๋ฅ๋ ๋น์ทํด ๋ณด์ ๋๋ค.
- ๊ทธ ์ธ์ ์ค๋ฅ์ธ ๊ฒฝ์ฐ๋ ์บ์ ์๋ฒ์์ ์ํ์ ์ค์งํ๊ณ , PIPE_ERROR ๋ฆฌํดํด์ผ ํ์ง ์๋ ์๊ฐํฉ๋๋ค.
- ๊ฒฐ๊ตญ, ์บ์ ์๋ฒ ๋์ ๋ถ๋ถ๋ ํจ๊ป ๊ณ ๋ ค๋์ด์ผ ํ์ง ์๋ ์๊ฐํฉ๋๋ค.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
END ์๋ต์ด ์๊ณ btree์ผ ๋ BKEY_MISMATCH, map, set์ผ ๋ ELEMENT_EXISTS๋ง ๋ฐ์ํ๋ค๋ฉด ๋ค์ Operation ๋ณด๋ด๋ ๊ฒ์๋ ๋์ํฉ๋๋ค.
PIPE_ERROR๊ฐ CLIENT_ERROR, SERVER_ERROR ๋ก ์ธํด ๋ฐ์ํ๋ค๋ฉด ์์ธ๊ฐ ๋ฐ์ํด failed result๋ฅผ ์กฐํํ ์ ์์ต๋๋ค. PIPE_ERROR๊ฐ command/memory overflow ๋ก ์ธํด ๋ฐ์ํ๋ค๋ฉด ์๋ฌ๋ฅผ ๋ฌด์ํ๊ณ ๋ชจ๋ ์๋ต์ ๊ธฐ์ค์ผ๋ก ์ ์ฒด ์ฑ๊ณต/์คํจ ์ฌ๋ถ๋ง ํ๋ณํฉ๋๋ค. ๋ฐ๋ผ์ "PIPE_ERROR" ๋ผ๋ ์๋ต์ด ์๋์ง future์์๋ ํ์ธํ ์ ์์ผ๋ฉฐ, failed result์ ๊ฒฐ๊ณผ๋ฅผ ๋ฃ์ ํ์๋ ์์ต๋๋ค.
Lines 139 to 148 in f60ed48
if (line.startsWith("END") || line.startsWith("PIPE_ERROR ")) { | |
/* ENABLE_MIGRATION if */ | |
if (needRedirect()) { | |
transitionState(OperationState.REDIRECT); | |
return; | |
} | |
/* ENABLE_MIGRATION end */ | |
cb.receivedStatus((successAll) ? END : FAILED_END); | |
transitionState(OperationState.COMPLETE); | |
} else if (line.startsWith("RESPONSE ")) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
๊ธฐ์กด pipe ์ฒ๋ฆฌ์๋ ์ผ๋ถ ๋ฌธ์ ๊ฐ ์๋ ๊ฒ ๊ฐ์ต๋๋ค.
- CLIENT_ERROR, SERVER_ERROR ๋ฑ์ ์ค๋ฅ๊ฐ ์๋ ๊ฒฝ์ฐ, java client๋ PIPE_ERROR ์๋ต๊น์ง ์ฝ์ง ์๋๋ค.
๊ทธ๋ฌ๋ฉด, ๋ค์ ์ฐ์ฐ์์ ์๋ต์ ์ฝ์ ์์ PIPE_ERROR ์๋ต์ ๋ณด๊ฒ ๋๋ ๋ฌธ์ ๊ฐ ์์ต๋๋ค.
๋๋ต ํ์ธํ ์ฌํญ์ด๋ผ, ์ด ๋ถ๋ถ์ ๊ดํ ์ฝ๋๋ ์ง์ ํ์ธํด ๋ณด๊ธฐ ๋ฐ๋๋๋ค. - ์๋ 2๊ฐ์ง PIPE_ERROR์ ๋ํด "END" ์๋ต์ ๋ฐ์ ๊ฒฝ์ฐ์ ๋์ผํ๊ฒ ์ฒ๋ฆฌํฉ๋๋ค.
์ฆ, ์ฒ๋ฆฌํ์ง ๋ชปํ ์ฐ์ฐ์ด ์กด์ฌํ๋๋ผ๋ ์ฒ๋ฆฌ๊ฐ ์๋ฃ๋ ๊ฒ์ผ๋ก ํ๋จํ๊ณ ์์ต๋๋ค.- PIPE_ERROR command overflow
- PIPE_ERROR memory overflow
์ด๋ฌํ pipe ์ฒ๋ฆฌ ์ค๋ฅ๋ ๋ณธ PR๊ณผ๋ ๋ณ๋๋ก ์ฒ๋ฆฌํด์ผ ํ์ฃ ?
๊ฒํ ๋ฐ๋๋๋ค.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
CLIENT_ERROR, SERVER_ERROR ๋ฑ์ ์ค๋ฅ๊ฐ ์๋ ๊ฒฝ์ฐ, ๋ฐ๋ก OperationException์ด throw ๋ฉ๋๋ค.
์ด ์์ธ๋ฅผ MemcachedConnection์์ catchํ์ฌ ์ฐ๊ฒฐ์ ๋๊ธฐ ๋๋ฌธ์ ๋ค์ ์ฐ์ฐ์ด ์๋ต์ ์ฝ์ ์ ์์ ๊ฒ์ผ๋ก ๋ณด์
๋๋ค.
arcus-java-client/src/main/java/net/spy/memcached/MemcachedConnection.java
Lines 948 to 952 in 45720de
} catch (OperationException e) { | |
qa.setupForAuth("operation exception"); // noop if !shouldAuth | |
getLogger().warn("Reconnection due to exception " + | |
"handling a memcached exception on %s.", qa, e); | |
lostConnection(qa, ReconnDelay.IMMEDIATE, "operation exception"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PIPE_ERROR command/memory overflow ์ค๋ฅ ์, ์ฒ๋ฆฌ๋์ง ๋ชปํ ์ฐ์ฐ์ ๋ํด
๋ณธ PR ์ฒ๋ผ CANCELED CollectionOperationStatus ์ค์ ํ๋ ๊ฒ์ด ์ข๊ฒ ์ต๋๋ค.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PIPE_ERROR command/memory overflow ์ค๋ฅ ์, ์ฒ๋ฆฌ๋์ง ๋ชปํ ์ฐ์ฐ์ ๋ํด
๋ณธ PR ์ฒ๋ผ CANCELED CollectionOperationStatus ์ค์ ํ๋ ๊ฒ์ด ์ข๊ฒ ์ต๋๋ค.
์ ์ฌํญ์ ๊ตฌํํ๊ธฐ ์ํด
์๋ ๋ก์ง์์ cb.receivedStatus()
ํธ์ถ ์์ ์ธ์ ๊ฐ์ด ์์ ๋์ด์ผ ํ ๊ฒ ๊ฐ์ต๋๋ค.
if (line.startsWith("END") || line.startsWith("PIPE_ERROR ")) {
/* ENABLE_MIGRATION if */
if (needRedirect()) {
transitionState(OperationState.REDIRECT);
return;
}
/* ENABLE_MIGRATION end */
cb.receivedStatus((successAll) ? END : FAILED_END);
transitionState(OperationState.COMPLETE);
} else if (line.startsWith("RESPONSE ")) {
์๋์ ๊ฐ์ด ํธ์ถํด์ผ ํ ๊ฒ ๊ฐ์ต๋๋ค.
cb.receivedStatus((index == count && successAll) ? END : FAILED_END);
๋ํ, ์๋ ๊ฒฝ์ฐ์ CENCEL ์ํ๋ฅผ ์ถ๊ฐํ๋ ๋ก์ง๋ ์์ ๋์ด์ผ ํฉ๋๋ค.
if (!rv.getOperationStatus().isSuccess()) {
rv.addEachResult(index + (idx * CollectionPipedInsert.MAX_PIPED_ITEM_COUNT),
new CollectionOperationStatus(false, "CANCELED", CollectionResponse.CANCELED));
latch.countDown();
}
์์ ๊ฐ์ด ๋ณ๊ฒฝ๋๋ฉด,
complete() ๋ก์ง์์์ ๊ฒ์ฌ ์์๋ ๋ณ๊ฒฝ๋์ด์ผ ํฉ๋๋ค.
์๋ ์กฐ๊ฑด์ด ๊ฐ์ฅ ๋จผ์ ๊ฒ์ฌ๋์ด์ผ ํ ๊ฒ์
๋๋ค.
if (!rv.getOperationStatus().isSuccess())
90a7ee5
to
df233d9
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
๋ฆฌ๋ทฐ ์๋ฃ
600f2b2
to
d232a01
Compare
@jhpark816 ๋ฆฌ๋ทฐ ๋ฐ์ํ์ต๋๋ค. |
@uhm0311 ๋ฆฌ๋ทฐ ๋ฐ๋๋๋ค. |
์ปจํ๋ฆญํธ ํ์ธํด์ฃผ์ธ์. |
237a9b6
to
f4aabe9
Compare
@@ -145,6 +145,10 @@ assert getState() == OperationState.READING | |||
/* ENABLE_MIGRATION end */ | |||
cb.receivedStatus((successAll) ? END : FAILED_END); | |||
transitionState(OperationState.COMPLETE); | |||
} else if (line.startsWith("PIPE_ERROR ")) { | |||
// command flow / memory flow |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PIPE_ERROR ์ข ๋ฅ๊ฐ ๋ฌด์์ธ๊ฐ์?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PIPE_ERROR๋ ๋ค์๊ณผ ๊ฐ์ ์ข ๋ฅ๊ฐ ์์ต๋๋ค.
PIPE_ERROR bad error
: SERVER_ERROR / CLIENT_ERROR ๋ฐ์ํ์ฌ ์์ธ๊ฐ ๋ฐํ๋ ์ํฉ์ด๋ฉฐ, ํด๋น ๋ ์ผ์ด์ค๋ ํ์ฌ ์ฒ๋ฆฌ์ค์ exception์ด throw๋๋ฏ๋ก ์ด ์ฝ๋์ ๋๋ฌํ์ง ์์ต๋๋ค.PIPE_ERROR command overflow
: pipelining ๊ฐ๋ฅํ command์ ์ต๋ ๊ฐ์๋ฅผ ๋์ด์ ์์ฒญ์ด ๋ค์ด์ ์์ธ๊ฐ ๋ฐํ๋ ์ํฉPIPE_ERROR memory overflow
: pipelining ์ฒ๋ฆฌ๋ฅผ ์ํ ๋ฉ๋ชจ๋ฆฌ ๊ณต๊ฐ์ด ๋ถ์กฑํ๊ฒ ๋์ด ํ์ดํ๋ผ์ธ ์ํ์ ๋ฉ์ถ๊ณ ์์ธ๊ฐ ๋ฐํ๋ ์ํฉ
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
command overflow์ memory overflow์ผ ๋ Redirect๋ฅผ ํ์ง ์๋ ์ด์ ์ ํ๋ ์ด์ ๋ ๋ฌด์์ธ๊ฐ์?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
command overflow์ผ ๋๋ ํด๋น operation์ผ๋ก ์์ฑ๋๋ pipe ๋ช
๋ น์ด์ ๊ฐ์๊ฐ ๋์ผํ๊ธฐ ๋๋ฌธ์ redirectํ์ฌ ๋๊ฐ์ ๋ช
๋ น์ ๋ณด๋ด๋ฉด ์คํจํ๋ฏ๋ก redirectํ ํ์์ฑ์ด ์์ด๋ณด์
๋๋ค.
memory overflow์ผ ๋๋ ํด๋น ์บ์ ๋
ธ๋์ memory ์ฉ๋์ด ๋ถ์กฑํ๋ ๊ฒ์ด๊ธฐ ๋๋ฌธ์, ๋ค๋ฅธ ๋
ธ๋๋ก redirect๋์์ ๋ ์ฑ๊ณตํ ๊ฐ๋ฅ์ฑ์ด ์์ผ๋ฏ๋ก redirect ์ฒ๋ฆฌํ๋ฉด ๋ ๊ฒ์ผ๋ก ๋ณด์
๋๋ค.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
์ฐธ๊ณ ์ฌํญ์ผ๋ก, ํ์ฌ ์๋ฒ ๊ตฌํ ๊ธฐ์ค
- single key pipe์์
NOT_MY_KEY
์PIPE_ERROR
๋ฅผ ํจ๊ป ์๋ตํ๋ ๊ฒฝ์ฐ๋ ์์ ๊ฒ์ ๋๋ค. - multi key pipe์์๋
NOT_MY_KEY
์PIPE_ERROR
๊ฐ ํจ๊ป ๋ฐ์ํ ์ ์์ต๋๋ค.
๋ค๋ง migration ๊ด๋ จ ๊ตฌํ์ ์์ง ์์ ํ ์ ๋ฆฌ๋ ์ํ๋ ์๋์ด์ ์ถํ ๋ณ๊ฒฝ๋ ๊ฐ๋ฅ์ฑ์ ์์ด ๋ณด์ ๋๋ค.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
single key pipe์์ NOT_MY_KEY์ PIPE_ERROR๋ฅผ ํจ๊ป ์๋ตํ๋ ๊ฒฝ์ฐ๋ ์์ ๊ฒ์ ๋๋ค.
์ ๊ทธ๋ฐ์ง ๊ฐ๋จํ ์ค๋ช ํด ์ฃผ์์ฃ .
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
์ ๊ทธ๋ฐ์ง ๊ฐ๋จํ ์ค๋ช ํด ์ฃผ์์ฃ .
์ฝ๋๋ฅผ ํ์ธํด ๋ณด๋, NOT_MY_KEY
๋ฐ์ ์ดํ invalid command ์์ ํ๋ ๊ฒฝ์ฐ์๋
NOT_MY_KEY
+ CLIENT_ERROR
+ PIPE_ERROR bad error
์๋ต์ด ๊ฐ๋ฅํด ๋ณด์
๋๋ค.
๊ทธ ์ธ์ ์ ์์ ์ธ ๋ช ๋ น์ผ๋ก ๊ตฌ์ฑ๋ single key pipe์์๋ ์๋์ ๊ฐ์ด ๋์ํฉ๋๋ค.
https://github.com/jam2in/arcus-memcached-EE/wiki/Response-format
-
pipe ์ฒ๋ฆฌ ์ค
NOT_MY_KEY
๋ฐ์ํ๋ฉด response buffer์NOT_MY_KEY
์๋ต์ ์ถ๊ฐํฉ๋๋ค.- ๋ง์ฝ ์ด ์๋ต์ ์ถ๊ฐํ๋ ๊ณผ์ ์์ ๋ฌธ์ ๊ฐ ์๊ธฐ๋ฉด,
NOT_MY_KEY
์๋ต ๋์
PIPE_ERROR
์๋ต์ด ์ถ๊ฐ๋๋ฉฐ pipe ์ข ๋ฃํ๊ณ ํด๋ผ์ด์ธํธ์๊ฒ ์๋ต์ ๋๋ ค์ค ๊ฒ์ ๋๋ค.
- ๋ง์ฝ ์ด ์๋ต์ ์ถ๊ฐํ๋ ๊ณผ์ ์์ ๋ฌธ์ ๊ฐ ์๊ธฐ๋ฉด,
-
single key pipe ๋ช ๋ น์์ ์ง์ ๋ช ๋ น์ด
NOT_MY_KEY
๋ฐ์ํ์ผ๋ฉด ํ์ฌ ๋ช ๋ น์ swallow(skip) ์ฒ๋ฆฌํฉ๋๋ค.- ์ํํ๋๋ผ๋
NOT_MY_KEY
๋ฐ์ํ ๊ฒ์ด๊ธฐ ๋๋ฌธ์ ๋๋ค.
- ์ํํ๋๋ผ๋
-
NOT_MY_KEY
1ํ ๋ฐ์ ์ดํ ๋ชจ๋ ๋ช ๋ น์ด swallow ์ฒ๋ฆฌ๋๋ฏ๋ก
PIPE_ERROR
๊ฐ ๋ฐ์ํ๋ ์ํฉ(cmd_ovfl, mem_ovfl, ERROR)์ด ์๊ณ ,
๋ง์ง๋ง ๋ช ๋ น๊น์ง swallow ์ฒ๋ฆฌ๋ ํEND
์ ํจ๊ป ํด๋ผ์ด์ธํธ์๊ฒ ์๋ต ๋ฐํํฉ๋๋ค.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
CLIENT_ERROR ์ด๋๋ผ๋ PIPE_ERROR๊น์ง ์ฝ๋๋ก ์์ ํ๋ค๋ฉด,
PIPE_ERROR์ด๋๋ผ๋ redirect ์ํํด์ผ ํ ๊ฒ์
๋๋ค.
๋ฐ๋ผ์, CLIENT_ERROR ์ด๋๋ผ๋ PIPE_ERROR๊น์ง ์ฝ๊ฒ ํ๋ ์์
์ ๋จผ์ ์งํํ๋ ๊ฒ์ด ์ข๊ฒ ์ต๋๋ค.
command overflow์ผ ๋๋ ํด๋น operation์ผ๋ก ์์ฑ๋๋ pipe ๋ช ๋ น์ด์ ๊ฐ์๊ฐ ๋์ผํ๊ธฐ ๋๋ฌธ์ redirectํ์ฌ ๋๊ฐ์ ๋ช ๋ น์ ๋ณด๋ด๋ฉด ์คํจํ๋ฏ๋ก redirectํ ํ์์ฑ์ด ์์ด๋ณด์ ๋๋ค.
memory overflow์ผ ๋๋ ํด๋น ์บ์ ๋ ธ๋์ memory ์ฉ๋์ด ๋ถ์กฑํ๋ ๊ฒ์ด๊ธฐ ๋๋ฌธ์, ๋ค๋ฅธ ๋ ธ๋๋ก redirect๋์์ ๋ ์ฑ๊ณตํ ๊ฐ๋ฅ์ฑ์ด ์์ผ๋ฏ๋ก redirect ์ฒ๋ฆฌํ๋ฉด ๋ ๊ฒ์ผ๋ก ๋ณด์ ๋๋ค.
์์ ๋ด์ฉ์ ๋ํ ์ฝ๋ฉํธ์ ๋๋ค.
- command overflow์์ redirectํ๋ฉด, ์ฒ๋ฆฌ๋์ง ์์ ์ฐ์ฐ๋ง ๋ณด๋ด๋ฏ๋ก ํด๋น ์ฐ์ฐ๋ค์ ์ฑ๊ณตํ ์ ์์ต๋๋ค.
- memory overflow์์ rediretํ๋ ์ด์ ๋ ์ผ๋ถ ์ฐ์ฐ์ด ์ฒ๋ฆฌ๋๊ธฐ ์ ์ ํด๋น key๊ฐ ๋ค๋ฅธ node๋ก ์ด์ ํ๊ธฐ ๋๋ฌธ์ด๋ฉฐ, ์ฒ๋ฆฌ๋์ง ๋ชปํ ์ฐ์ฐ๋ง์ ์ด์ ๋ node๋ก ๋ณด๋ด์ด ์ฒ๋ฆฌํด์ผ ํ๊ธฐ ๋๋ฌธ์ ๋๋ค.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ํด๋ผ์ด์ธํธ๊ฐ ์ ์์ ์ผ๋ก ๋์ํ๋ค๋ฉด command overflow๊ฐ ๋ฐ์ํ์ง ์์ ๊ฒ์ ๋๋ค. ์ ์๊ฐ์๋ command overflow๊ฐ ๋ฐ์ํ๋ฉด redirectํ์ง ์๊ณ ์์ธ๋ฅผ ๋ฐ์์์ผ ๋ฌธ์ ๊ฐ ์๋ ์ํฉ์์ ์๋ฆฌ๋ ๊ฒ์ด ์ข์ ๊ฒ ๊ฐ์ต๋๋ค.
memory overflow์ ๋ํด์๋ ๋ง์ํ์ ์ฌํญ์ ๋์ํฉ๋๋ค.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
๋ฆฌ๋ทฐ ์๋ฃ
latch.countDown(); | ||
} else if (!rv.getOperationStatus().isSuccess()) { | ||
// If this operation failed, remaining subsequent operation | ||
// should not be added and should be marked as cancelled. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
complete() ๋ก์ง์ ์๋ฒฝํ๊ฒ ๋ง๋ค๋ ค๋ฉด,
PIPE_ERROR์ด๋๋ผ๋ ๋ชจ๋ ์๋ต์ ์ฝ์ด๋ด๋ ๊ธฐ๋ฅ์ด ๋จผ์ ๊ตฌํ๋์ด์ผ ํ ๊ฒ ๊ฐ์ต๋๋ค.
์ด๋ฅผ ์ด์๋ก ์ฌ๋ฆฌ๊ณ ์์
์ ์์ํ๋ฉด ์ข๊ฒ ์ต๋๋ค.
|| rv.getOperationStatus().getResponse() == CollectionResponse.CANCELED) { | ||
// countdown if this is last op | ||
latch.countDown(); | ||
} else if (!rv.getOperationStatus().isSuccess()) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PIPE_ERROR command/memory overflow ์ค๋ฅ ์, ์ฒ๋ฆฌ๋์ง ๋ชปํ ์ฐ์ฐ์ ๋ํด
๋ณธ PR ์ฒ๋ผ CANCELED CollectionOperationStatus ์ค์ ํ๋ ๊ฒ์ด ์ข๊ฒ ์ต๋๋ค.
์ ์ฌํญ์ ๊ตฌํํ๊ธฐ ์ํด
์๋ ๋ก์ง์์ cb.receivedStatus()
ํธ์ถ ์์ ์ธ์ ๊ฐ์ด ์์ ๋์ด์ผ ํ ๊ฒ ๊ฐ์ต๋๋ค.
if (line.startsWith("END") || line.startsWith("PIPE_ERROR ")) {
/* ENABLE_MIGRATION if */
if (needRedirect()) {
transitionState(OperationState.REDIRECT);
return;
}
/* ENABLE_MIGRATION end */
cb.receivedStatus((successAll) ? END : FAILED_END);
transitionState(OperationState.COMPLETE);
} else if (line.startsWith("RESPONSE ")) {
์๋์ ๊ฐ์ด ํธ์ถํด์ผ ํ ๊ฒ ๊ฐ์ต๋๋ค.
cb.receivedStatus((index == count && successAll) ? END : FAILED_END);
๋ํ, ์๋ ๊ฒฝ์ฐ์ CENCEL ์ํ๋ฅผ ์ถ๊ฐํ๋ ๋ก์ง๋ ์์ ๋์ด์ผ ํฉ๋๋ค.
if (!rv.getOperationStatus().isSuccess()) {
rv.addEachResult(index + (idx * CollectionPipedInsert.MAX_PIPED_ITEM_COUNT),
new CollectionOperationStatus(false, "CANCELED", CollectionResponse.CANCELED));
latch.countDown();
}
์์ ๊ฐ์ด ๋ณ๊ฒฝ๋๋ฉด,
complete() ๋ก์ง์์์ ๊ฒ์ฌ ์์๋ ๋ณ๊ฒฝ๋์ด์ผ ํฉ๋๋ค.
์๋ ์กฐ๊ฑด์ด ๊ฐ์ฅ ๋จผ์ ๊ฒ์ฌ๋์ด์ผ ํ ๊ฒ์
๋๋ค.
if (!rv.getOperationStatus().isSuccess())
} | ||
} | ||
return false; | ||
return operationStatus.get().getResponse() == CollectionResponse.CANCELED; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
๋ง์ง๋ง op์ ๋ํด isCancelled() ํธ์ถํด์ผ ํ์ง ์๋์?
return ops.get(ops.size() - 1).isCancelled();
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
์ฌ์ฉ์๊ฐ future.cancel์ ํธ์ถํ๊ฑฐ๋ ๋ด๋ถ์ ์ผ๋ก op.cancel์ด ํธ์ถ๋๋ฉด BaseOperationImpl์ cancel ๋ฉ์๋๊ฐ ํธ์ถ๋ฉ๋๋ค.
ํด๋น cancel ๋ฉ์๋ ๋ด๋ถ์์๋ wasCancelled ๋ฉ์๋๋ฅผ ํธ์ถํ๊ฒ ๋๋๋ฐ, ์ด๋ ์์ ํด๋์ค์ธCollectionPipedInsertOperationImpl์ ๊ตฌํ๋์ด ์๊ณ , receivedStatus์ ์ํด future์ ์ํ์ ์ฐ์ด๊ฒ ๋ฉ๋๋ค.
๋ฐ๋ผ์ operationStatus๋ง์ผ๋ก๋ cancel ์ฌ๋ถ๋ฅผ ํ์ธ ๊ฐ๋ฅํฉ๋๋ค.
arcus-java-client/src/main/java/net/spy/memcached/protocol/BaseOperationImpl.java
Lines 95 to 107 in f60ed48
public final boolean cancel(String cause) { | |
if (callbacked.compareAndSet(false, true)) { | |
cancelled = true; | |
if (handlingNode != null) { | |
cause += " @ " + handlingNode.getNodeName(); | |
} | |
cancelCause = "Cancelled (" + cause + ")"; | |
wasCancelled(); | |
callback.complete(); | |
return true; | |
} | |
return false; | |
} |
Lines 182 to 185 in f60ed48
@Override | |
protected void wasCancelled() { | |
getCallback().receivedStatus(STORE_CANCELED); | |
} |
@@ -145,6 +145,10 @@ assert getState() == OperationState.READING | |||
/* ENABLE_MIGRATION end */ | |||
cb.receivedStatus((successAll) ? END : FAILED_END); | |||
transitionState(OperationState.COMPLETE); | |||
} else if (line.startsWith("PIPE_ERROR ")) { | |||
// command flow / memory flow |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
CLIENT_ERROR ์ด๋๋ผ๋ PIPE_ERROR๊น์ง ์ฝ๋๋ก ์์ ํ๋ค๋ฉด,
PIPE_ERROR์ด๋๋ผ๋ redirect ์ํํด์ผ ํ ๊ฒ์
๋๋ค.
๋ฐ๋ผ์, CLIENT_ERROR ์ด๋๋ผ๋ PIPE_ERROR๊น์ง ์ฝ๊ฒ ํ๋ ์์
์ ๋จผ์ ์งํํ๋ ๊ฒ์ด ์ข๊ฒ ์ต๋๋ค.
command overflow์ผ ๋๋ ํด๋น operation์ผ๋ก ์์ฑ๋๋ pipe ๋ช ๋ น์ด์ ๊ฐ์๊ฐ ๋์ผํ๊ธฐ ๋๋ฌธ์ redirectํ์ฌ ๋๊ฐ์ ๋ช ๋ น์ ๋ณด๋ด๋ฉด ์คํจํ๋ฏ๋ก redirectํ ํ์์ฑ์ด ์์ด๋ณด์ ๋๋ค.
memory overflow์ผ ๋๋ ํด๋น ์บ์ ๋ ธ๋์ memory ์ฉ๋์ด ๋ถ์กฑํ๋ ๊ฒ์ด๊ธฐ ๋๋ฌธ์, ๋ค๋ฅธ ๋ ธ๋๋ก redirect๋์์ ๋ ์ฑ๊ณตํ ๊ฐ๋ฅ์ฑ์ด ์์ผ๋ฏ๋ก redirect ์ฒ๋ฆฌํ๋ฉด ๋ ๊ฒ์ผ๋ก ๋ณด์ ๋๋ค.
์์ ๋ด์ฉ์ ๋ํ ์ฝ๋ฉํธ์ ๋๋ค.
- command overflow์์ redirectํ๋ฉด, ์ฒ๋ฆฌ๋์ง ์์ ์ฐ์ฐ๋ง ๋ณด๋ด๋ฏ๋ก ํด๋น ์ฐ์ฐ๋ค์ ์ฑ๊ณตํ ์ ์์ต๋๋ค.
- memory overflow์์ rediretํ๋ ์ด์ ๋ ์ผ๋ถ ์ฐ์ฐ์ด ์ฒ๋ฆฌ๋๊ธฐ ์ ์ ํด๋น key๊ฐ ๋ค๋ฅธ node๋ก ์ด์ ํ๊ธฐ ๋๋ฌธ์ด๋ฉฐ, ์ฒ๋ฆฌ๋์ง ๋ชปํ ์ฐ์ฐ๋ง์ ์ด์ ๋ node๋ก ๋ณด๋ด์ด ์ฒ๋ฆฌํด์ผ ํ๊ธฐ ๋๋ฌธ์ ๋๋ค.
throw new CheckedOperationTimeoutException(duration, unit, elapsed, timedOutOps); | ||
} | ||
Operation lastOp = ops.get(ops.size() - 1); | ||
if (!latch.await(duration, unit) && lastOp.getState() != OperationState.COMPLETE) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
latch.await()
ํธ์ถ ์ ์ lastOp ๊ฐ์ฒด๋ฅผ ๋ฏธ๋ฆฌ ๊ตฌํด๋๋ ๊ฒ์ด ์ด์ํฉ๋๋ค.
latch.await()
๋์์๋ ์๋ก์ด op๊ฐ ์์ฑ๋์ด ๋ฑ๋ก๋ ์ ์๊ธฐ ๋๋ฌธ์
๋๋ค.
๊ทธ๋ฆฌ๊ณ , lastOp๊ฐ ์ค์ future์ ๋ฑ๋ก๋ ๋ง์ง๋ง op๊ฐ ์๋๋ผ
์ค์ ์์ฒญํด์ผ ํ ๋ง์ง๋ง op์ด์ด์ผ ํ์ง ์๋๊ฐ์?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
๊ทธ๋ฆฌ๊ณ , lastOp๊ฐ ์ค์ future์ ๋ฑ๋ก๋ ๋ง์ง๋ง op๊ฐ ์๋๋ผ
์ค์ ์์ฒญํด์ผ ํ ๋ง์ง๋ง op์ด์ด์ผ ํ์ง ์๋๊ฐ์?
์ฌ๋ฌ Op ๊ฐ์ฒด ์ฐ์ฐ ์ํ ์ค ํ๋์ Op์์ ์์ธ๋ ์๋ชป๋ ์๋ต์ด ๋ฐ์ํ๋ฉด future์ ๋ค์ Op๋ฅผ ๋ฃ์ง ์๊ณ latch๋ฅผ countdown์ํต๋๋ค.
๋ฐ๋ผ์ future์์๋ ์ค์ ์์ฒญํด์ผ ํ ๋ง์ง๋ง Op๋ฅผ ํ์ธํ ํ์ ์์ด ์คํ๋์๋ ๊ฐ์ฅ ๋ง์ง๋ง Op๋ง ํ์ธํ๋ ๊ฒ์
๋๋ค.
๐ Related Issue
โจ๏ธ What I did