Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ENHANCE: Moved cache list change logic from IO thread to CacheManger thread. #649

Open
wants to merge 1 commit into
base: develop
Choose a base branch
from
Open

ENHANCE: Moved cache list change logic from IO thread to CacheManger thread. #649

wants to merge 1 commit into from

Conversation

brido4125
Copy link
Collaborator

@brido4125 brido4125 commented Aug 4, 2023 •
edited
Loading

Motivation

현재 로직에서는 IO 쓰레드가
hash ring의 update로직을 수행한다.
이를 다른 CacheManager 데몬 쓰레드에게 이관하게
IO 쓰레드의 작업량을 감소시킨다.

관련 이슈

https://github.com/jam2in/arcus-works/issues/406

@brido4125 brido4125 self-assigned this Aug 4, 2023
@uhm0311

This comment was marked as resolved.

Sign in to view
@brido4125

This comment was marked as resolved.

Sign in to view
@uhm0311

This comment was marked as resolved.

Sign in to view
@brido4125 brido4125 force-pushed the enhance/updateHash branch 2 times, most recently from 9a955aa to 139665b Compare August 9, 2023 01:18
@brido4125 brido4125 requested a review from uhm0311 August 9, 2023 01:27
@brido4125

This comment was marked as resolved.

Sign in to view
@brido4125 brido4125 marked this pull request as draft August 18, 2023 08:16
@brido4125 brido4125 marked this pull request as ready for review August 21, 2023 03:03
@jhpark816

This comment was marked as resolved.

Sign in to view
@oliviarla

This comment was marked as resolved.

Sign in to view
@jhpark816

This comment was marked as resolved.

Sign in to view
oliviarla

This comment was marked as resolved.

Sign in to view

@brido4125 brido4125 force-pushed the enhance/updateHash branch 2 times, most recently from 3e3a46d to 3a75ede Compare March 11, 2024 07:56
jhpark816
jhpark816 reviewed Mar 21, 2024
View reviewed changes
Copy link
Collaborator

@jhpark816 jhpark816 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

리뷰 완료

src/main/java/net/spy/memcached/CacheManager.java Show resolved Hide resolved
src/main/java/net/spy/memcached/CacheManager.java Show resolved Hide resolved
src/main/java/net/spy/memcached/CacheManager.java Show resolved Hide resolved
oliviarla

This comment was marked as resolved.

Sign in to view

@brido4125

This comment was marked as resolved.

Sign in to view
@uhm0311

This comment was marked as resolved.

Sign in to view
jhpark816
jhpark816 reviewed Mar 29, 2024
View reviewed changes
Copy link
Collaborator

@jhpark816 jhpark816 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

리뷰 완료

src/main/java/net/spy/memcached/CacheManager.java Show resolved Hide resolved
@uhm0311 uhm0311 force-pushed the develop branch 3 times, most recently from 07918e6 to ae2a82f Compare April 2, 2024 07:53
@brido4125 brido4125 marked this pull request as draft April 3, 2024 03:13
@brido4125 brido4125 marked this pull request as ready for review April 3, 2024 04:58
jhpark816
jhpark816 reviewed Apr 18, 2024
View reviewed changes
Copy link
Collaborator

@jhpark816 jhpark816 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

일부 리뷰

src/main/java/net/spy/memcached/CacheManager.java Show resolved Hide resolved
src/main/java/net/spy/memcached/CacheManager.java

synchronized (this) {
notifyAll(); // awake CacheManager Thread.
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

closing() 메소드가 같은 일을 합니다.
wakeup() 으로 용어를 변경하고, wakeup() 호출합시다.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

alter_list 변경 시에는 wakeup 시키지 않나요?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

alter_list 변경 시에는 wakeup 시키지 않나요?

processAlterListResult 콜백에 의해 commandAlterListChange가 호출되면
MemcachedConnection.setAlterNodesChange에 의해 wakeup 됩니다.

cacheList와 alterList의 변경을 감지하는 콜백 메서드가 별도로 구성됩니다.

alterList 변경 반영의 경우 주키퍼 클라이언트의 스레드에 의해 로직이 수행됩니다.

jhpark816
jhpark816 reviewed Apr 18, 2024
View reviewed changes
src/main/java/net/spy/memcached/CacheManager.java
@@ -398,6 +398,17 @@ public void run() {
waitBeforeRetryMonitorCheck(1000L - elapsed);
}
}

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@brido4125 @uhm0311

아래 코멘트가 hide된 곳에 들어가서, 다시 코멘트합니다.

wait() 중에 notifyAll()에 의해 깨어나더라도 다시 wait() 하는 코드가 있습니다.
동작의 의미는 유지하면서 wait()는 한 곳에만 수행되게 해 주세요.

즉, waitBeforeRetryMonitorCheck() 메소드를 2회 호출하지 않아야 합니다.

jhpark816
jhpark816 reviewed Apr 18, 2024
View reviewed changes
src/main/java/net/spy/memcached/CacheManager.java
getLogger().error("Cache List update error in ArcusClient {}, exception = {}",
ac.getName(), e.getCause());
}
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@brido4125 @uhm0311

중요한 논의가 필요한 부분입니다.

기존엔 각 client의 io thread가 동시에 hash ring update 진행하였지만
현재는 cache manager thread 혼자서 모든 client의 hash ring update 진행합니다. 따라서, update가 반영되는 데 있어 지연이 생길 수 있습니다.

Cache Manager에 locator 하나만을 두고 이용하는 방안을 강구할 수 있나요?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

이는 제가 예전에 리뷰했던 내용과 일치하며, 변경 사항이 아주 많아질 예정이기 때문에 별도의 PR로 반영한다는 대답을 들었습니다.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

언급되었던 사항이군요.

현재 PR이 기존 동작보다 hashring update 지연을 발생시킬 수 있어,
Single hashring 유지하는 방안이 현재 PR 범위에 포함되어야 할 것 같습니다.

Single hahsring 유지 방안을 PoC로 진행해 보는 것이 좋겠습니다.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@uhm0311 uhm0311 uhm0311 approved these changes

@jhpark816 jhpark816 jhpark816 left review comments

@oliviarla oliviarla oliviarla approved these changes

Assignees

@brido4125 brido4125

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@brido4125 @uhm0311 @jhpark816 @oliviarla