Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Segments not appearing in local deep-storage (PVC) #177

Open
prajapati-rasik opened this issue Sep 3, 2024 · 0 comments
Open

Segments not appearing in local deep-storage (PVC) #177

prajapati-rasik opened this issue Sep 3, 2024 · 0 comments

Comments

@prajapati-rasik
Copy link

Deployment : Kubernetes cluster (with zookeeper ) using druid-operator
Metadata store : mysql
Deep storage : Multi-attach PVC
Druid version : 30.0.0

I am running a cluster using tiny-cluster.yaml with addition to 3 middlemanagers node. When i am doing batch ingestion through tasks api, segment creation and publishing is not happing, though the task is showing success. Deepstorage PVC is empty, historical loaddropqueue is also empty and I don't see any errors in logs also.

I am able to create directories in PVC, so don't think its problem with PVC. What am i doing wrong here?

Also I am ingesting a json only using a python library. Is there any guidelines on correct way to format data to pass it in ingestion schema?

In druid-io/druid-operator#277, I see that it is suggested to use hdfs instead of local for clustered environment, is it still the case?

NAME                                                 READY   AGE
statefulset.apps/druid-tiny-cluster-brokers          1/1     9m50s
statefulset.apps/druid-tiny-cluster-coordinators     1/1     9m19s
statefulset.apps/druid-tiny-cluster-historicals      1/1     13m
statefulset.apps/druid-tiny-cluster-middlemanagers   3/3     13m
statefulset.apps/druid-tiny-cluster-routers          1/1     9m18s
statefulset.apps/tiny-cluster-zk                     1/1     3h30m
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant