llm sherpa deployed in eks cluster with 4vcpu and 16gb ram not working properly

[gireesh99]

I have an EKS cluster and my api is running on it and llmsherpa is running on a seperate worker node in the cluster with llm sherpa as the only pod running on this worker node.

I am facing this issue:
raise ConnectionError(err, request=request) requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

my manifest file :
apiVersion: apps/v1
kind: Deployment
metadata:
name: nlm-ingestor # Replace with a more descriptive name if needed
spec:
replicas: 1 # Adjust the number of replicas as needed
selector:
matchLabels:
app: nlm-ingestor
template:
metadata:
labels:
app: nlm-ingestor
spec:
nodeSelector:
node_type: nlm-ingestor
containers:
- name: nlm-ingestor
image: ghcr.io/nlmatics/nlm-ingestor:latest
ports:
- containerPort: 5001
name: nlm-ingestor
protocol: TCP

 ----------------------------
 my service :
 apiVersion: v1

kind: Service
metadata:
name: nlm-ingestor-service
spec:
selector:
app: nlm-ingestor
ports:

• protocol: TCP
  port: 5010
  targetPort: 5001
  nodePort: 30006
  name: nlm-ingestor
  type: NodePort

and my llmsherpa url is llmsherpa_api_url = "http://nlm-ingestor-service:5010/api/parseDocument?renderFormat=all"

if the file is very small is it is working, if the file is larger than 1mb it is raising the above error.

any leads what's the issue or how to resolve this would be a great help!

[ansukla]

There is no size restriction on the service side. This could be happening because of some restrictions in the eks front (load balancer etc.) component that is in between the client and the running server. Fact that it cuts off at 1mb which is a very small size points to that being the reason.

…

On Thu, Jul 4, 2024 at 12:07 AM gireesh99 ***@***.***> wrote: I have an EKS cluster and my api is running on it and llmsherpa is running on a seperate worker node in the cluster with llm sherpa as the only pod running on this worker node. I am facing this issue: raise ConnectionError(err, request=request) requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) my manifest file : apiVersion: apps/v1 kind: Deployment metadata: name: nlm-ingestor # Replace with a more descriptive name if needed spec: replicas: 1 # Adjust the number of replicas as needed selector: matchLabels: app: nlm-ingestor template: metadata: labels: app: nlm-ingestor spec: nodeSelector: node_type: nlm-ingestor containers: - name: nlm-ingestor image: ghcr.io/nlmatics/nlm-ingestor:latest ports: - containerPort: 5001 name: nlm-ingestor protocol: TCP ---------------------------- my service : apiVersion: v1 kind: Service metadata: name: nlm-ingestor-service spec: selector: app: nlm-ingestor ports: - protocol: TCP port: 5010 targetPort: 5001 nodePort: 30006 name: nlm-ingestor type: NodePort and my llmsherpa url is llmsherpa_api_url = " http://nlm-ingestor-service:5010/api/parseDocument?renderFormat=all" if the file is very small is it is working, if the file is larger than 1mb it is raising the above error. any leads what's the issue or how to resolve this would be a great help! — Reply to this email directly, view it on GitHub <#74>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AALJTIW7XFRCGV3NRLYE6G3ZKTDBHAVCNFSM6AAAAABKKXIDZOVHI2DSMVQWIX3LMV43ASLTON2WKOZSGM4DSOBZGI3DAMI> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>