Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Redis socket closed unexpectedly #158

Open
morganney opened this issue Jan 6, 2024 · 0 comments
Open

Redis socket closed unexpectedly #158

morganney opened this issue Jan 6, 2024 · 0 comments
Labels
api Related to the api package question Further information is requested redis

Comments

@morganney
Copy link
Owner

This has been occurring in the test environment since initial deploy. There is plenty of references about it online, here is on example from SO. As long as the error is handled, the connection with redis is re-established, but the cause is still unknown.

Looking at the log timestamps, they seem to be occurring almost exactly every hour:

morgan@mactop busmap (develop) $ copilot svc logs -n api -e test --start-time 2024-01-06T18:00:00.272Z
copilot/api/3f10a59ec6fb4 {"level":"ERROR","time":"2024-01-06T19:16:32.272Z","pid":1,"hostname":"ip-foo.us-west-2.compute.internal","err":{"type":"SocketClosedUnexpectedlyError","message":"Socket closed unexpectedly","stack":"Error: Socket closed unexpectedly\n    at Socket.<anonymous> (/app/node_modules/@redis/client/dist/lib/client/socket.js:194:118)\n    at Object.onceWrapper (node:events:629:26)\n    at Socket.emit (node:events:514:28)\n    at TCP.<anonymous> (node:net:337:12)"},"msg":"Redis unexpected error."}
copilot/api/3f10a59ec6fb4 {"level":"INFO","time":"2024-01-06T19:16:32.273Z","pid":1,"hostname":"ip-foo.us-west-2.compute.internal","msg":"Redis reconnecting."}
copilot/api/3f10a59ec6fb4 {"level":"ERROR","time":"2024-01-06T20:16:32.276Z","pid":1,"hostname":"ip-foo.us-west-2.compute.internal","err":{"type":"SocketClosedUnexpectedlyError","message":"Socket closed unexpectedly","stack":"Error: Socket closed unexpectedly\n    at Socket.<anonymous> (/app/node_modules/@redis/client/dist/lib/client/socket.js:194:118)\n    at Object.onceWrapper (node:events:629:26)\n    at Socket.emit (node:events:514:28)\n    at TCP.<anonymous> (node:net:337:12)"},"msg":"Redis unexpected error."}
copilot/api/3f10a59ec6fb4 {"level":"INFO","time":"2024-01-06T20:16:32.277Z","pid":1,"hostname":"ip-foo.us-west-2.compute.internal","msg":"Redis reconnecting."}
copilot/api/3f10a59ec6fb4 {"level":"ERROR","time":"2024-01-06T21:16:32.284Z","pid":1,"hostname":"ip-foo.us-west-2.compute.internal","err":{"type":"SocketClosedUnexpectedlyError","message":"Socket closed unexpectedly","stack":"Error: Socket closed unexpectedly\n    at Socket.<anonymous> (/app/node_modules/@redis/client/dist/lib/client/socket.js:194:118)\n    at Object.onceWrapper (node:events:629:26)\n    at Socket.emit (node:events:514:28)\n    at TCP.<anonymous> (node:net:337:12)"},"msg":"Redis unexpected error."}
copilot/api/3f10a59ec6fb4 {"level":"INFO","time":"2024-01-06T21:16:32.286Z","pid":1,"hostname":"ip-foo.us-west-2.compute.internal","msg":"Redis reconnecting."}

The database was not initialized with a schema, so there should not be any session activity in terms of writing through the redis socket. There is probably a default timeout in play somewhere that is being triggered, and the socket closed whereupon a new one is regenerated based on the underlying default strategy employed by node-redis.

Google "Error: Socket closed unexpectedly" to get plenty of other tidbits.

Find the underlying cause, see if there is a reasonable fix, or if the solution is to simply accept there is a timeout (or that we want there to be one), understand why this occurs, live with it and move on.

@morganney morganney added question Further information is requested api Related to the api package redis labels Jan 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api Related to the api package question Further information is requested redis
Projects
None yet
Development

No branches or pull requests

1 participant