Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[#569] fix stored connections to match desired connection timeout #570

Draft
wants to merge 8 commits into
base: main
Choose a base branch
from
Draft

[#569] fix stored connections to match desired connection timeout #570

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
2a55bc2
[_569] fix stored connections to match desired connection timeout
d-w-moore Jun 20, 2024
ee0e215
[_569] allow disabling connection timeouts
d-w-moore Jun 21, 2024
8aae500
review correction
d-w-moore Jun 24, 2024
b4e9f2d
setting connection_timeout to zero is an error
d-w-moore Jun 24, 2024
f8ae904
README changes for connection_timeouts
d-w-moore Jun 24, 2024
e642053
testing
d-w-moore Jun 25, 2024
9d48444
correction
d-w-moore Jun 25, 2024
6577667
more corrections
d-w-moore Jun 25, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions irods/pool.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,13 @@ def method_(self,*s,**kw):

DEFAULT_APPLICATION_NAME = 'python-irodsclient'

def _adjust_timeout_to_pool_default(conn):
set_timeout = conn.socket.gettimeout()
desired_value = conn.pool.connection_timeout
if desired_value == set_timeout:
return
conn.socket.settimeout(desired_value)

class Pool(object):

def __init__(self, account, application_name='', connection_refresh_time=-1, session = None):
Expand Down Expand Up @@ -57,6 +64,7 @@ def _conn(self, conn_): setattr( self._thread_local, "_conn", conn_)

@attribute_from_return_value("_conn")
def get_connection(self):
new_conn = False
with self._lock:
try:
conn = self.idle.pop()
Expand All @@ -73,6 +81,7 @@ def get_connection(self):
# code more predictable as we are not relying on when garbage collector is called
conn.disconnect()
conn = Connection(self, self.account)
new_conn = True
logger.debug("Created new connection with id: {}".format(id(conn)))
except KeyError:
conn = Connection(self, self.account)
d-w-moore marked this conversation as resolved.
Show resolved Hide resolved
Expand All @@ -87,6 +96,9 @@ def get_connection(self):

logger.debug("Adding connection with id {} to active set".format(id(conn)))

if not new_conn:
_adjust_timeout_to_pool_default(conn)

logger.debug('num active: {}'.format(len(self.active)))
logger.debug('num idle: {}'.format(len(self.idle)))

Expand Down
3 changes: 1 addition & 2 deletions irods/session.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -350,8 +350,7 @@ def connection_timeout(self):
@connection_timeout.setter
def connection_timeout(self, seconds):
self._cached_connection_timeout = seconds
if seconds is not None:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Many systems also use 0 to indicate a non-value (or forever)… consider checking for 0 or None?

Copy link
Collaborator Author

@d-w-moore d-w-moore Jun 21, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is a good point as evidently 0 means set nonblocking, and I believe we want to disallow that.
None , however, in my opinion should be permissible as it is (for all intents) equivalent to a very large number.

Copy link
Collaborator Author

@d-w-moore d-w-moore Jun 21, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In other words session.connection_timeout = None is clear and correct in intent, if you take it to mean "eliminate timeouts totally for the given session."

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed that None conveys the intent.

I was suggesting that if this is a user-facing setting, that setting it to either 0 or None would be equivalent and correct. If this is not user-facing, then... carry on.

Copy link
Collaborator Author

@d-w-moore d-w-moore Jun 21, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The docs now encourage the use of:

session_object.connection_timeout = <some_number>

as a way to dictate how long the client socket should wait until declaring a server peer as down and giving up, as it were.

So, yes, it is user-facing....

But 0 and None are not equivalent here, actually. 0 would invalidate the socket for further use by the PRC, by turning on non-blocking behavior (ie null-length values could now be read from the socket when it has not yet received any bytes of the expected server response) whereas None declares we never want to declare the server peer down. i.e. "turn off timeouts completely"

Copy link
Collaborator Author

@d-w-moore d-w-moore Jun 21, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If Windows is a different case, though, I should re-evaluate

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No worries - if None is the only correct answer - please consider leaving a comment somewhere user-facing... that will let them know that 0 behaves differently and why.

Copy link
Collaborator Author

@d-w-moore d-w-moore Jun 21, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. I'll update the docs to caution against setting 0 (which doesn't yet cause an immediate error , but should) or even too low a value, as the latter would have an effect down the line of raising a NetworkError the first time the server "responds too slowly"... which (as long as the server is alive and still working) would be technically incorrect practice on the application writer's part.

self.pool.connection_timeout = seconds
self.pool.connection_timeout = seconds

@staticmethod
def get_irods_password_file():
Expand Down