Errors while configuring proxy for ST2 interaction with JIRA

[zsmanjot]

We have a scenario where in our environment JIRA could not be directly accessible from ST2 but has to first go through proxy.

But when we configure the proxy settings , it does not help and gives error.

All the proxy setting done as part of using JIRA pack is as below under the highlighted files:

When trying running the action using JIRA action it gives error as below.

[root@stackstorm3 ~]# st2 run jira.create_issue summary="TEST" project="OPERATION" type="operations-framework" . id: 66c8881a97a284d989cfbfb2 action.ref: jira.create_issue context.user: st2admin parameters: project: OPERATION summary: TEST type: operations-sopframework status: failed start_timestamp: Fri, 23 Aug 2024 13:01:14 UTC end_timestamp: Fri, 23 Aug 2024 13:01:15 UTC result: exit_code: 1 result: None stderr: "/opt/stackstorm/virtualenvs/jira/lib/python3.8/site-packages/urllib3/connectionpool.py:1099: InsecureRequestWarning: Unverified HTTPS request is being made to host 'XXXXXXXX-bc-proxy1'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings warnings.warn( /opt/stackstorm/virtualenvs/jira/lib/python3.8/site-packages/urllib3/connectionpool.py:1099: InsecureRequestWarning: Unverified HTTPS request is being made to host 'XXXXXXXX-bc-proxy1'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings warnings.warn( Traceback (most recent call last): File "/opt/stackstorm/st2/lib/python3.8/site-packages/python_runner/python_action_wrapper.py", line 395, in <module> obj.run() File "/opt/stackstorm/st2/lib/python3.8/site-packages/python_runner/python_action_wrapper.py", line 214, in run output = action.run(**self._parameters) File "/opt/stackstorm/packs/jira/actions/create_issue.py", line 26, in run issue = self._client.create_issue(fields=data) File "/opt/stackstorm/virtualenvs/jira/lib/python3.8/site-packages/jira/client.py", line 1858, in create_issue r = self._session.post(url, data=json.dumps(data)) File "/opt/stackstorm/virtualenvs/jira/lib/python3.8/site-packages/requests/sessions.py", line 637, in post return self.request("POST", url, data=data, json=json, **kwargs) File "/opt/stackstorm/virtualenvs/jira/lib/python3.8/site-packages/jira/resilientsession.py", line 247, in request elif raise_on_error(response, **processed_kwargs): File "/opt/stackstorm/virtualenvs/jira/lib/python3.8/site-packages/jira/resilientsession.py", line 72, in raise_on_error raise JIRAError( jira.exceptions.JIRAError: JiraError HTTP 400 url: https://jira.xxxxxxxxxxx.com/rest/api/2/issue \t \tresponse headers = {'Date': 'Fri, 23 Aug 2024 13:01:15 GMT', xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx} \tresponse text = {"errorMessages":[],"errors":{"summary":"Field 'summary' cannot be set. It is not on the appropriate screen, or unknown."}} " stdout: '' [root@stackstorm3 ~]#

[skiedude]

I am in no way connected to this pack. But observing the 400 error. Are you able to run a simple jira.get_issue action on an existing issue?

[zsmanjot]

Hi @skiedude

jira.get_issue also does not work with the proxy setup that i am using:

Gives below error:

end_timestamp: Mon, 18 Nov 2024 16:13:56 UTC
result:
exit_code: 1
result: None
stderr: "Traceback (most recent call last):
File "/opt/stackstorm/st2/lib64/python3.8/site-packages/st2common/runners/utils.py", line 132, in get_action_class_instance
action_instance = action_cls(**kwargs)
TypeError: init() got an unexpected keyword argument 'action_service'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/opt/stackstorm/st2/lib/python3.8/site-packages/python_runner/python_action_wrapper.py", line 395, in
obj.run()
File "/opt/stackstorm/st2/lib/python3.8/site-packages/python_runner/python_action_wrapper.py", line 213, in run
action = self._get_action_instance()
File "/opt/stackstorm/st2/lib/python3.8/site-packages/python_runner/python_action_wrapper.py", line 290, in _get_action_instance
action_instance = get_action_class_instance(
File "/opt/stackstorm/st2/lib64/python3.8/site-packages/st2common/runners/utils.py", line 143, in get_action_class_instance
action_instance = action_cls(**kwargs)
File "/opt/stackstorm/packs/jira/actions/lib/base.py", line 18, in init
self._client = self._get_client()
File "/opt/stackstorm/packs/jira/actions/lib/base.py", line 51, in _get_client
client = JIRA(server=config['url'], options={"headers": headers, "verify": config['verify']}, proxies={"http": config['http_proxy'], "https": config['https_proxy']})
File "/opt/stackstorm/virtualenvs/jira/lib/python3.8/site-packages/jira/client.py", line 660, in init
si = self.server_info()
File "/opt/stackstorm/virtualenvs/jira/lib/python3.8/site-packages/jira/client.py", line 3599, in server_info
j = self._get_json("serverInfo")
File "/opt/stackstorm/virtualenvs/jira/lib/python3.8/site-packages/jira/client.py", line 4358, in _get_json
else self._session.get(url, params=params)
File "/opt/stackstorm/virtualenvs/jira/lib/python3.8/site-packages/requests/sessions.py", line 602, in get
return self.request("GET", url, **kwargs)
File "/opt/stackstorm/virtualenvs/jira/lib/python3.8/site-packages/jira/resilientsession.py", line 246, in request
raise exception
File "/opt/stackstorm/virtualenvs/jira/lib/python3.8/site-packages/jira/resilientsession.py", line 224, in request
response = super().request(
File "/opt/stackstorm/virtualenvs/jira/lib/python3.8/site-packages/requests/sessions.py", line 589, in request
resp = self.send(prep, **send_kwargs)
File "/opt/stackstorm/virtualenvs/jira/lib/python3.8/site-packages/requests/sessions.py", line 703, in send
r = adapter.send(request, **kwargs)
File "/opt/stackstorm/virtualenvs/jira/lib/python3.8/site-packages/requests/adapters.py", line 682, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))
"
stdout: ''
[root@stackstorm3 ~]#

[zsmanjot]

Have checked and curl via proxy is working on stackstorm box. It means there is some configuration that needs to be done on pack side so that it could go through proxy.

Although we did mention the use of proxy in files highlighted above but it does not work.

[floatingstatic]

@zsmanjot sorry for the delay. Thanks for testing this with curl. Have you tested the same using the python jira package directly to confirm it works how you expect? Example: https://stackoverflow.com/a/40243355
Unfortunately I don't have an easy way to test this at the moment so hoping you can do some of the lifting on this since you already have a proxy set up.

Also I observe from your initial comment you are using an https url in the http_proxy setting. Is that intentional? Have you tried only passing https_proxy with the https url?

Thanks!

[floatingstatic]

A bit more detail it seems like this functionality is something python-jira forwards on to the requests library here (maybe?): https://github.com/pycontribs/jira/blob/ef10e1a481813fb134ca6232225e4734302418d8/jira/client.py#L4539

Ref: https://requests.readthedocs.io/en/latest/api/#requests.Session.proxies
https://stackoverflow.com/questions/8287628/proxies-with-python-requests-module

The only reason I think we need to dig into this with python-jira is because the st2 pack here just forwards this stuff on to the jira lib, and then requests, etc. If we can figure out the correct invocation and get this working direct in that library it should be easy to adapt the pack if needed to accommodate whatever params we need to forward on (if it is not capable of doing this currently).

[floatingstatic]

Also not sure if this is still relevant but it is open: pycontribs/jira#1138

[zsmanjot]

@zsmanjot sorry for the delay. Thanks for testing this with curl. Have you tested the same using the python jira package directly to confirm it works how you expect? Example: https://stackoverflow.com/a/40243355 Unfortunately I don't have an easy way to test this at the moment so hoping you can do some of the lifting on this since you already have a proxy set up.

Also I observe from your initial comment you are using an https url in the http_proxy setting. Is that intentional? Have you tried only passing https_proxy with the https url?

Thanks!

Hi @floatingstatic

Thanks for jumping on this again. Ya that was typo and not intentional. But the settings on server side are accordingly wherein we are using https url in https_proxy and http url in http_proxy.
Yes tried passing https_proxy with the https url but still it is an issue.
What i suspect is the need for proxy setting in JIRA pack it self. Though we ourselves tried to mention it but not sure if ST2 is going to pick it as it has many dependent libraries.

[zsmanjot]

A bit more detail it seems like this functionality is something python-jira forwards on to the requests library here (maybe?): https://github.com/pycontribs/jira/blob/ef10e1a481813fb134ca6232225e4734302418d8/jira/client.py#L4539

Ref: https://requests.readthedocs.io/en/latest/api/#requests.Session.proxies https://stackoverflow.com/questions/8287628/proxies-with-python-requests-module

The only reason I think we need to dig into this with python-jira is because the st2 pack here just forwards this stuff on to the jira lib, and then requests, etc. If we can figure out the correct invocation and get this working direct in that library it should be easy to adapt the pack if needed to accommodate whatever params we need to forward on (if it is not capable of doing this currently).

Sure @floatingstatic
Let me review these configs and will try to port it and see if it helps.

[zsmanjot]

Hi @floatingstatic

Just to update here, it stands sorted now. We defined environment variable in Stackstorm box and used -e option while testing it out , like below:

st2 run -e jira.get_issue issue_key="OPS-12345"

Without explicilty mentioning this option, it does give errors.

Moreover in workflows we first had to write the core.local task to fetch the env variable and passed it in subsequent task to be used.

[floatingstatic]

@zsmanjot "it stands sorted" meaning we can close this issue? From what I gather the python jira lib may not be plumbing the explicit proxies setting all the way down to requests and the use of environment vars works because its something urllib3 picks up as a dependency of requests. Using -e seems tedious but maybe that works for you. This is why I'd like to confirm that first and foremost the python jira package works when explicitly passed proxies instead of being reliant on environment variables. If it doesn't I'd open an issue with the maintainers of jira. Second to that, assuming that is sorted and working we can look at how to properly plumb this into the python jira package if that is not working from this pack but I suspect its probably not a problem with the pack with config schema overrides.

Either way from what I gather at the moment this is possibly something that may be an issue in the jira package and not this pack. Please confirm how you would like to proceed.

[zsmanjot]

Hi @floatingstatic

This is why I'd like to confirm that first and foremost the python jira package works when explicitly passed proxies instead of being reliant on environment variables.
--> No , with explicit settings in JIRA under '/opt/stackstorm/packs/jira/actions/lib/base.py' did not help. Thinking correct file was targeted.
Since my case demands proxy environment only for JIRA so , it would have been easier to use proxy settings in JIRA pack only.
Now , the resort is to use proxy as an environment variable and to use that variable to pass into input parameters of JIRA action because the pother way it does not work.

For now, if we think environment variable is the only viable and valid solution then i have no issues in closing this case.

[floatingstatic]

@zsmanjot Sorry I mean take this pack completely out of the equation, and test the python library... something like:

from jira.client import JIRA

jira = JIRA(
    options={"server": jira_url},
    basic_auth=(jira_username, jira_password),
    proxies={"https": "https://ip:port"},
)

If you cannot get that to work I'd focus efforts on fixing that in the python library. Sorry for any confusion, I've never used JIRA via a proxy and cannot see what you see so I'm doing my best to try to understand and help guide you to a solution here.

[zsmanjot]

Got you rpoint @floatingstatic . Let me try writing the code and see if it works. Will update here.

[zsmanjot]

Hi @floatingstatic

Tried it with explicit script but it did not help.

Getting below errors:-

Traceback (most recent call last):
File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 662, in urlopen
self._prepare_proxy(conn)
File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 948, in _prepare_proxy
conn.connect()
File "/usr/lib/python3.8/site-packages/urllib3/connection.py", line 342, in connect
self._tunnel()
File "/usr/lib64/python3.8/http/client.py", line 901, in _tunnel
(version, code, message) = response._read_status()
File "/usr/lib64/python3.8/http/client.py", line 277, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File "/usr/lib64/python3.8/socket.py", line 669, in readinto
return self._sock.recv_into(b)
ConnectionResetError: [Errno 104] Connection reset by peer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib/python3.8/site-packages/requests/adapters.py", line 439, in send
resp = conn.urlopen(
File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 719, in urlopen
retries = retries.increment(
File "/usr/lib/python3.8/site-packages/urllib3/util/retry.py", line 436, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='jira.XXXXXXXXX', port=443): Max retries exceeded with url: /rest/api/2/serverInfo (Caused by ProxyError('Cannot connect to proxy.', ConnectionResetError(104, 'Connection reset by peer')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "test_jira.py", line 21, in
jira = JIRA(options=jira_options, basic_auth=(jira_email, jira_api_token))
File "/usr/local/lib/python3.8/site-packages/jira/client.py", line 660, in init
si = self.server_info()
File "/usr/local/lib/python3.8/site-packages/jira/client.py", line 3599, in server_info
j = self._get_json("serverInfo")
File "/usr/local/lib/python3.8/site-packages/jira/client.py", line 4358, in _get_json
else self._session.get(url, params=params)
File "/usr/lib/python3.8/site-packages/requests/sessions.py", line 548, in get
return self.request('GET', url, **kwargs)
File "/usr/local/lib/python3.8/site-packages/jira/resilientsession.py", line 246, in request
raise exception
File "/usr/local/lib/python3.8/site-packages/jira/resilientsession.py", line 224, in request
response = super().request(
File "/usr/lib/python3.8/site-packages/requests/sessions.py", line 535, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python3.8/site-packages/requests/sessions.py", line 648, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python3.8/site-packages/requests/adapters.py", line 510, in send
raise ProxyError(e, request=request)
requests.exceptions.ProxyError: HTTPSConnectionPool(host='jira.XXXXXXXXXXXX', port=443): Max retries exceeded with url: /rest/api/2/serverInfo (Caused by ProxyError('Cannot connect to proxy.', ConnectionResetError(104, 'Connection reset by peer')))
[root@stackstorm4 ]#

[floatingstatic]

This is in your traceback:

jira = JIRA(options=jira_options, basic_auth=(jira_email, jira_api_token))

You are missing proxies (see my example)

[zsmanjot]

No , i have defined it like this:

`# Proxy settings
proxy_dict = {
'http': 'http://xxxxxxxxxxxxxxxx,
'https': 'https://xxxxxxxxxxxxxxxxxxxx'
}

jira_options = {
'server': jira_server,
'proxies': proxy_dict
}

jira = JIRA(options=jira_options, basic_auth=(jira_email, jira_api_token))
`

[floatingstatic]

@zsmanjot I think thats wrong, see: https://github.com/pycontribs/jira/blob/ef10e1a481813fb134ca6232225e4734302418d8/jira/client.py#L480

options and proxies are two different things in the client constructor.

[zsmanjot]

Hi @floatingstatic

Tried it the other way too as you have mentioned but no luck. I will again pick this up and try if i succeed but i am constantly getting errors that it cant connect to proxy.

[floatingstatic]

OK, that sounds like a problem with jira and you can open an issue with them on this here: https://github.com/pycontribs/jira/issues

If that doesn't work there is little we can do about it in this pack given we just wrap everything around the jira python package

[zsmanjot]

Ok sure @floatingstatic

I will open up an issue and follow up there. Thanks for all your help. I will update here once i hear from them.

[skiedude]

A little bit related. Our servers that run stackstorm can't reach the "public" internal facing URL for jira. So we use a DMZ DNS url to reach jira from the servers network.

That means in my config/jira.yaml I'm actually using the alternate DNS name.

---
auth_method: basic
password: password
project: ID
url: https://jira.company.net #this would normally be jira.corp.company.com
username: user
validate: true
verify: false

Have you tried configuring the pack directly with your Proxy URL?

I've submitted a PR to the Jira pip project because they don't handle certain actions when you are using a proxy or alternate URL. This causes certain pack actions here to timeout.
pycontribs/jira#1925