reopening #1872 chalice deploy command failing with ResourceConflictException UpdateFunctionConfiguration exception

[kraj007]

Reopening previous issue again
#1872
@jamesls

This is blocking for me. I am already using the latest chalice version 1.26.4 , still getting this issue. I saw similar kind of issue for claudiajs.
claudiajs/claudia#226

Also checked , https://aws.amazon.com/blogs/compute/coming-soon-expansion-of-aws-lambda-states-to-all-functions/

one of the options suggested is to use "aws:states:opt-out" in description but in case of chalice it internally creating lambda and then tries to update. How can I pass description in chalice deploy command.

We are in kind of similar situation of claudiajs/claudia#226 where we deployed the stack on one environment and next deployment onwards its continuously failing.
also to confirm , I had discussion with AWS premium support , they suggested to check with you guys on this.

[kraj007]

Hello Team,

Can anybody please help me on this issue ?

[kraj007]

we got one workaround from AWS support to update the lambda description and add "aws:states:opt-out" as mentioned here
https://aws.amazon.com/blogs/compute/coming-soon-expansion-of-aws-lambda-states-to-all-functions/

This workaround worked as of now as in first attempt our lambda got created however next time we ran our jenkins pipelines it was failing.
We do not have option to provide the lambda description in chalice deploy command. So not sure if this workaround will work every time.

Also @jamesls can you please check once here

chalice/chalice/awsclient.py

Line 913 in 61f4345

if result['LastUpdateStatus'] != 'Successful':

if this getting evaluated ? I think we need to check like " inside ' for condition
if if result['LastUpdateStatus'] != '"Successful"'

[jamesls]

I'm currently looking into this, but just wanted to quickly comment that I'd double check you are using the latest version 1.26.4. I noticed that when I compared the line numbers in your stack trace to the version of awsclient.py at version 1.26.4 (shown here), the line numbers don't match the method calls which suggests you're using an earlier version. Maybe printing a chalice --version before your chalice deploy command would help confirm this.

Also, if you are using the latest versions of Chalice, you would be seeing a different error if updating the Lambda function fails. Here's the relevant code snippet:

chalice/chalice/awsclient.py

Lines 913 to 921 in 65557cc

	if result['LastUpdateStatus'] != 'Successful':
	self._wait_for_function_update(function_name)
	return result
	def _wait_for_function_update(self, function_name):
	# type: (str) -> None
	client = self._client('lambda')
	waiter = client.get_waiter('function_updated')
	waiter.wait(FunctionName=function_name)

This means that we're waiting until the function is active before calling the the UpdateFunctionConfiguration operation, which is the API call that's failing in your error logs. If anything went wrong, I would expect to see a failure in the .wait() API call, which would say something like we timed out waiting for the function update to be successful.

In order for this error to happen we'd have to make it past these lines of code, which would mean we saw the LastUpdateStatus be Successful, but for some reason not have it actually be ready to be updated again. That would be really unexpected if that was the case, but first let's confirm you're using the latest version of Chalice.

[kraj007]

@jamesls I have added logging to check the chalice version and found that jenkins instance while chalice deploy command using older chalice version 1.22.1. This is strange issue in our case as all our requirements files have explicitly using chalice .26.4 version.

We have now explicitly doing pip install before chalice deploy again with latest version and I can confirm this issue is solved as of now. Meanwhile our devops team checking why older version is present in jenkins instances.

[jamesls]

Awesome, glad you were able to get everything working for you!