python lambda debug: missing "debugpy" with "buildArguments: --manifest..."

[xarpy]

System details (run the AWS: About Toolkit command)

• OS: Ubuntu 22.04.3 LTS
• Visual Studio Code version: 1.84.2
• AWS Toolkit version: v1.99.0

Question

Hi guys, I'm having some problems compiling the project in debug mode, with the AWS Toolkit, due to versioning and structure problems in the project, made in python. We are currently fixing all these points, however we still can't compile the project, could you help us?

Below is the AWS_TOOLKIT log

2023-11-23 09:25:31 [INFO]: log level: info
2023-11-23 09:25:31 [INFO]: endpoints: retrieving AWS endpoints data
2023-11-23 09:25:31 [INFO]: OS: Linux x64 6.2.0-37-generic
2023-11-23 09:25:31 [INFO]: Visual Studio Code extension host:  1.84.2
2023-11-23 09:25:31 [INFO]: AWS Toolkit:  1.99.0
2023-11-23 09:25:31 [INFO]: node: 18.15.0
2023-11-23 09:25:31 [INFO]: electron: 25.9.2
2023-11-23 09:25:32 [INFO]: telemetry cache not found: '/home/xarpy/.config/Code/User/globalStorage/amazonwebservices.aws-toolkit-vscode/telemetryCache'
2023-11-23 09:25:32 [WARN]: AwsContext: no default region in credentials profile, falling back to us-east-1: 
2023-11-23 09:30:32 [INFO]: telemetry: sent batch (size=9)
2023-11-23 09:36:30 [INFO]: CloudFormationTemplateRegistry: building with: Watching **/*.{yaml,yml}, Untitled Files, Excluding /.*devfile\.(yaml|yml)/i, /.*[/\\]\.aws-sam([/\\].*|$)/
2023-11-23 09:36:30 [INFO]: CloudFormationTemplateRegistry: failed to process: file:///home/xarpy/project/payments/.pre-commit-config.yaml
2023-11-23 09:36:30 [INFO]: CloudFormationTemplateRegistry: failed to process: file:///home/xarpy/project/payments/specs/api/bau.yaml
2023-11-23 09:36:30 [INFO]: CloudFormationTemplateRegistry: failed to process: file:///home/xarpy/project/payments/specs/api/baul-webhook.yaml
2023-11-23 09:36:30 [INFO]: CloudFormationTemplateRegistry: processed 1 items, skipped 3
2023-11-23 09:36:30 [WARN]: SAM debug: missing AWS credentials (Toolkit is not connected)
2023-11-23 09:36:30 [INFO]: SAM CLI location (version: 1.103.0): /usr/local/bin/sam
2023-11-23 09:36:31 [INFO]: Preparing to debug locally: Lambda "app.lambda_handler"
2023-11-23 09:36:31 [INFO]: Building SAM application...
2023-11-23 09:36:31 [INFO]: SAM CLI not configured, using SAM found at: '/usr/local/bin/sam'
2023-11-23 09:36:31 [INFO]: Command: (not started) [/usr/local/bin/sam build --build-dir /tmp/aws-toolkit-vscode/vsctkTzYQ18/output --template /tmp/aws-toolkit-vscode/vsctkTzYQ18/app___vsctk___template.yaml --base-dir /home/xarpy/project/payments --manifest /tmp/aws-toolkit-vscode/vsctkTzYQ18/debug-requirements.txt --manifest requirements/dev.in]
2023-11-23 09:36:33 [INFO]: Build complete.
2023-11-23 09:36:33 [INFO]: Starting SAM application locally
2023-11-23 09:36:33 [INFO]: SAM CLI not configured, using SAM found at: '/usr/local/bin/sam'
2023-11-23 09:36:33 [INFO]: AWS.running.command
2023-11-23 09:36:33 [INFO]: Command: (not started) [/usr/local/bin/sam local invoke payments --template /tmp/aws-toolkit-vscode/vsctkTzYQ18/output/template.yaml --env-vars /tmp/aws-toolkit-vscode/vsctkTzYQ18/env-vars.json -d 5858 --debugger-path /home/xarpy/.vscode/extensions/amazonwebservices.aws-toolkit-vscode-1.99.0/resources/debugger --debug-args /tmp/lambci_debug_files/py_debug_wrapper.py --listen 0.0.0.0:5858 --wait-for-client --log-to-stderr]
2023-11-23 09:36:54 [INFO]: Attaching debugger to SAM application...
2023-11-23 09:37:10 [ERROR]: SamLaunchRequestError: Failed to attach debugger [DebuggerRetryLimit]
2023-11-23 09:40:33 [INFO]: telemetry: sent batch (size=3)

Another log, from output on vscode is this:

23 Nov 2023 12:36:52,306 [ERROR] (rapid) Init failed error=Runtime exited with error: exit status 1 InvokeID=
Prepending Lambda task root to path: /var/task
Starting debugger...
2023-11-23 09:36:52 [ERROR]: Traceback (most recent call last):
  File "/tmp/lambci_debug_files/py_debug_wrapper.py", line 46, in <module>
    runpy.run_module('debugpy', run_name='__main__')
  File "/var/lang/lib/python3.10/runpy.py", line 220, in run_module
    mod_name, mod_spec, code = _get_module_details(mod_name)
  File "/var/lang/lib/python3.10/runpy.py", line 140, in _get_module_details
    raise error("No module named { raw: true }" % mod_name)
ImportError: No module named debugpy

END RequestId: 1a4ade06-b5d7-4c23-ac01-95f2f65bd988
REPORT RequestId: 1a4ade06-b5d7-4c23-ac01-95f2f65bd988	Init Duration: 0.13 ms	Duration: 43.80 ms	Billed Duration: 44 ms	Memory Size: 128 MB	Max Memory Used: 128 MB	


Command stopped: "sam local invoke"
2023-11-23 09:36:54 [INFO]: Attaching debugger to SAM application...

Even though I've already inserted the debugpy package into the requirements, this hasn't changed.

And my launch.json is that:

{
    // Use o IntelliSense para saber mais sobre os atributos possíveis.
    // Focalizar para exibir as descrições dos atributos existentes.
    // Para obter mais informações, acesse: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "type": "aws-sam",
            "request": "direct-invoke",
            "name": "BP:Payments",
            "invokeTarget": {
                "target": "code",
                "projectRoot": "payments",
                "lambdaHandler": "app.lambda_handler"
                // "templatePath": "${workspaceFolder}/payments/template.yaml",
                // "logicalId": "BankPaymentStack"
            },
            "lambda": {
                "runtime": "python3.10",
                "payload": {},
                "environmentVariables": {"ENV":"Dev"}
            },
            "sam":{
                "buildArguments": ["--manifest", "requirements/dev.in"]
            }
        }
    ]
}

Yes, we have a template.yml, but it's more confusing to understand than the code itself, so much so that I can't compile the project either from the template.yml or from the code itself, and every time the system crashes it shows a connection error, unable to connect to port 5858.

[justinmk3]

It looks like sam build succeeded so this probably isn't a build issue, but a sam arguments issue.

  "sam":{
    "buildArguments": ["--manifest", "requirements/dev.in"]
  }

Try without that, it may be conflicting with the one set by the Toolkit (which tries to inject debugpy automatically):

aws-toolkit-vscode/src/shared/sam/debugger/pythonSamDebug.ts

Line 151 in 0b63c71

manifestPath = await makePythonDebugManifest({

Are you using "layers"? #2468

[xarpy]

Hi @justinmk3 , after few hours, trying to fixing and restructure another parts on the project, I suppose to be fixed the first issue, but right now has another. I was tried as like you said, but doesn't make difference, when excluded the sam context from launch.json.

Yes, I rebuilt the template, as well as updating the path to the requirements, being a folder, making it easier to organize development and production environments, which is the reason why I use the --manifest parameter.

And about the layers. I'm not sure, but now with the CDK CLI, the AWS Toolkit, in the vscode, told me that I just needed to run the cdk synth command, and now I can see the resources, on the api lambda.

Here is the launch.json remade and the new error logs.

Output log

2023-11-24 13:54:16 [INFO]: Preparing to debug locally: Lambda "app.sqs_handler"
2023-11-24 13:54:16 [INFO]: Building SAM application...
2023-11-24 13:54:16 [INFO]: Command: (not started) [/usr/local/bin/sam build --build-dir /tmp/aws-toolkit-vscode/vsctkmEKf7W/output --template /tmp/aws-toolkit-vscode/vsctkmEKf7W/app___vsctk___template.yaml --base-dir /home/xarpy/project/payments/ --manifest /tmp/aws-toolkit-vscode/vsctkmEKf7W/debug-requirements.txt --manifest /home/xarpy/project/payments/requirements/dev.in]
2023-11-24 13:54:16 [INFO]: Building codeuri: /home/xarpy/project/payments/ runtime: python3.10 metadata: {} architecture: x86_64 functions: payaments

2023-11-24 13:54:16 [INFO]:  Running PythonPipBuilder:ResolveDependencies

2023-11-24 13:54:17 [INFO]: 
Build Failed

2023-11-24 13:54:18 [INFO]: Error: PythonPipBuilder:ResolveDependencies - Failed to find a Python runtime containing pip on the PATH.

New Launch.json

{
    // Use o IntelliSense para saber mais sobre os atributos possíveis.
    // Focalizar para exibir as descrições dos atributos existentes.
    // Para obter mais informações, acesse: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "type": "aws-sam",
            "request": "direct-invoke",
            "name": "BP:Payments",
            "invokeTarget": {
                "target": "code",
                "projectRoot": "${workspaceFolder}/payments",
                "lambdaHandler": "app.sqs_handler"
                // "templatePath": "${workspaceFolder}/payments/template.yaml",
                // "logicalId": "BankPaymentsCbStack"
            },
            "lambda": {
                "runtime": "python3.10",
                "payload": {},
                "environmentVariables": {"ENV":"Dev"}
            },
            "sam":{
                "buildArguments": ["--manifest", "${workspaceFolder}/payments/requirements/dev.in"]
            }
        }
    ]
}

[justinmk3]

2023-11-24 13:54:18 [INFO]: Error: PythonPipBuilder:ResolveDependencies - Failed to find a Python runtime containing pip on the PATH.

That message is from sam.

• One way to solve it is to use a "container build", so that sam builds the project in the container instead of on your local machine. Set sam.containerBuild = true in launch.json https://docs.aws.amazon.com/toolkit-for-vscode/latest/userguide/serverless-apps-run-debug-config-ref.html
• Another option is to install pip or make sure your $PATH is giving sam the correct python.
  • If you are using python virtualenv, then that needs to be considered also.
  • https://github.com/search?q=repo%3Aaws%2Faws-sam-cli+python+pip&type=issues

[xarpy]

Hello @justinmk3 , I tried the first option, it almost worked, but it broke internally with this error below. About the second option, yes we use several different python envs, how can I indicate to sam my python env easily in this case? I don't see any reference for this on the documentation...

2023-11-24 17:26:35 [INFO]: Preparing to debug locally: Lambda "app.sqs_handler"
2023-11-24 17:26:35 [INFO]: Building SAM application...
2023-11-24 17:26:35 [INFO]: Command: (not started) [/usr/local/bin/sam build --build-dir /tmp/aws-toolkit-vscode/vsctk0yO0eS/output --template /tmp/aws-toolkit-vscode/vsctk0yO0eS/app___vsctk___template.yaml --base-dir /home/xarpy/project/payments --use-container --manifest /tmp/aws-toolkit-vscode/vsctk0yO0eS/debug-requirements.txt --manifest /home/xarpy/project/payments/requirements/dev.in]
2023-11-24 17:26:36 [INFO]: Starting Build inside a container

2023-11-24 17:26:36 [INFO]: Building codeuri: /home/xarpy/project/payments runtime: python3.10 metadata: {} architecture: x86_64 functions: payments

2023-11-24 17:26:39 [INFO]: 
Fetching public.ecr.aws/sam/build-python3.10:latest-x86_64 Docker container image....
2023-11-24 17:26:39 [INFO]: ..
2023-11-24 17:26:39 [INFO]: 
Mounting /home/xarpy/project/payments as /tmp/samcli/source:ro,delegated, inside runtime container

2023-11-24 17:27:03 [INFO]: 
Build Succeeded

Built Artifacts  : output
Built Template   : output/template.yaml

Commands you can use next
=========================
[*] Validate SAM template: sam validate
[*] Invoke Function: sam local invoke -t output/template.yaml
[*] Test Function in the Cloud: sam sync --stack-name {{stack-name}} --watch
[*] Deploy: sam deploy --guided --template-file output/template.yaml

2023-11-24 17:27:04 [INFO]:  Running PythonPipBuilder:ResolveDependencies
 Running PythonPipBuilder:CopySource

2023-11-24 17:27:04 [INFO]: Build complete.
2023-11-24 17:27:04 [INFO]: Starting SAM application locally
2023-11-24 17:27:04 [INFO]: AWS.running.command
Invoking app.sqs_handler (python3.10)
Requested to skip pulling images ...

Mounting /tmp/aws-toolkit-vscode/vsctk0yO0eS/output/payments as /var/task:ro,delegated, inside runtime container
START RequestId: 135e6d4b-ecca-4332-877d-8c1310b38028 Version: $LATEST
Prepending Lambda task root to path: /var/task
Starting debugger...
Debugger waiting for client...
2023-11-24 17:27:10 [INFO]: Attaching debugger to SAM application...
2023-11-24 17:27:11 [INFO]: Debugger attached
[ERROR] FileNotFoundError: [Errno 2] No such file or directory: 'node'
Traceback (most recent call last):
  File "/var/lang/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/var/task/app.py", line 8, in <module>
    import aws_cdk as cdk
  File "/var/task/aws_cdk/__init__.py", line 1422, in <module>
    from ._jsii import *
  File "/var/task/aws_cdk/_jsii/__init__.py", line 13, in <module>
    import aws_cdk.asset_awscli_v1._jsii
  File "/var/task/aws_cdk/asset_awscli_v1/_jsii/__init__.py", line 13, in <module>
    __jsii_assembly__ = jsii.JSIIAssembly.load(
  File "/var/task/jsii/_runtime.py", line 55, in load
    _kernel.load(assembly.name, assembly.version, os.fspath(assembly_path))
  File "/var/task/jsii/_kernel/__init__.py", line 299, in load
    self.provider.load(LoadRequest(name=name, version=version, tarball=tarball))
  File "/var/task/jsii/_kernel/providers/process.py", line 354, in load
    return self._process.send(request, LoadResponse)
  File "/var/task/jsii/_utils.py", line 23, in wrapped
    stored.append(fgetter(self))
  File "/var/task/jsii/_kernel/providers/process.py", line 349, in _process
    process.start()
  File "/var/task/jsii/_kernel/providers/process.py", line 260, in start
    self._process = subprocess.Popen(
  File "/var/lang/lib/python3.10/subprocess.py", line 971, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/var/lang/lib/python3.10/subprocess.py", line 1863, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
Exception ignored in: <function _NodeProcess.__del__ at 0x7f251a44ea70>
Traceback (most recent call last):
  File "/var/task/jsii/_kernel/providers/process.py", line 228, in __del__
    self.stop()
  File "/var/task/jsii/_kernel/providers/process.py", line 291, in stop
    assert self._process.stdin is not None
AttributeError: '_NodeProcess' object has no attribute '_process'
24 Nov 2023 20:27:30,344 [ERROR] (rapid) Init failed error=Runtime exited with error: exit status 1 InvokeID=
Prepending Lambda task root to path: /var/task
Starting debugger...
2023-11-24 17:27:30 [ERROR]: Traceback (most recent call last):
  File "/tmp/lambci_debug_files/py_debug_wrapper.py", line 46, in <module>
    runpy.run_module('debugpy', run_name='__main__')
  File "/var/lang/lib/python3.10/runpy.py", line 220, in run_module
    mod_name, mod_spec, code = _get_module_details(mod_name)
  File "/var/lang/lib/python3.10/runpy.py", line 140, in _get_module_details
    raise error("No module named { raw: true }" % mod_name)
ImportError: No module named debugpy

END RequestId: 011c188a-a68c-469f-9de5-275957d899cd
REPORT RequestId: 011c188a-a68c-469f-9de5-275957d899cd	Init Duration: 0.14 ms	Duration: 20045.22 ms	Billed Duration: 20046 ms	Memory Size: 128 MB	Max Memory Used: 128 MB	


Command stopped: "sam local invoke"