Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

I'm not able to mock s3 with typescript #265

Open
victorgsp opened this issue Jan 28, 2022 · 5 comments
Open

I'm not able to mock s3 with typescript #265

victorgsp opened this issue Jan 28, 2022 · 5 comments
Labels
question A question needs to be answered before progress can be made on this issue technical A technical issue that requires understanding of the code, infrastructure or dependencies

Comments

@victorgsp
Copy link

victorgsp commented Jan 28, 2022

Hi guys, i'm not able to mock s3 with typescript

packages version:

"aws-sdk": "2.1065.0"
"aws-sdk-mock": "^5.6.2"

my code

  import * as AWS from "aws-sdk";
  class S3Provider {
    public async getS3ObjectList() {
      const s3 = new AWS.S3();
  
      const params: AWS.S3.ListObjectsV2Request = {
        Bucket: Environment.RAW_DATA_BUCKET_NAME,
      };
      const list: AWS.S3.ListObjectsV2Output = await s3
        .listObjectsV2(params)
        .promise();
      return list;
    }

my test

...
import * as AWSMock from "aws-sdk-mock";
import * as AWS from "aws-sdk";

describe("Mytest", () => {
 it("should return a correct value", async () => {
   AWSMock.setSDKInstance(AWS);

   AWSMock.mock("S3", "listObjectsV2", output);

   const provider = new getS3ObjectList();
   const result = await provider.getS3ObjectList();
   ...

Error:

UriParameterError: Expected uri parameter to have length >= 1, but found "" for params.Bucket

  at ParamValidator.fail (node_modules/aws-sdk/lib/param_validator.js:50:37)
  at ParamValidator.validateUri (node_modules/aws-sdk/lib/param_validator.js:167:14)
  at ParamValidator.validateString (node_modules/aws-sdk/lib/param_validator.js:160:12)
  at ParamValidator.validateScalar (node_modules/aws-sdk/lib/param_validator.js:132:21)
  at ParamValidator.validateMember (node_modules/aws-sdk/lib/param_validator.js:96:21)
  at ParamValidator.validateStructure (node_modules/aws-sdk/lib/param_validator.js:77:14)
  at ParamValidator.validateMember (node_modules/aws-sdk/lib/param_validator.js:90:21)
  at ParamValidator.validate (node_modules/aws-sdk/lib/param_validator.js:34:10)
  at Request.VALIDATE_PARAMETERS (node_modules/aws-sdk/lib/event_listeners.js:132:42)
  at Request.callListeners (node_modules/aws-sdk/lib/sequential_executor.js:106:20)

`

@nelsonic nelsonic added question A question needs to be answered before progress can be made on this issue technical A technical issue that requires understanding of the code, infrastructure or dependencies labels Jan 28, 2022
@thomaux
Copy link
Collaborator

thomaux commented Jan 31, 2022

Hi @victorgsp, if I'm not mistaken the mock lib will still perform parameter validation. You're getting this error because the environment variable Environment.RAW_DATA_BUCKET_NAME is likely not set in your tests

@victorgsp
Copy link
Author

Hi @victorgsp, if I'm not mistaken the mock lib will still perform parameter validation. You're getting this error because the environment variable Environment.RAW_DATA_BUCKET_NAME is likely not set in your tests

Hi @thomaux, thanks for the answer. Following your explanation, I put the code below in my test suite:

beforeAll(() => {    
     process.env =  Object.assign(process.env, {    
 	    RAW_DATA_BUCKET_NAME:  "notexistbucketname",    
     });    
 });

Now I received the following error:
NoSuchBucket: The specified bucket does not exist

Apparently, my test is using the real aws-sdk instead of the mock.

@thomaux
Copy link
Collaborator

thomaux commented Jan 31, 2022

@victorgsp it does seem like that yes. Is the full code publicly available so I could have a look? If not, could you maybe share the full files for the source and test?

The mock needs to be set before the AWS service is created, from your code snippet that does seem to happen, so I'm unsure where it's going wrong at the moment

@victorgsp
Copy link
Author

@victorgsp it does seem like that yes. Is the full code publicly available so I could have a look? If not, could you maybe share the full files for the source and test?

The mock needs to be set before the AWS service is created, from your code snippet that does seem to happen, so I'm unsure where it's going wrong at the moment

It's not a public repository, but the files are with following content:

S3Provider.ts

import * as AWS from "aws-sdk";
import { Environment } from "../../../core/commons/constants/environment";

class S3Provider {
  public async getS3ObjectList() {
    const s3 = new AWS.S3();

    const params: AWS.S3.ListObjectsV2Request = {
      Bucket: Environment.RAW_DATA_BUCKET_NAME,
    };
    const list: AWS.S3.ListObjectsV2Output = await s3
      .listObjectsV2(params)
      .promise();
    return list;
  }
}

export { S3Provider };

S3Provider.spec.ts

import * as AWS from "aws-sdk";
import * as AWSMock from "aws-sdk-mock";
import { ListObjectsV2Output } from "aws-sdk/clients/s3";
import { S3Provider } from "../S3Provider";

const output: ListObjectsV2Output = {
  Contents: [
    {
      Key: "PACKAGE_NAME|2|2011-10-05T14:48:00.000Z|[email protected]/",
    },
    {
      Key: "PACKAGE_NAME|2|2011-10-05T14:48:00.000Z|[email protected]/File1.txt",
    },
    {
      Key: "PACKAGE_NAME|2|2011-10-05T14:48:00.000Z|[email protected]/File2.txt",
    },
  ],
};

describe("RawDataFileProvider", () => {
  beforeAll(() => {
    process.env = Object.assign(process.env, {
      RAW_DATA_BUCKET_NAME: "notexistbucketname",
    });
  });
  it("should return the correct RawDataPackages", async () => {
    AWSMock.restore();
    AWSMock.setSDKInstance(AWS);

    AWSMock.mock("S3", "listObjectsV2", output);

    const provider = new S3Provider();
    const result = await provider.getS3ObjectList();

    expect(result).toEqual({
      packages: [
        {
          numberOfFiles: 2,
          packageName: "PACKAGE_NAME",
          uploadDate: new Date("2011-10-05T14:48:00.000Z"),
          userEmail: "[email protected]",
          updateCompleted: false,
          files: [
            {
              name: "PACKAGE_NAME|2|2011-10-05T14:48:00.000Z|[email protected]/File1.txt",
              isUploadedFile: false,
            },
            {
              name: "PACKAGE_NAME|2|2011-10-05T14:48:00.000Z|[email protected]/File2.txt",
              isUploadedFile: false,
            },
          ],
        },
      ],
    });
  });
});

@thomaux
Copy link
Collaborator

thomaux commented Jan 31, 2022

Hmm strange, it works when I copy both those files locally, with the following configuration:

package.json

{
  "name": "sandbox",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "jest"
  },
  "author": "",
  "license": "ISC",
  "devDependencies": {
    "@types/jest": "^27.4.0",
    "aws-sdk-mock": "^5.6.2",
    "jest": "^27.4.7",
    "ts-jest": "^27.1.3",
    "typescript": "^4.5.5"
  }
}

jest.config.js

module.exports = {
    collectCoverage: false,
    preset: 'ts-jest',
    testEnvironment: 'node',
    testRegex: '(.*spec.ts)$',
    transform: {
      '.ts': 'ts-jest',
    },
    verbose: true,
  };

Running npm test results in this output:

 FAIL  ./S3Provider.spec.ts (6.204 s)
  RawDataFileProvider
    ✕ should return the correct RawDataPackages (17 ms)

  ● RawDataFileProvider › should return the correct RawDataPackages

    expect(received).toEqual(expected) // deep equality

    - Expected  - 13
    + Received  +  5

      Object {
    -   "packages": Array [
    +   "Contents": Array [
          Object {
    -       "files": Array [
    +       "Key": "PACKAGE_NAME|2|2011-10-05T14:48:00.000Z|[email protected]/",
    +     },
          Object {
    -           "isUploadedFile": false,
    -           "name": "PACKAGE_NAME|2|2011-10-05T14:48:00.000Z|[email protected]/File1.txt",
    +       "Key": "PACKAGE_NAME|2|2011-10-05T14:48:00.000Z|[email protected]/File1.txt",
          },
          Object {
    -           "isUploadedFile": false,
    -           "name": "PACKAGE_NAME|2|2011-10-05T14:48:00.000Z|[email protected]/File2.txt",
    -         },
    -       ],
    -       "numberOfFiles": 2,
    -       "packageName": "PACKAGE_NAME",
    -       "updateCompleted": false,
    -       "uploadDate": 2011-10-05T14:48:00.000Z,
    -       "userEmail": "[email protected]",
    +       "Key": "PACKAGE_NAME|2|2011-10-05T14:48:00.000Z|[email protected]/File2.txt",
          },
        ],
      }

      34 |         const result = await provider.getS3ObjectList();
      35 |
    > 36 |         expect(result).toEqual({
         |                        ^
      37 |             packages: [
      38 |                 {
      39 |                     numberOfFiles: 2,

      at S3Provider.spec.ts:36:24
      at fulfilled (S3Provider.spec.ts:5:58)

Test Suites: 1 failed, 1 total
Tests:       1 failed, 1 total
Snapshots:   0 total
Time:        6.266 s
Ran all test suites.
npm ERR! Test failed.  See above for more details.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question A question needs to be answered before progress can be made on this issue technical A technical issue that requires understanding of the code, infrastructure or dependencies
Projects
None yet
Development

No branches or pull requests

3 participants