Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add lamda data client support #2224

Merged
merged 18 commits into from
Dec 4, 2024

Conversation

stocaaro
Copy link
Member

@stocaaro stocaaro commented Nov 12, 2024

Problem

Users would like to use the Gen2 data client interface to access their APIs from application functions (lambda's).

Our docs offer instructions on this experience, which stop short of indicating how the customer should provide the model introspection schema needed to make the whole experience work.

Changes

  1. Build the model introspection schema
    1. Build the Model Introspection Schema (MIS) in the data factory
    2. Construct an S3 bucket with the MIS as an object
    3. Grant get permission to the MIS to any function that is granted resource permissions to the data schema
    4. Add the bucket name and object key for the MIS to SSM (where it will be read
  2. Add config function for user ease
    1. Add a configuration function we can internally import from import { getAmplifyClientsConfiguration } from "@aws-amplify/backend/function/runtime";

Example: amplify/functions/todo-count/handler.ts

import type { Handler } from "aws-lambda";
import { Amplify } from "aws-amplify";
import { generateClient } from "aws-amplify/data";
import type { Schema } from "../../data/resource";

import { getAmplifyDataClientConfig } from "@aws-amplify/backend/function/runtime";
import { env } from "$amplify/env/todo-count";

const { resourceConfig, libraryOptions } = await getAmplifyDataClientConfig(env)
Amplify.configure(resourceConfig, libraryOptions);

const client = generateClient<Schema>();

export const handler: Handler = async () => {
  const todos = (await client.models.Todo.list()).data;
  return todos.length;
};

Corresponding docs PR, if applicable:
aws-amplify/docs#8096

Validation

  • Unit tests added
  • Manually tested in an example application
  • E2E tests added for data access and unrelated user use of S3Client

Checklist

  • If this PR includes a functional change to the runtime behavior of the code, I have added or updated automated test coverage for this change.
  • If this PR requires a change to the Project Architecture README, I have included that update in this PR.
  • If this PR requires a docs update, I have linked to that docs PR above.
  • If this PR modifies E2E tests, makes changes to resource provisioning, or makes SDK calls, I have run the PR checks with the run-e2e label set.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Copy link

changeset-bot bot commented Nov 12, 2024

🦋 Changeset detected

Latest commit: 2b7b02f

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 3 packages
Name Type
@aws-amplify/backend Minor
@aws-amplify/backend-function Minor
@aws-amplify/backend-data Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@stocaaro stocaaro marked this pull request as ready for review November 13, 2024 23:12
@stocaaro stocaaro requested review from a team as code owners November 13, 2024 23:12
@stocaaro stocaaro force-pushed the stocaaro/client-schema/function-runtime-config branch from 517f304 to eb41f64 Compare November 14, 2024 15:36
Comment on lines 1 to 11
// eslint-disable-next-line @typescript-eslint/naming-convention
import * as __export__runtime from './runtime/index.js';
// eslint-disable-next-line @typescript-eslint/naming-convention
import * as __export__get_amplify_clients_configuration from './runtime/get_amplify_clients_configuration.js';

/*
Api-extractor does not ([yet](https://github.com/microsoft/rushstack/issues/1596)) support multiple package entry points
Because this package has a submodule export, we are working around this issue by including that export here and directing api-extract to this entry point instead
This allows api-extractor to pick up the submodule exports in its analysis
*/
export { __export__runtime, __export__get_amplify_clients_configuration };

This comment was marked as resolved.

Copy link
Contributor

@Amplifiyer Amplifiyer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes look good to me. The only missing part is an e2e test where we assert that lambda function can access the data client at runtime. E.g.

Copy link
Member

@sobolk sobolk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@stocaaro stocaaro force-pushed the stocaaro/client-schema/function-runtime-config branch from 1ebc7f0 to bf16475 Compare November 15, 2024 16:28
@stocaaro stocaaro force-pushed the stocaaro/client-schema/function-runtime-config branch from bf16475 to d1dec7f Compare November 15, 2024 17:12
@stocaaro

This comment was marked as resolved.

@stocaaro stocaaro changed the title feat: Add backend-function data config with runtime support feat: Add lamda data client support Nov 15, 2024
@stocaaro stocaaro added the run-e2e Label that will include e2e tests in PR checks workflow label Nov 15, 2024
@stocaaro stocaaro removed the run-e2e Label that will include e2e tests in PR checks workflow label Nov 15, 2024
@stocaaro stocaaro added the run-e2e Label that will include e2e tests in PR checks workflow label Nov 18, 2024
Copy link
Member

@sobolk sobolk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

.changeset/brave-cheetahs-repeat.md Outdated Show resolved Hide resolved
sobolk
sobolk previously approved these changes Nov 19, 2024
throw new Error('Data and function project must include api_key');
}

// const dataUrl = clientConfig.data?.url;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is not good.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mean the commented code (removed), or is there something about the error thrown validating the api key that isn't good?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mean the commented code (removed)

Yes.

Comment on lines 162 to 195
await this.assertDataFunctionCallSucceeds(apolloClient);
await this.assertNoopWithImportCallSucceeds(apolloClient);
}

private assertDataFunctionCallSucceeds = async (
apolloClient: ApolloClient<NormalizedCacheObject>
): Promise<void> => {
const response = await apolloClient.query<number>({
query: gql`
query todoCount {
todoCount
}
`,
variables: {},
});

assert.deepEqual(response.data, { todoCount: 0 });
};

private assertNoopWithImportCallSucceeds = async (
apolloClient: ApolloClient<NormalizedCacheObject>
): Promise<void> => {
const response = await apolloClient.query<number>({
query: gql`
query noopImport {
noopImport
}
`,
variables: {},
});

assert.deepEqual(response.data, { noopImport: 'STATIC TEST RESPONSE' });
};
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't seem to see any code that would call the lambda.
If the feature is about lambda accessing appsync, we should be invoking lambda and asseserting that lambda intracted with appsync.

The test as is doesn't prove that lambda is working.

Some ideas how to test lambda can be found here - we assert that lambda interacts with s3 bucket there, similar strategy can be applied here for appsync.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both of these tests call a query, which each call a lambda and the response assertions are specific to the lambda being called (todoCount and noopImport).

I considered testing the lambda's in isolation (without wiring these calls through AppSync query calls), but that use-case is a simpler subset of the use-cases covered by these tests.

Can you help me understand the coverage gap that needs to be filled?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. There's no coverage gap then.

Can you add JSDoc comment to both private assert... methods explaining that they indeed call the lambda and how ? For the reader it's not obvious that GQL calls translate to lambda invocations here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Understood. JSDoc added

@stocaaro stocaaro merged commit 5cbe318 into main Dec 4, 2024
84 of 85 checks passed
@stocaaro stocaaro deleted the stocaaro/client-schema/function-runtime-config branch December 4, 2024 18:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
run-e2e Label that will include e2e tests in PR checks workflow
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants