Use Aurora Serverless with AWS Lambda and RDS Data API

This lab will show you how to connect to and interact with Amazon Aurora Serverless database clusters using AWS Lambda functions and the RDS Data API.

This lab contains the following tasks:

  1. Create a Lambda execution role
  2. Create a Lambda function
  3. Connect to the database using the RDS Data API
  4. Observe Aurora Serverless Cluster Scaling up and down

1. Create a Lambda execution role

Before you create an AWS Lambda function, you need to configure an IAM execution role. This will contain the permissions you are granting the function to interact with AWS resources via the APIs. Open the Identity and Access Management (IAM) service console. Choose Roles from the left hand side menu, if it isn’t already selected, and click Create role.

rdsconsole

In the Select type of trusted entity section, choose AWS service. Next, in the Choose a use case section, choose Lambda, then click Next: Permissions.

rdsconsole

Click the Create policy button in the Attach permissions policies section.

rdsconsole

In the new browser tab that opens up, toggle to the JSON interface tab. Delete the existing content, paste the policy listed below in the JSON editor, and substitute the [SecretARN] placeholder with the ARN of the secret you created in the previous lab. Click Review policy.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogGroup",
                "logs:CreateLogStream",
                "logs:PutLogEvents"
            ],
            "Resource": [
                "arn:aws:logs:*:*:*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "rds-data:*"
            ],
            "Resource": [
                "*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "secretsmanager:GetSecretValue"
            ],
            "Resource": [
                "[SecretARN]"
            ]
        }
    ]
}

rdsconsole

Assign the IAM policy the name auroralab-serverless-policy, then click Create policy.

rdsconsole

Once the policy has been created successfully, you can return to the other browser tab and continue configuring the role. You can also close the policy browser tab.

rdsconsole

Back on the browser tab for creating the role, click the refresh icon in the top right of the policy list, then use the filter input field to search for the name of the policy you have just created. Select that policy, and click Next: tags.

rdsconsole

Skip the Add tags section, and click Next: Review. Then assign the role the name auroralab-serverless-role, and click Create role.

rdsconsole

2. Create a Lambda function

Open the AWS Lambda service console.

Choose Functions from the left hand side menu, if it isn’t already selected, and click Create function.

rdsconsole

Choose the option to Author from scratch, set the Function name to auroralab-serverless-function and select Node.js 12.x for Runtime. Under Permissions, expand the sub-section called Change default execution role. In the Execution role dropdown, select Use an existing role, then in the Existing role dropdown, select the execution role you have created previously, named auroralab-serverless-role. Click Create function.

rdsconsole

Make sure the Configuration tab is selected. In the Function code section, paste the code snipped below into the editor, and change the placeholders as follows:

Placeholder Description Where to find it
[ClusterARN] The ARN of the serverless database cluster resource. RDS Data API will establish connectivity with this database on your behalf. See the previous lab: Create an Aurora Serverless DB Cluster at step 1. Create a serverless DB cluster.
[SecretARN] The ARN of the secret used to store the database credentials. RDS Data API will access this secret and connect to the database using those credentials. See the previous lab: Create an Aurora Serverless DB Cluster at step 2. Create a secret to store the credentials.
// require the AWS SDK
const AWS = require('aws-sdk')
const rdsDataService = new AWS.RDSDataService()

exports.handler = (event, context, callback) => {
  // prepare SQL command
  let sqlParams = {
    secretArn: '[SecretARN]',
    resourceArn: '[ClusterARN]',
    sql: 'select relname from pg_class;',
    database: 'mylab',
    includeResultMetadata: true
  }

  // run SQL command
  rdsDataService.executeStatement(sqlParams, function (err, data) {
    if (err) {
      // error
      console.log(err)
      callback('Query Failed')
    } else {
      // init
      var rows = []
      var cols =[]

      // build an array of columns
      data.columnMetadata.map((v, i) => {
        cols.push(v.name)
      });

      // build an array of rows: { key=>value }
      data.records.map((r) => {
        var row = {}
        r.map((v, i) => {
          if (v.stringValue !== "undefined") { row[cols[i]] = v.stringValue; }
          else if (v.blobValue !== "undefined") { row[cols[i]] = v.blobValue; }
          else if (v.doubleValue !== "undefined") { row[cols[i]] = v.doubleValue; }
          else if (v.longValue !== "undefined") { row[cols[i]] = v.longValue; }
          else if (v.booleanValue !== "undefined") { row[cols[i]] = v.booleanValue; }
          else if (v.isNull) { row[cols[i]] = null; }
        })
        rows.push(row)
      })

      // done
      console.log('Found rows: ' + rows.length)
      callback(null, rows)
    }
  })
}

Click Deploy to save your code changes.

rdsconsole

Scroll down to the Basic settings section, and click the Edit button.

rdsconsole

Change the function Timeout to 1 min 0 sec, then click Save. We are increasing the timeout as it will take longer to respond to the first query against the serverless DB cluster. The compute capacity will be allocated only when the first request is received.

rdsconsole

3. Connect to the database using the RDS Data API

Now you are ready to connect to the database from a Lambda function, by using the RDS Data API. The function doesn’t bundle a database driver, it simply uses a RESTful AWS API call to send the SQL query: select relname from pg_class; and retrieves the result as a JSON data structure. This is accomplished in a minimal number of lines of code.

Execute the function by clicking the Test button.

You will be asked to configure a test event the first time you try. The format and content for the test event are not relevant for this exercise, so pick any Event template, such as Hello World, input a memorable Event name, such as MyTestEvent and click Create.

rdsconsole

If you have created a new test event, you may need to click the Test button again. After the function has completed running you can review the results, and see the function response. You may need to expand the Details sub-section of the Execution result: succeeded notification to see the results.

rdsconsole

Your first attempt to test the function may fail. In certain cases running the first command and activating the serverless DB cluster for the first time can take several seconds longer than the function or internal Data API timeouts. Try again and if you are still experiencing difficulties, notify one of the workshop support staff.

4. Observe Aurora Serverless Cluster Scaling up and down

Go back the RDS Dashboard, select Databases on the left menu and click the Aurora Serverless Cluster name.

rdsconsole

Select the Monitoring tab and click on the Serverless Database Capacity (Count) graph to check the Serverless Database Capacity Cloudwatch metrics for the Aurora Serverless cluster.

rdsconsole

You will notice that Aurora capacity units (ACU) automatically scaled to 8 units when it received requests from the Lambda function. If you observe the same metrics after sometime, you will notice that the ACU automatically scaled down to 0 gradually because we specified to pause the cluster after 5 minutes of inactivity.

rdsconsole