Serverless Architectures using AWS Lambda

lambda As a developer, managing a Linux operating system is pretty far down the list of things I want to be doing. As Werner Vogels said:
“I’ve tried hugging a lot of servers in my life and believe me, they don’t hug you back. They hate you.”

The following post describes how you can break free of managing servers in AWS while maintaining industry standard security. The key enablers of this approach are S3, Lambda and Security Token Service (STS).

This might be a good time to grab a coffee. There’s quite a lot to cover, but it’s worth it. There’s no servers to manage, and you only pay for the time when your code is actually executing.

Working Demo

Click Here to see a simple Todo List application built using the techniques described below.  You’ll need to register an account before you can get in to play around. Note, you don’t have to provide an email, and I promise not to spam you.

High Level Approach

architecture
  • The UI of your application is best rendered client side in Javascript which allows you to use a simple, static web server. There are a number of battle hardened frameworks for this (e.g. AngularJS) and it gives the best user experience.
  • Amazon S3 provides a robust (although simple) web server. All of the static html, css and js files for your application should be served from S3.
  • Your application services for logging in and accessing data will be built as Lambda functions. These functions will read and write from your database and provide JSON responses.
  • Cognito is an identity service which is integrated with Lambda. Users authenticated with Cognito are known to the Lambda execution environment when they invoke a function.
  • STS will generate temporary AWS credentials (API key and secret key) for users of the application.  These temporary credentials are used by the client application to invoke the AWS API (and thus invoke Lambda).
  • DynamoDB provides a fully managed NoSQL database. DynamoDB is not essential for a serverless application, but is used as an example here.

S3 Configuration

Configuring a S3 bucket to host a static website is easy, and is well documented.

The key things to note are:

  • The name of your bucket must match the domain name you will use to access the content (eg. “todo.tonyfendall.com”)
  • You then need to setup a CName for your domain which points to the bucket endpoint
    todo.tonyfendall.com CNAME http://todo.tonyfendall.com.s3-website-ap-southeast-2.amazonaws.com/

User Authentication

AWS Cognito supports user authentication via a number of different social networks (Facebook, Google+, Twitter, etc), but also supports “Developer Identity” which a term AWS use to describe users authenticated by your application itself.  In this case your application must first decide who the user is (i.e. by verifying a username and password), and then pass your unique identifier for the user to Cognito to get an OpenID token (the same type of token the social networks above issue when a user logs in).

The following snippet of NodeJS code shows how an OpenID token can be obtained within a Lambda function. Note that if Cognito has not seen your user identifier before then a new Identity will be created, otherwise the existing identity for this user will be reused.

if( !validateCredentials(username, password) ) {	
  context.fail("Invalid username or password");
  return;
}

var params = {
  IdentityPoolId: '<Your Cognito Identity Pool Id>',
  Logins: {
    '<Your Cognito Developer Provider Name>': username
  }
};

var aws = require('aws-sdk');
var cognito = new aws.CognitoIdentity();

cognito.getOpenIdTokenForDeveloperIdentity(params, function(err, data) {
  if (err) {
    context.fail(err);
    return;
  }
  var openIDToken = data.Token;
}

Once an OpenID token has been obtained this can be used to obtain temporary AWS credentials from the Security Token Service using the ‘Assume Role with Web Identity’ API call.  The credentials obtained must be associated with an AWS IAM role with the permissions you want the user to have (typically permission to invoke specific Lambda functions).  The following snippet of NodeJS code shows how temporary credentials can be obtained within a Lambda function.

var params = {
  WebIdentityToken: openIDToken,
  RoleArn: 'arn:aws:iam::<account-id>:role/<role-name>',
  // SessionName can be anything unique to this user
  RoleSessionName: username+'_authenticated'
};

var aws = require('aws-sdk');
var sts = new aws.STS();

STS.assumeRoleWithWebIdentity(params, function(err, data) {
  if (err) {
    context.fail(err);
    return;
  }
  
  var credentials = data.Credentials;
  // credentials.AccessKeyId
  // credentials.SecretAccessKey
  // credentials.SessionToken
  // credentials.Expiration
}

Within the Todo sample app the above code exists within a function called loginUser. This function accepts a username and password and returns AWS credentials (that can be used to call secured Lambda functions) if the login is successful. Note that to allow the loginUser function to be invoked, generic AWS credentials (that only allow access to the login function) are hard coded within the client application.

Security Model

It should be noted that the security model described above is the same as any standard web application using session based security (i.e. where users are issued a session token).

  • The temporary AWS credentials function as a session id.  They are un-guessable and associated with a specific user and role.
  • Lambda automatically picks up the identity of the user from the credentials used to invoke the function (this appears in the variable context.identity).  There’s no opportunity for the user to tamper with this value.
  • The permissions of the user are evaluated by AWS before your Lambda function is invoked, so you do not need to manually check the rights of the user in your function.

Using AWS credentials ensures that the base security of your application is as good as the security of AWS.  Enforcement of this security is also outsourced to the AWS API, and AWS have a vested interest in ensuring this is always rock solid.

Lambda Configuration

Lambda code can be written in Javascript, Java or Python. In my experience Java functions have a much slower cold start (60 seconds versus 2 seconds for Javascript) and require more base memory, so it’s worth avoiding Java for any functions which the user will be waiting for in real time.

Functions are uploaded as a zip of uncompiled code although the zip can contain libraries. After uploading your code, you need to specify:

  • The runtime to use (Java, NodeJS or Python).
  • The handler function within your code to invoke as the entry point. For NodeJS this is the filename plus the variable defined on the exports object. Eg. loginuser.handler (or index.handler if you use the inline editor).
  • The AWS role your function should have (if it needs to access other AWS resources such as DynamoDB).
  • The memory allocation for your function and the execution timeout limit. Note that Lambda charges you for the memory and CPU time consumed by your function.

Your function receives two arguments. “event” is the data that the caller has passed to your function (may be null), and “context” contains information about the context your function is running within. The following code prints out the identity of the calling user if it is available:

exports.handler = function(event, context) {
  console.log(JSON.stringify(event, null, 2));
  
  if(!context.identity || !context.identity.cognitoIdentityId) {
    context.fail('Caller did not authenticate with Cognito associated credentials');
    return;
  }

  console.log('Caller identity is '+context.identity.cognitoIdentityId);
}

Cognito Configuration

Within Cognito an Identity Pool needs to be created which will store the registered users of your application.

Your identity pool has the following settings:

  • Identity pool name – e.g. the name of your app.
  • IAM role to give to unauthenticated users.
  • IAM role to give to authenticated users.

In the Identity Providers section you also need to click on custom and specify developer provider name. This name is simply a server side identifier for your app that you need to specify in your code (see snippet above).

The IAM role for authenticated users needs to grant the user access to your secure Lambda functions that should only be accessible by logged in users. The IAM policy format to give access to a Lambda function is as follows:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Stmt1447188072000",
            "Effect": "Allow",
            "Action": [
                "lambda:InvokeFunction"
            ],
            "Resource": [
                "arn:aws:lambda:<region>:<account-id>:function:<function-name>"
            ]
        }
    ]
}

Invoking Functions from the Client

The final step is to have your client call your Lambda functions directly when required. The following code snippet will invoke a lambda function using the AWS JavaScript SDK. Remember your initial login function will require static/hard coded access keys, but other functions should use the temporary access keys obtained via Cognito and STS.

// Set credentials for accessing AWS APIs
AWS.config.update({
  accessKeyId: '<access key>',
  secretAccessKey: '<secret key>',
  sessionToken: '<session token>', // only required for temporary credentials
  region: '<region code>'
});
		
var params = {
  FunctionName: '<Lambda function name>',
  LogType: 'Tail',
  InvocationType: 'RequestResponse',
  Payload: JSON.stringify(payload) // If function arguments need to be passed
};

var lambda = new AWS.Lambda();
lambda.invoke(params, callback); // Callback should be function(error, data)

Cost Breakdown

In addition to saving the time and effort required to manage servers, serverless architectures like the one described above are also incredibly cheap in AWS due to only paying for the capacity you need. The simple cost breakdown of this type of application is as follows:

Fixed Costs

Service Monthly Cost
S3 $0.03/GB
DynamoDB Free if less than 25GB of data stored
Lambda No fixed costs
Cognito Free
IAM and STS Free

Variable Costs

Service Monthly Cost
S3 $0.004/10000 requests
$0.09/GB of data transferred (first GB free)
DynamoDB $0.47/requested writes per second capacity
$0.09/requested reads per second capacity
Lambda First million invocations per month free
~$0.000001/invocation there after
Cognito Free
IAM and STS Free

Running this type of application is extremely cheap – especially at moderate loads. The demo todo application costs ~$0.55/month to keep running (mainly just DynamoDB costs).

API Gateway

A large number of online examples exist describing how to link API Gateway endpoints to Lambda functions to build serverless applications. However, in my experience this is completely unnecessary.

API Gateway defines a RESTful HTTP request which can map to your Lambda function. However the AWS SDK allows even easier access to your Lambda functions. The API used by the SDK is of course based on HTTP requests also, just not RESTful ones.