The following post describes how you can break free of managing servers in AWS while maintaining industry standard security. The key enablers of this approach are S3, Lambda and Security Token Service (STS).

This might be a good time to grab a coffee. There’s quite a lot to cover, but it’s worth it. There’s no servers to manage, and you only pay for the time when your code is actually executing.

Working Demo

Click Here to see a simple Todo List application built using the techniques described below.  You’ll need to register an account before you can get in to play around. Note, you don’t have to provide an email, and I promise not to spam you.

High Level Approach

• The UI of your application is best rendered client side in Javascript which allows you to use a simple, static web server. There are a number of battle hardened frameworks for this (e.g. AngularJS) and it gives the best user experience.
• Amazon S3 provides a robust (although simple) web server. All of the static html, css and js files for your application should be served from S3.
• Your application services for logging in and accessing data will be built as Lambda functions. These functions will read and write from your database and provide JSON responses.
• Cognito is an identity service which is integrated with Lambda. Users authenticated with Cognito are known to the Lambda execution environment when they invoke a function.
• STS will generate temporary AWS credentials (API key and secret key) for users of the application.  These temporary credentials are used by the client application to invoke the AWS API (and thus invoke Lambda).
• DynamoDB provides a fully managed NoSQL database. DynamoDB is not essential for a serverless application, but is used as an example here.

S3 Configuration

Configuring a S3 bucket to host a static website is easy, and is well documented.

The key things to note are:

• The name of your bucket must match the domain name you will use to access the content (eg. “todo.tonyfendall.com”)
• You then need to setup a CName for your domain which points to the bucket endpoint
  todo.tonyfendall.com CNAME http://todo.tonyfendall.com.s3-website-ap-southeast-2.amazonaws.com/

User Authentication

AWS Cognito supports user authentication via a number of different social networks (Facebook, Google+, Twitter, etc), but also supports “Developer Identity” which a term AWS use to describe users authenticated by your application itself.  In this case your application must first decide who the user is (i.e. by verifying a username and password), and then pass your unique identifier for the user to Cognito to get an OpenID token (the same type of token the social networks above issue when a user logs in).

The following snippet of NodeJS code shows how an OpenID token can be obtained within a Lambda function. Note that if Cognito has not seen your user identifier before then a new Identity will be created, otherwise the existing identity for this user will be reused.

if( !validateCredentials(username, password) ) {	
  context.fail("Invalid username or password");
  return;
}

var params = {
  IdentityPoolId: '<Your Cognito Identity Pool Id>',
  Logins: {
    '<Your Cognito Developer Provider Name>': username
  }
};

var aws = require('aws-sdk');
var cognito = new aws.CognitoIdentity();

cognito.getOpenIdTokenForDeveloperIdentity(params, function(err, data) {
  if (err) {
    context.fail(err);
    return;
  }
  var openIDToken = data.Token;
}

Once an OpenID token has been obtained this can be used to obtain temporary AWS credentials from the Security Token Service using the ‘Assume Role with Web Identity’ API call.  The credentials obtained must be associated with an AWS IAM role with the permissions you want the user to have (typically permission to invoke specific Lambda functions).  The following snippet of NodeJS code shows how temporary credentials can be obtained within a Lambda function.

var params = {
  WebIdentityToken: openIDToken,
  RoleArn: 'arn:aws:iam::<account-id>:role/<role-name>',
  // SessionName can be anything unique to this user
  RoleSessionName: username+'_authenticated'
};

var aws = require('aws-sdk');
var sts = new aws.STS();

STS.assumeRoleWithWebIdentity(params, function(err, data) {
  if (err) {
    context.fail(err);
    return;
  }
  
  var credentials = data.Credentials;
  // credentials.AccessKeyId
  // credentials.SecretAccessKey
  // credentials.SessionToken
  // credentials.Expiration
}

Within the Todo sample app the above code exists within a function called loginUser. This function accepts a username and password and returns AWS credentials (that can be used to call secured Lambda functions) if the login is successful. Note that to allow the loginUser function to be invoked, generic AWS credentials (that only allow access to the login function) are hard coded within the client application.

Security Model

It should be noted that the security model described above is the same as any standard web application using session based security (i.e. where users are issued a session token).

• The temporary AWS credentials function as a session id.  They are un-guessable and associated with a specific user and role.
• Lambda automatically picks up the identity of the user from the credentials used to invoke the function (this appears in the variable context.identity).  There’s no opportunity for the user to tamper with this value.
• The permissions of the user are evaluated by AWS before your Lambda function is invoked, so you do not need to manually check the rights of the user in your function.

Using AWS credentials ensures that the base security of your application is as good as the security of AWS.  Enforcement of this security is also outsourced to the AWS API, and AWS have a vested interest in ensuring this is always rock solid.

Lambda Configuration

Lambda code can be written in Javascript, Java or Python. In my experience Java functions have a much slower cold start (60 seconds versus 2 seconds for Javascript) and require more base memory, so it’s worth avoiding Java for any functions which the user will be waiting for in real time.

Functions are uploaded as a zip of uncompiled code although the zip can contain libraries. After uploading your code, you need to specify:

• The runtime to use (Java, NodeJS or Python).
• The handler function within your code to invoke as the entry point. For NodeJS this is the filename plus the variable defined on the exports object. Eg. loginuser.handler (or index.handler if you use the inline editor).
• The AWS role your function should have (if it needs to access other AWS resources such as DynamoDB).
• The memory allocation for your function and the execution timeout limit. Note that Lambda charges you for the memory and CPU time consumed by your function.

Your function receives two arguments. “event” is the data that the caller has passed to your function (may be null), and “context” contains information about the context your function is running within. The following code prints out the identity of the calling user if it is available:

exports.handler = function(event, context) {
  console.log(JSON.stringify(event, null, 2));
  
  if(!context.identity || !context.identity.cognitoIdentityId) {
    context.fail('Caller did not authenticate with Cognito associated credentials');
    return;
  }

  console.log('Caller identity is '+context.identity.cognitoIdentityId);
}

Cognito Configuration

Within Cognito an Identity Pool needs to be created which will store the registered users of your application.

Your identity pool has the following settings:

• Identity pool name – e.g. the name of your app.
• IAM role to give to unauthenticated users.
• IAM role to give to authenticated users.

In the Identity Providers section you also need to click on custom and specify developer provider name. This name is simply a server side identifier for your app that you need to specify in your code (see snippet above).

The IAM role for authenticated users needs to grant the user access to your secure Lambda functions that should only be accessible by logged in users. The IAM policy format to give access to a Lambda function is as follows:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Stmt1447188072000",
            "Effect": "Allow",
            "Action": [
                "lambda:InvokeFunction"
            ],
            "Resource": [
                "arn:aws:lambda:<region>:<account-id>:function:<function-name>"
            ]
        }
    ]
}

Invoking Functions from the Client

The final step is to have your client call your Lambda functions directly when required. The following code snippet will invoke a lambda function using the AWS JavaScript SDK. Remember your initial login function will require static/hard coded access keys, but other functions should use the temporary access keys obtained via Cognito and STS.

// Set credentials for accessing AWS APIs
AWS.config.update({
  accessKeyId: '<access key>',
  secretAccessKey: '<secret key>',
  sessionToken: '<session token>', // only required for temporary credentials
  region: '<region code>'
});
		
var params = {
  FunctionName: '<Lambda function name>',
  LogType: 'Tail',
  InvocationType: 'RequestResponse',
  Payload: JSON.stringify(payload) // If function arguments need to be passed
};

var lambda = new AWS.Lambda();
lambda.invoke(params, callback); // Callback should be function(error, data)

Cost Breakdown

In addition to saving the time and effort required to manage servers, serverless architectures like the one described above are also incredibly cheap in AWS due to only paying for the capacity you need. The simple cost breakdown of this type of application is as follows:

Fixed Costs

Variable Costs

Running this type of application is extremely cheap – especially at moderate loads. The demo todo application costs ~$0.55/month to keep running (mainly just DynamoDB costs).

API Gateway

A large number of online examples exist describing how to link API Gateway endpoints to Lambda functions to build serverless applications. However, in my experience this is completely unnecessary.

API Gateway defines a RESTful HTTP request which can map to your Lambda function. However the AWS SDK allows even easier access to your Lambda functions. The API used by the SDK is of course based on HTTP requests also, just not RESTful ones.

Like most problems, it turns out that plotting points on a map is quite easy if the data is perfect.  However, if the data is imperfect then things get a whole lot trickier.  Below is a description of the problem and how I solved it.

The Data

The data I used was a table of driving distances (in miles) between major US cities.  The US highway system isn’t straight lines from point to point, so this introduces a certain amount of discrepancy into the numbers and ensures that we can’t just plot them out mathematically.  For example, the distance between Miami and New Orleans listed below is 892 miles because the road goes around the cost, however, the straight line distance between the cities across the water is closer to 670 miles.

The Solution

Since we cannot just plot the points out mathematically the next easiest solution is to use a force directed graph approach.  The basic approach is as follows:

1. Lay the points out randomly in 2D space
2. If two points are too far apart, move them a little closer together
   If two points are too close together, move them slightly further apart
3. Repeat step two until the points stop moving or until you get bored of waiting

You can see a working copy of my solution below.  Full source code is available here

Note that the “Total Error” is calculated by comparing the current distance between each pair of cities with the target distance.  If this are going well then this number should get smaller with each iteration of the algorithm.  The smallest total error possible with the highway data above appears to be about 3680 due to the discrepancies in the data mentioned above.

If you need help figuring out what the map should look like you can check here

The Interesting Stuff

This solution is easy to code and gives pretty good results.  However there are a few interesting things that result from this algorithm:

The orientation of the map is random and map is often flipped left to right
The source data is only distances with no direction, so the algorithm can’t tell which way around things should be.

The solution is very sensitive to local minima
Roughly 50% of the time Seattle of Miami will get stuck on the wrong side of the map and because the points repel each other they can’t pass through to be on the correct side.  This is similar in concept to a blind man trying to get to the top of a mountain by always walking uphill.  He will always find a high point, but he may not find the highest point.  A possible solution to this problem would be to run the map simulation several times and then take the best answer found.

It’s hard to know when to stop the simulation
Force directed graphs are generally slow compared to other algorithms so we generally don’t want to run the simulation longer than we have to.  However, knowing when things have stopped and wont get any better can be quite hard.  Often, during the simulation things can seem to become static for a while with very little change between iterations before suddenly two points pop past each other and things begin rapidly improving again.  Knowing when to stop the solution (without having a human watching it) is key to success.

I learnt a lot putting this little app together.  All in all, it was an afternoon well wasted.

In an effort remedy the problem the owners of the building hired an engineer to come in and recommend ways to speed up the elevator service.  The engineer reviewed the existing elevator system and all compatible upgrade options.  Eventually the engineer came back with his recommendation:

The most cost effective way to improve the elevator service was to cover the elevator doors with mirrors.

The idea was unconventional to say the least, but it was so much cheaper than replacing the existing elevators that the building owners decided to try it.  The employees did not realize that no further modifications had been made, and everyone agreed that the new ‘upgraded’ elevators were much faster than the old ones.

The moral of the story is that sometimes it’s easier to simply distract the user while they are waiting than speed up the thing they are waiting for.  In the elevator story the people were distracted by having something to look at while they were waiting (themselves!) and this was enough to make the wait not feel as long.  This story has always stuck with me and I am always on the look out for places where I can use this principle within my applications.

With this goal in mind I created an animating progress bar skin.  The progress bar fills up at the same speed as any other, but as it does so the texture occasionally slides from left to right. This gives the impression that the bar is moving faster than it is and the whole wait feels much more positive

Check out the example below to see the principle in action.  The top progress bar has a regular skin applied and moves painfully slowly.  The bottom progress bar moves at the same speed but has an animating skin applied.  It’s easy to see that it feels like things are progressing much faster.

The full source code is available here.  Feel free to take it and use it in any way which us useful to you.  This skin has been built using Degrafa so you can easily change the colour scheme and use it in other applications.

The below example is something I created while playing with Degrafa Paths (using my movable components library again).  Paths were a huge revelation for me because they allow you to create skins (buttons etc) with arbitrarily complex shapes.  Now able to create speech bubbles and starbursts without having to embed a PNG asset.  That’s really cool

Try it Out:

The full source code is available here.  Feel free to take it and use it in any way which us useful to you.

Luckily for me Facebook provide a prebuilt solution for this which you can drop into your page with only a few lines of Javascript.  They even provide a demo application.

Unluckily for me, after following Facebook’s instructions perfectly, the resizing didn’t work!

The behaviour I observed was:

I confirmed that the demo application Facebook provided worked correctly in all browses, but I just couldn’t get mine to work the same.  Eventually I removed lines from the Facebook demo application one-by-one until I discovered what caused my error.

In the end it turns out that it was simply a case of my application not specifying a DOCTYPE in the main html page.  After I inserted the DOCTYPE tag from the demo application everything worked perfectly.

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">

It’s a pity that Facebook didn’t mention anywhere in their documentation that this line is critical to the operation of the iframe resizing code.  I’m glad things are working for me now though, and hopefully other people will be able to learn from my mistake.

Recently I have been playing around with the source code some more and in particular have been trying to see if I could reuse the movable component libraries I wrote.  As part of this I have tidied the code up a bit and have extracted it from the rest of the application.  Check out the demo below.

How it Works:
The movable component framework is based on the MovableCanvas class.  This class can contain movable components and is responsible for managing selections and copy/paste etc.  Into the MovableCanvas we can place any component we wish as long as it extends the class MovableComponent.  In the example below the image components are MovableComponents which contain an Image.

Try it Out:

The full source code is available here. Feel free to take it and use it in any way which us useful to you.

Things to Look Out For:

• The MovableComponent class contains a public function called clone().  You must override this function in order for the copy/paste functionality to work.
• It is also possible to force MovableComponents to maintain their aspect ratio using the maintainAspectRation property.

The challenge this week was to accurately simulate the movements of a car in 2D.  Most people don’t realize this, but when a car turns it is rotating about a point outside itself, and that point is perpendicular to the wheels.  It’s a little hard to explain, but if you check out the demo below then it all should become clear.

This app was challenging mainly due to the geometry involved, and I definitely had to scratch my head for a while before I remembered how to rotate one point around another.  I’m pretty happy with how it turned out however.  It was definitely an afternoon well spent.

Try it Out:

The full source code is available here. Feel free to take it and use it in any way which us useful to you.

A few people have me why I didn’t allow you to control the car using the arrow keys.  The answer is simply that this demo is more impressive when you can appreciate the analogue nature of car steering.

The strength of being a developer however is in the custom functionality we can add.  Recently I have put together a DrawableOverlay Flex component which can be used to allow the user to annotate/draw on any part of the application.

The example application below shows how this component works in practice.  The DrawableOverlay component can be placed over any other components in the application (in this case a Image and a Container) and the user is then able to annotate anywhere within the DrawableOverlay area.

Try it Out:

The full source code is available here.  Feel free to take it and use it in any way which us useful to you.  Come back soon to see me extend this simple example to show off some of the things which are possible.

This is my opportunity to start afresh and I’m really looking forward to it.

Welcome to my new home on the web, it’s great to have you here!