This post is a bit of a geeky one – I hope it’ll be useful for you, even if you haven’t used AWS yet. This isn’t a technical deep dive post, but I hope I can sell you on the idea of adding reporting to your backups and tempt you to look at serverless hosting.

First let’s define the technical things I’m going to talk about:

• AWS (Amazon Web Services) – cloud based hosting and services for websites
• Lambda (one of the AWS services) – runs your code for you
• S3 bucket (Amazon Simple Storage Service – folders full of files stored on AWS
• API – a way for computers and scripts to talk to each other
• IAM (AWS Identity and Access Management)
• Slack – a group chat application we use as a company to communicate
• DevOps – a software engineering practice that aims at unifying software development and operations with a focus on automation

As a company we have spent the last few months getting our feet wet with AWS in a bigger way than we have in the past. We recently launched a happy SOUNZ into AWS Sydney and they’re just the start of our focus on AWS for our medium-large clients.

Within AWS, Lambda is their ‘serverless compute service’. It runs your code when you want it to run, otherwise it does not charge you. An analogy is takeaway food. You pay for what you eat and you don’t need to buy all the ingredients and have a place to cook it. If you don’t buy takeaways it costs you nothing to have them available and if you suddenly host a party you can easily get a large quantity of food.

As a team we have been using AWS Lambda around the edges of our devops systems to give us a good view of what is happening with our infrastructure. It’s early days, but there’s a lot of potential in the service. I’ll be talking about one of our scripts today, but we have several in use today, and many more planned for development.

Reporting on backups

Backups are one of those things we take for granted will always work – that’s not the case. There’s a lot that can get in the way of a backup working – low disk space and connectivity issues to name a couple. It’s important that we know that our backups are working and alert a human if they are not.

We store our backups in an S3 bucket (folder) replicated between geographic regions and so we monitor that place. We decided that a good first pass at this would be a daily script that looks at all the backups in our S3 bucket and ensures there’s a recent backup for all our client projects – both the database and files. You can see the result of that running below.

Lambda backups script displaying success ticks in our Slack channel

We love Ruby, but Lambda does not support Ruby as of writing this so instead we wrote the script in Python. There’s a couple of good reasons to use Python – you get a built in library for using the AWS API (boto) and it’s faster to start up than node.js. Python 2 and 3 are supported.

Benefits of running a script like this with AWS Lambda

You could run this script on any server with Python but there are a few advantages to AWS.

Cost

Lambda is very cheap or even free. This script runs for around 3 minutes per day and we are charged only for this running time, but because Lambda offers a free tier that does not expire after 12 months we are currently running our Lambda scripts for free.

Access control

Because we are running this inside AWS, monitoring an AWS bucket, we can use IAM rules rather than access keys for access control, which is a real bonus for us as it makes the rules around who can access what much cleaner.

Trigger types

We run our script Monday-Friday at 5am, but in the future we could trigger it using an API for a reporting tool, or have it verify backups right after they are uploaded instead of daily with events. We’ve got a lot of potential to grow the script.

Error reporting

If any of our Lambda functions fail, we have a rule that will alert the Ops team at the company and look into it. It took 5 minutes to set that up.

Looking forward

Lambda has a lot of potential if your use case for it is right. We have been discussing using it for standalone tasks such as image resizing, data processing, and basic API calls.and it would be a good fit for that. By combining it with other AWS services you can make something pretty nifty with a minimal amount of code and a low ongoing infrastructure cost that scales with use. Lambda is also a high-availability service, and since it is an application platform managed by AWS itself, has a lower operational maintenance cost.

As a company we’re not adopting it for our client projects yet, but we are using it ourselves so we can confidently discuss the pros and cons when a client asks about it or when we see the right opportunity.

Serverless in future projects

We can see a lot of future potential in the use of Lambda in that it offers a lot of benefit when you need to scale resources and cost as demand grows. Great examples could include chat bots and other apps that heavily interact with third-party APIs. Lambda could mean delivery of custom apps with scalability being very easy.

If you’re using Lambda with success we’d love to hear what you’re doing and lessons learned.