Learn from our blog

We love sharing our knowledge about all things tech and community. 

The Twelve Days of AWS: Redshift

To talk about Redshift we probably need to talk about Data Warehousing, as Redshift is a fully managed Data Warehouse package. This is not putting your data in a ‘digital warehouse’ to go and gather virtual dust on the back of a virtual shelf somewhere and is more akin to a single source of truth for the state of a business and its information. Whilst Redshift is based on PostgreSQL and can be queried with normal SQL, it is not meant to be a DataBase per se.

12 Days of AWS Day 12 written around snowflakes with a penguin wearing a candy cane sweater

The Twelve Days of AWS: Data Pipelines

Extract, Transform & Load (ETL) is the name of the game when it comes to Data Pipelines. 

The Extract portion will acquire data from some source(s), which will then pass through Transform where some alteration may be needed to that data, to then end up Loaded into another storage format, such as Redshift, S3, to name a couple.

12 Days of AWS Day 10 written around snowflakes with two penguins holding balloons

The Twelve Days of AWS: EC2

Despite being what most people think about when mentioning AWS, I left their Elastic Compute Cloud (EC2) until now, mostly because personally I don’t use it all that much.

With the advent of Lambda, EC2 can feel a bit more laborious as there is a lot more set up involved than with serverless solutions.

12 Days of AWS Day 9 written around snowflakes with a penguin opening a gift

The Twelve Days of AWS: CloudWatch

Speaking of CloudWatch… all things logging belong here. CloudWatch will track metrics on your EC2 instances, SQS, SNS entries and more. You will have graphing of data, and be able to keep track of everything happening on your AWS stack.

12 Days of AWS Day 8 written around snowflakes with a penguin wearing a red beanie hat

The Twelve Days of AWS: CloudFormation

On the 7th day of AWS, I would like to mention something that almost stays hidden behind the scenes when using Serverless, but is critical to its use.

CloudFormation allows you to spin up a stack of resources from a config file. It deals with the order of operations, timing and error handling on your behalf and makes it pretty easy to keep on top of all the elements of the stack.

12 Days of AWS Day 1 written around snowflakes with a penguin wearing a winter hat

The Twelve Days of AWS: SNS

Simple Notification Service (SNS) is another great tool for distributed operations but is different from SQS in various ways including:

12 Days of AWS Day 6 written around snowflakes with a penguin holding a lollipop

The Twelve Days of AWS: SQS

Simple Queue Service (SQS) is an indispensable tool for many batch operations and concurrent events within, and even outside of, the AWS infrastructure. There are two types of queues: Standard or FIFO. The standard queue, where messages enter the queue and exit in the most performance effective manner, can be batched into groups of messages and can be used to trigger Lambdas.

12 Days of AWS Day 1 written around snowflakes with a penguin building a snowman

The Twelve Days of AWS: IAM

AWS Identity and Access Management (IAM) is where you set up users and groups to create and assign roles and permissions to those users. You may want to assign a user with specific access to certain functionality that will be used by a Lambda, or a user login for someone to log into AWS and view logs. All of these things are done here in IAM.

12 Days of AWS Day 4 written around snowflakes with a penguin ice skating

The Twelve Days of AWS: Serverless

While this is not an actual AWS tool, it is something so useful for setting up Lambdas that it deserves a day to itself.

When looking for how to get started with Lambdas I ran into Serverless pretty quickly. It’s a framework that does a lot of the nitty-gritty and heavy lifting involved with configuring the elements that surround the code written inside the Lambdas.

12 Days of AWS Day 3 written around snowflakes with a penguin wearing a winter sweater

The Twelve Days of AWS: Lambda

When I needed to perform small operations on files stored in S3, I turned to this next tool in the AWS arsenal, Lambda. Lambda is effectively a ‘serverless’ script running platform, whereas of the time of writing, code in Node.js, Python, Java, Ruby, Go, C# or PowerShell will be run whenever triggered by specific events. A few examples of these events are: S3 Object uploaded, Amazon Alexa, and Manual trigger via the Lambda UI.

12 Days of AWS Day 2 written around snowflakes with a penguin wearing earmuffs dancing