Application Services

Serverless Services

Serverless services are where we have a service that runs and we don’t have the underlying server to manage, so we don’t have EC2 instances, for example with an Operating System and configuration and maintenance that we need to do. So that means that we lose all of that responsibility for managing the underlying server. It’s all done for us. They can automatically scale and they charge you just for what you use. So serverless services have become extremely popular with many modern cloud applications.

What I’m going to show you here is how a series of different services can be chained together. So let’s say we have an S3 static website. S3 is a Simple Storage Service and it is also serverless because there are no servers to manage. It’s just a storage service that you consume. You don’t manage any of the underlying infrastructure. So let’s say there’s a static website and the user uploads a file using the static website. Then what happens is using an event notification, the static website will then notify a Lambda function. A Lambda function is a type of serverless service, which is a compute service, and it enables us to run code. So we’ve stored some code in Lambda and when it receives the event, the code executes and in this case, the code is going to process the file that was uploaded to the bucket. Then it might save a copy to another bucket. It might also put a message into a queue. So a message queue is a way of storing information that’s going to be processed by another component. The queue could then notify another Lambda function. It can then process the data and maybe it stores a record in a DynamoDB table in a database. So the record gets stored in the database and maybe the Lambda function also sends a notification using an SNS topic that goes out to an email address. So this is just an example of how you can use a number of different serverless services. All of these are serverless. None of them have EC2 instances that you need to manage and all of them are used on a paper use basis. So you can pay just for what you use and you can scale them very easily as well.

AWS Lambda Functions

AWS Lambda is a serverless service which enables you to run compute functions. So that means it’s able to process code and it does so in response to triggers. Here we have a Lambda function and a developer is going to upload some code. So the developer simply has their own code, uploads it to create a function. Now, some events will occur to trigger the function. This could be a CLI event, the API, or the SDK, or it could be a scheduled event. So the Lambda function executes at a specific point in time. When that happens, the code gets executed and you only pay for the time that the code is actually running. So while your code is sitting there in the Lambda function not executing, you’re not paying anything, and then it will scale seamlessly based on demand. So it’s a great and really often a cost-effective solution for running your application code.

Create a Simple Lambda Function

In the AWS Management Console, go to Services > Compute > Lambda. So I’m now in the Lambda Console and I can simply Create function.

What we’re going to do is there are a few options here we are just going to Author from scratch. You can also choose a Blueprint, a Container image, or Browse serverless app repository. I’m just going to call this MySimpleFunction. We’ll leave the default runtime here on Node.js. Now, Lambda needs permissions, so it needs permissions to be able to interact with other services just like with EC2 we have an IAM role and we attach it using an instance profile. It’s the same sort of thing, but it’s called a default execution role.

So what will happen by default is Lambda will create a new role with basic permissions. I don’t need to change that. You can select roles if you want o, but let’s just allow it to create a simple role.

Under Advanced settings, there are a couple of additional options, including the ability to connect to VPC. Now click the Create function at the bottom of the page.

In our Lambda function, we can see it has its own Function ARN, so that’s the Amazon Resource Name.

Under the Code tab, you can sometimes see the code that you’ve uploaded.

We can Test our function. We can Monitor our function through Amazon CloudWatch. Each time you execute your function, it’s called invocation and you can have multiple invocations happening at a time. If you click on Configuration, you can see there are quite a few options here for how you can configure your function. Under Aliases, this is where you can actually create something called an Alias, which points to different versions of your function. Let’s just go back to Test and what I’m going to do is just call this MyTest and then I can save those changes.

I now have a saved test event and if I clock on Test, it will simply execute.

It literally just comes back and says Hello from Lambda!. So this is your Hello World application on Lambda. The status code 200 just means success. So the execution of the function succeeded. You can see here the amounts in milliseconds that it ran for and how many you were billed for. So you’re billed based on two things, the amount of time your function runs and the amount of memory that’s assigned to your function. Now, you can also click on logs here and that will take us over to CloudWatch logs.

Here we have something called Log streams. So CloudWatch Logs is a collection service for the logs from your applications and your systems.

So we can see the very same information for that execution of our function here. We can see the billed duration and memory size and we can see the start and the end.

If we now go back to Lambda > Functions > MyTest > Monitor, we should see that we have at least one invocation. So it’s a single dot here. We have a single invocation.

Application Integration Services

There are a few services in AWS that are known as the Application Integration Services and these services help you to communicate or pass information between different application components.

Amazon Simple Queue Service (SQS)

Now, this is a service that’s used for decoupling. So what does that mean? Here we have a Web Tier for our application. So that’s where our connections are coming in from customers on the Internet and then an App Tier where data gets processed. With direct integration, we’re actually connecting the Web Tier directly to the App Tier. Now, the problem with this is the App Tier must keep up the workload because the messages are getting sent straight to it. So what happens if you suddenly go viral and have a huge amount of traffic hitting your Web Tier? Suddenly, you got thousands of orders coming in and perhaps your App Tier can’t scale fast enough or can’t keep up with work because it’s got more processing to do than the Web Tier. In that case, you have some failures, potentially lost orders.

Figure 1.1 Direct Integration

Instead, we could use decoupled integration. So here we have the same Web Tier and App Tier, but we put a queue in the middle, an SQS queue. This time what happens is the Web Tier is going to place messages into the queue and the App Tier is going to process those messages. It’s actually going to poll the queue and check for messages. So that means it doesn’t matter if your Web Tier suddenly gets hit with a huge amount of traffic, it can simply put its messages into the queue and the app tier may be a bit delayed, but it’s going to process them at some point. So that’s decoupling and you can use SQS as one of the services you can use for decoupling.

Figure 1.2 Decoupled Integration

Amazon MQ

Amazon Simple Notification Service (SNS)

This is a service for sending notifications. We have something called an SNS Topic and you can create multiple SNS topics in your account. Now, on the left-hand side here, we have publishers. A Publisher is a service like EC2 or CloudWatch or the Simple Storage Service. On the right-hand side, you have Subscribers which subscribe to the SNS topic. These could be services like Amazon SQS, can be email or SMS, and it could be AWS Lambda or other applications. So the publisher is going to send a message, a notification to a topic and that can then be forwarded through, it’s pushed to the subscriber. Publishers can send notifications to thousands of subscribers. So for example, with email, there could be thousands of email addresses that are subscribed to a topic and the SNS service will send those notifications out to all those emails.

AWS Step Functions

Step Functions is a service for orchestrating parts of your application. So you can build out a visual workflow. Let’s say the workflow looks like this. First, you do something and that could be a Lambda function that gets executed. Then the workflow wants to check something. Is it a Yes or No answer? That could then move on to Creating something or Send notification. Each of these components of the application could be a separate Lambda function that processes information. Then perhaps it waits and after it’s completed, something else might happen. Check the results perhaps. And again, each component can be a separate Lambda function.

AWS Simple Workflow Service (SWF)

Here, what happens is you’re using a similar kind of workflow, but it has a slightly different use case. Now, SWF has been around a very long time, much longer than Step Functions. So here, let’s say we have a customer placing an order. The order gets verified, the credit card gets charged, the product gets processed and then we have human involvement. So perhaps the human needs to take something out of the automated workflow and maybe then they just scan it with a barcode reader and this is where SWF really comes in handy. It’s good for these human-enabled workflows. Then maybe it goes back into an automated system and gets shipped and the order is complete.

Application Integration Services Comparison

ServiceWhat it doesExample use cases
Simple Queue ServiceMessaging queue; store and forward patternsBuilding distributed/decoupled applications
Simple Notification ServiceSet up, operate, and send notifications from the cloudSend email notification when CloudWatch alarm is triggered
Step FunctionsOut-of-the-box coordination of AWS service components with visual workflowOrder processing workflow
Simple Workflow ServiceNeed to support external processes or specialized execution logicHuman-enabled workflows like an order fulfillment system or for procedural requests NOTE: AWS recommends that for new applications customers consider Step Functions instead of SWF
Amazon MQMessage broker service for Apache Active MQ and RabbitMQNeed a message queue that supports industry-standard APIs and protocols; migrate queues to AWS

Amazon EventBridge / CloudWatch Events

Amazon EventBridge is a new name for a service which was also known as Amazon CloudWatch.

Amazon EventBridge

Well, it processes state changes in your resource. So let’s say we have an event source, in this case, Amazon EC2 and something happens, the EC2 instance is terminated. So this is the event. That goes through to the Amazon EventBridge event bus and then we can have a rule and the rule might say that you’re going to forward an SNS notification to somebody to let them know what’s just happened. So that’s very simply how it works and it can process lots of different state changes in our resources. So in this case, the email goes out. Now, just remember, this is also known as CloudWatch events as well.

Here’s an example of creating an event rule in Amazon EventBridge. So here, we have the service provider as AWS. The service name is EC2 and we’re looking for an EC2 instance state change notification and we’re describing the specific state here that we want to be notified. In this case, that’s terminated events. In fact, you can specify an individual instance ID as well.

And then we’d get a result, something like this, and that would be forwarded through. And we can quite easily configure an SNS notification to send somebody an email to let them know that this instance has been terminated.

Create an Event-Driven Application

Simple Event-Driven Application

Event-Driven Applications are where an event in one resource leads to an event and another one. So in the diagram below, this is going to be us at the bottom here and we’re going to create an SNS topic. What we’re going to do is submit a notification to the topic through the AWS Management Console. Now, the SNS topic is going to be subscribed to an Amazon SQS queue. So the message will actually get put into the queue. The queue is then going to trigger a Lambda target. So it’s then going to process that message from the queue and what it’s going to do is simply write an event into CloudWatch logs. So whatever message we put in the SNS topic, we should then find it in CloudWatch logs. So very simple application, but it shows you how we can use multiple services together.

Let’s head over to the AWS Management Console. The first thing we’re going to create is the Lambda Function. So let’s head to Services > Computer > Lambda > Functions > Create function. Now we’ll just call this one WriteToCWLogs because we’re going to be writing to CloudWatch logs. We’re going to leave it as Node.js and we don’t change any settings here. So let’s create the function.

So the function has been created and what we’re going to do is here we’re going to click on this little plus and choose New File.

So what you need to do is just copy all of this code to your clipboard.

exports.handler = async function(event, context) {
  event.Records.forEach(record => {
    const { body } = record;
    console.log(body);
  });
  return {};
}

Paste that in. Go to File > Save. We’re going to call this one index.js, the same as the file name that’s already in there, and just save and overwrite.

We can then click on Deploy and that deploys our code. So our function is now ready.

The next thing we’re going to do is the SQS queue. We’re creating the function, then the queue, then we’re going to put a message into the queue and just test that the function works and writes an event to CloudWatch logs. Then we’re going to add the SNS topic and publish a message here and at least at that point, we know that this piece of the solution is actually working already because we’ve tested it. Back in the AWS Management Console, we need to go to Services > Application Integration > Simple Queue Service. In SQS we’re going to create a queue and I’m simply going to call it MySimpleQueue. Scroll down. We don’t need to make any changes here and just Create queue.

Now, the next thing we need to do is click on Lambda triggers and we want to set up a trigger so that the queue will trigger the Lambda function. Let’s see what happens if we try and do that now.

I don’t think it’s going to work, but it’s going to be interesting to see why. So let’s choose WriteToCWLogs, the function we want to use, and click on Save.

We get this error message and it’s telling us that the execution role for Lambda doesn’t have the required permissions. Specifically, it doesn’t have the received message API action privilege on SQS.

So what we can do is go back to our Lambda function, click on Configuration > Permissions, and then click on where we’ve got the execution role (WriteToCWLogs-role-1-xvaxmle).

If we look at the JSON, we can see that we have some privileges to CloudWatch logs, for example, but nothing really else. We need some permissions to SQS. Click on Add permissions > Attach policies.

I’m just going to type SQS up here and then we can see this one, which is the AWSLambdaSQSQueueExecutionRole. Select this policy and click on Attach policies.

If we have a look at this policy, we’ll see that we have that SQS received message action and that’s allowed. So that should fix our problem.

Coming back to SQS, let’s try and save this one again.

This time it works. So we now have the trigger configured. You can see it’s under the creating status here, just refresh and you’ll see the status will change to Enabled.

So what we can do is try and send a message by clicking Send and receive messages and I’m just going to type Test message from SQS and click on Send Message. So we’ve submitted a message to the queue and what should happen is it should trigger our Lambda function and then we should see something in CloudWatch logs.

So coming back to Lambda, let’s go to Monitor and you may or may not see an invocation quite yet, but either way, what we need to do is click on View logs in CloudWatch and this will tell us whether it worked or not.

If you see a log stream here, that’s all good. That means that your Lambda function has successfully processed that message from the queue, it’s been invoked. Click on the log stream and we can see that we have these events and we get a start, then we get the body and the end and a report. You can see in this one here, it says INFO Test message from SQS.

Now we need to go to SNS and we’re going to set up a notification for our topic. Go to Services > Application Integration > Simple Notification Service > Topics > Create topic. Choose the Standard option here. I’ll just call this MySNSTopic. Then scroll down, we don’t need to change anything else. Just Create topic.

So we have a Topic, but we also need a subscription. So click on Subscriptions > Create subscription.

Under Topic ARN, we can see the topic we have, MySNSTopic. For Protocol, we’re going to choose Amazon SQS, and then for our Endpoint, we choose our MySimpleQueue. That’s all we need to do, Create subscription.

Now we need to go back over to SQS. In SQS, I’m going to go back up to MySimpleQueue > SNS subscriptions. You’ll see your MySNSTopic already attached.

Back in SNS, we go back to Topic level and then choose Publish message.

I’m going to call this My message from SNS. Then in the Message body, we just put the Message body from SNS Topic, and then we just click on Publish message.

So that should now go all the way through to CloudWatch logs. We just submitted an SNS topic. The SQS queue is subscribed to the topic so the message should end up in the queue. Then the queue triggers a Lambda function which processes that information and puts an entry in CloudWatch logs. So all that’s left is to go to CloudWatch logs and see if it’s there. Click on refresh. Now we can see we have some more messages that have come through. So you can see the type of Notification. You can see the My message from SNS from the SNS subject, the message body which is the Message body from SNS topic.

Now, all that remains is to make sure we don’t end up paying any fees on an ongoing basis. So the key thing that we need to stop is our SQS queue. Go back to SQS > Queues > MySimpleQueue > Delete. That’s really the only service that’s going to cost us any money. If you want to, you can go and delete your SNS topic as well as your Lambda function as well. But they won’t cost you anything as long as they’re not being used, they’re not being triggered in any way, notifications aren’t going through your SNS topic, os it won’t cost you any money.

Amazon API Gateway

Amazon API Gateway is a service that functions as a gateway for your APIs. It’s like a front-end API that sits in front of the other APIs that are internal to AWS. So Amazon API Gateway sits within a region. We create an API Gateway and then we can connect to it over the Internet using various clients. We could have a mobile client. It could be a service somewhere on the Internet or in your company data center that’s talking to API Gateway, could be another website that’s utilizing your API. But it’s a front-end API that you’re publishing out onto the Internet. And this is an example of a RestAPI as well. Now, on the back-end, the API Gateway might be talking to AWS Lambda. So it’s forwarding on requests. It could be forwarding requests into a VPC where you might have Lambda functions in a VPC, you might have EC2 instances, load balancers, and that kind of thing and those can be in private or in public subnets. You can pretty much talk to any service through API Gateway so you can configure different types of integration to talk to different services. So it’s your front-end API that then talks to the various different APIs within AWS. You can also talk to public API points as well so you can go back out again and communicate with an API running somewhere else. Maybe it’s on your on-premises data center that you’ve published it to the Internet via some kind of Internet edge device, or it could be some other third-party service on the Internet. So think of the API as the front door for applications to access data, business logic, or functionality of the back-end services. The API Gateway is the front-end of your application and then it will be able to communicate using various different APIs to the back-end services within AWS or external to AWS as well.

Exam-Cram

Application Services

Serverless services include:

AWS Lambda Functions

Amazon Simple Queue Service (SQS)

Amazon MQ

Amazon Simple Notification Service (SNS)

AWS Step Functions

Amazon Simple Workflow Service (SWF)

Amazon EventBridge

Amazon API Gateway

I found you guys kind and approachable.

hope I can contribute to the team anytime soon