Introduction to cloud financial management on AWS
Cloud financial management (sometimes also referred to as FinOps) is about managing the ongoing cost of cloud services.
Who should care about cloud financial management? Basically, anyone consuming IaaS or PaaS services — from IT, DevOps, developers, architects and naturally finance department.
When we start consume IaaS or PaaS services, we realized that almost any service has its pricing model — we just need to read the service documentation.
Some of the services’ pricing model are easy to understand, such as EC2 (you pay for the amount of time and EC2 instance was up and running), and some of services’ pricing model can be harder to calculate (you pay for the number of times the function was called in a month and the amount of memory allocated to the function).
In this post, we will review the tools that AWS offer us to manage cost.
Step 1 — Cost management for beginners
The first thing that AWS recommend for new customers is to use Amazon CloudWatch to create billing alarms.
Even if you cannot estimate your monthly cost, create a billing alarm (for example — send me email whenever the charges are above 200$). When time goes by, you will be able to adjust the value, per your account usage pattern.
To read more information about billing alarms, see:
If you already know that certain department is using specific AWS account and has a known budget, use AWS budgets, to create a monthly, quarterly or even yearly budget, and configure the budget interface to send you notifications whenever the amount of money consumed is about certain threshold of your pre-defined budget.
To read more about AWS budget creation, see:
https://docs.aws.amazon.com/cost-management/latest/userguide/budgets-create.html
If you wish to visualize your resource consumption over period of time, see trends, generate reports and customize the resource consumption information, use AWS Cost Explorer.
To read more about AWS Cost Explorer, see:
https://docs.aws.amazon.com/cost-management/latest/userguide/ce-what-is.html
Finally, if you wish to receive recommendations about saving costs, you have an easy tool called AWS Trusted Advisor.
The tool helps you get recommendations about cost optimization, performance, security and more.
This tool is the easiest way to get insights about how to save cost on AWS platform.
To read more about AWS Trusted Advisor, see:
https://aws.amazon.com/premiumsupport/knowledge-center/trusted-advisor-cost-optimization
Step 2 — Resource tagging and rightsizing
One of the best ways to detect and monitor cost over time and per business case (project, division, environment, etc.) is to use tagging.
You add descriptive tag for each and every resource you create, that will allow you later on to know which resources has been consumed — for example, which EC2 instances, public IP’s, S3 buckets and RDS instances, all relate to the same project.
For more information about AWS cost allocation tags, see:
https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/cost-alloc-tags.html
If you manage multiple AWS accounts, all relate to the same AWS organization, it is considered best practice to configure all account costs in a single place, also known as consolidated billing.
You will define which AWS account will store billing information, and redirect all AWS accounts in your organization to this central account.
Using consolidated billing, will allow you to achieve volume discount, for example — volume discount for the total data transferred from multiple AWS accounts to the Internet, instead of separate charge per AWS account.
For more information about consolidated billing, see:
https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/consolidated-billing.html
By using compute services such as Amazon EC2 or Amazon RDS, you might be wasting money, by not using the right size (amount of memory/CPU) per your actual resource demand (for example — paying on large instance, when it is underutilized).
Tools such as AWS Trusted Advisor mentioned earlier, will help you get insights and recommend you to change instance size, to save money.
Another tool that can assist you choose an optimal size for your instances is AWS Compute Optimizer, which scan your AWS environment and generate recommendations for optimizing your compute resources.
For more information about AWS Cost Optimizer, see:
https://docs.aws.amazon.com/compute-optimizer/latest/ug/getting-started.html
Even when using storage services such as Amazon S3, you can save money, by using the right storage class per actual use (for example Amazon S3 standard for big data analytics, Amazon S3 Glacier for archive, etc.)
There are two options for optimizing S3 cost:
· Using lifecycle policies, you configure how much time will an object stay in specific storage class without using the object, before it moves to a cheaper tier (until the object finally moves into deep archive tier or even deleted completely).
For more information about setting lifecycle policies, see:
https://docs.aws.amazon.com/AmazonS3/latest/userguide/how-to-set-lifecycle-configuration-intro.html
· Using S3 Intelligent-Tiering, objects will automatically move to the most cost-effective storage tier by their access frequency. Unlike lifecycle policies, object might move between hot storage (such as S3) to archive storage (such as S3 Glacier or deep archive), and vice versa, if an object in an archive tier suddenly was accessed, it will move to hot tier (such as S3).
For more information about S3 Intelligent-tiering, see:
https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-intelligent-tiering.html
Another simply tip for saving cost is to remove unused resources — from underutilized EC2 instance, unassigned public IP address, unattached EBS volume, etc.
AWS Trusted Advisor can assist you discover underutilized or unused resources.
For more information, see:
Step 3 — Get to know your workloads (cloud optimization)
When you deploy your workload for the first time, you don’t have enough information about its potential usage and cost.
You might choose too small or too large instance type, you might be using too expensive storage tier, etc.
One of the ways to save cost on development or test environments, which might not need to run over weekends or after working hours, is to use AWS Instance scheduler — a combination of tagging and Lambda function, which allow you to schedule instance (both EC2 and RDS) shutdown on pre-defined hours.
For more information about AWS instance scheduler, see:
https://aws.amazon.com/premiumsupport/knowledge-center/stop-start-instance-scheduler
If your workload can survive sudden shutdown and return to function from the moment it stopped (such as video rendering, HPC workloads for genomic sequencing, etc.) and you wish to save money, use AWS Spot instances, which allows you to save up to 90% of the cost, as compared to on-demand cost.
For more information about Spot instances, see:
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-spot-instances.html#spot-get-started
If your workload has the same usage pattern for long period of time (without shutdown or restart), consider one of the following options:
· Amazon EC2 Reserved Instances — allows you to reserve capacity for 1 or 3 years in advanced, and save up to 72% of the on-demand price.
For more information, see:
https://aws.amazon.com/ec2/pricing/reserved-instances/buyer
· Compute savings plans — commitment to use EC2 instances, regardless of instance family, size, AZ or region. Allows saving up 66% of on-demand price.
· EC2 instance saving plans — commitment to use specific instance family in specific region. Allows saving up to 72% of on-demand price.
For more information, see:
https://aws.amazon.com/savingsplans/faq/#Compute_.26_EC2_Instances_Savings_Plans
Summary
In this introduction post, we have reviewed the most common tools from AWS for detecting, managing and optimizing cost.
Using automated tools, allows organizations to optimize their resource consumption cost over time and over large scale and constant changing environments.
About The Author
Eyal Estrin is a cloud and information security architect, the owner of the blog Security & Cloud 24/7, with more than 20 years in the IT industry.
You can connect with him on Twitter and LinkedIn.
