Creating a hourly scheduled backup
If you already using aws rds instances , you may face this issue when it comes to the production environments.
All database need a closest restore point if some disaster happens. But do we actually setup such as a hourly backup using current backup option ? No in AWS if you refer most of the instances only support up to daily backups.
Yes it is not having an option for a hourly backups. In this post I will give an solution for an alternative method by using an AWS service, AWS Backup.
First you need to identify your environments and actual customer base because you must have an idea of importance of protecting your data if there is some disaster or failure happens.
Let move with the original topic as how to over come with hourly backups.
To achieve this we have two optimal solutions
1.Use a lambda function,s3 and trigger an event to take a snapshot. Use may have to use AWS Event bridge(With event rules) or Scheduler (Recently introduced by AWS which is a cost effective way than event rules) - Old Way
- Create a lambda function and invoke it hourly via scheduler or event rule
- Update the lambda implementation to trigger the backup and save to a s3 bucket
Here I am not going to talk with the implementation as this purpose of the post is not about it.
2.Using AWS Backup Service inorder order achieve the given problem
Please refer the following steps to implement this approach
- Create the cloudformation template
- Deploy the resources
- Monitor resources and backups with
Using a CloudFormation template to create AWS Backup Service
AWSTemplateFormatVersion: 2010-09-09
Description: 'RDS MySQL Backup Service'
Parameters:
CreatedBy:
Description: Who is creating the cloudformation stack
Type: String
Default: CodePipeline
BackupPlanName:
Description: Enter the name of the backup plan (Required)
Type: String
Default: "hourly-backup"
CronExpression:
Description: Enter the cron expression for your backup plan (Required). Currently setup to occur hourly.
Type: String
Default: "cron(0 * ? * * *)"
Retention:
Description: This value will identify how many days your backup will be expired after (Required)
Type: String
Default: 3
Metadata:
AWS::CloudFormation::Interface:
ParameterGroups:
-
Label:
default: BackupPlan Configurations (Mandatory)
Parameters:
- BackupPlanName
-
Label:
default: Backup Rule configuration
Parameters:
- CronExpression
- Retention
Resources:
BackupVault:
Type: "AWS::Backup::BackupVault"
Properties:
BackupVaultName: !Sub ${BackupPlanName}-vault
AccessPolicy:
Version: '2012-10-17'
Statement:
-
Sid: 'Vault-Access-Policy'
Effect: Deny
Principal: "*"
Action: "backup:DeleteRecoveryPoint"
Resource:
- "*"
BackupPlan:
Type: "AWS::Backup::BackupPlan"
Properties:
BackupPlan:
BackupPlanName: !Ref BackupPlanName
BackupPlanRule:
-
RuleName: !Sub ${BackupPlanName}-rule
TargetBackupVault: !Ref BackupVault
ScheduleExpression: !Ref CronExpression
Lifecycle:
DeleteAfterDays: !Ref Retention
StartWindowMinutes: 60
CompletionWindowMinutes: 120
DependsOn: BackupVault
TagBasedBackupSelection:
Type: "AWS::Backup::BackupSelection"
Properties:
BackupSelection:
SelectionName: !Sub ${BackupPlanName}-job-assignment
IamRoleArn: !Sub "arn:aws:iam::${AWS::AccountId}:role/service-role/AWSBackupDefaultServiceRole"
ListOfTags:
-
ConditionType: "STRINGEQUALS"
ConditionKey: "aws:cloudformation:stack-name"
ConditionValue: !Sub "rds"
BackupPlanId: !Ref BackupPlan
DependsOn: BackupPlan
Make sure you update your CloudFormation execution role with below permissions.
{
"Action": [
"backup:*"
],
"Resource": "*",
"Effect": "Allow",
"Sid": "backup"
},
{
"Action": [
"kms:*"
],
"Resource": "*",
"Effect": "Allow",
"Sid": "kms"
},
{
"Action": [
"backup-storage:*"
],
"Resource": "*",
"Effect": "Allow",
"Sid": "backupstorage"
}
Backup vault: A container for storing backups. Backup vaults are created in AWS Regions and can be used to store backups of AWS resources from multiple accounts.
Backup plan: A set of instructions that defines how backups are created and stored. Backup plans can be used to automate the backup process and ensure that backups are created on a regular basis.
Recovery point: A snapshot of a resource that can be used to restore the resource to a previous state. Recovery points are created by backup plans and stored in backup vaults.
Resource assignment/backup selection: A set of instructions that defines which resources should be backed up. Resource assignments can be used to back up specific resources or groups of resources.
You may refer following links for more information and get familiarize with the available features.
-Backup plan
https://docs.aws.amazon.com/aws-backup/latest/devguide/API_BackupPlan.html
https://docs.aws.amazon.com/aws-backup/latest/devguide/creating-a-backup-plan.html
-Backup vault
https://docs.aws.amazon.com/aws-backup/latest/devguide/vaults.html
-Backup rule
https://docs.aws.amazon.com/aws-backup/latest/devguide/API_BackupRule.html
-Backup Assignee/Selection
https://docs.aws.amazon.com/aws-backup/latest/devguide/API_BackupSelection.html
-Backup Vault
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-backup-backupvault.html
Visual Design of our Backup
Snapshots vs Continuous backups
When you doing deployment and getting backups you will get a question about doing a comparison with backup types. Because there are several options and selections are vary.
You can get more information by following below link
https://docs.aws.amazon.com/aws-backup/latest/devguide/integrate-cloudformation-with-aws-backup.html
https://www.nucleustechnologies.com/blog/aws-snapshot-vs-backup/
References
Pricing - https://aws.amazon.com/backup/pricing/
https://www.druva.com/documents/pf/white-papers/8-tips-to-simplify-aws-backup-and-recovery.pdf
