Restore DynamoDb data from S3 incremental backups
example-restore-to-point.js
var dynamoIncrementalRestore = require('../');
console.log(dynamoIncrementalRestore.logbuildList({
Prefix: "ServiceFolder/TableFolder/ClientID", // Where the files sit. Use parent folders and it should recursively find all child document/files e.g. Formsmith/Formsmith.Blue.FormSchema/543
DestinationTableName: "Example Table Name" ,// e.g. Formsmith.Blue.FormSchema
Bucket: "au-backup-bucket-name", // e.g. pageup-dynamo-backup
restoreToPointInTime: '2020-11-14T23:50:32.000Z', //the last point in time you want to update a version as an ISO-8601 string
region: "ap-southeast-2" //Dynamo Region. Change for each datacenter
}));This will be fine to run both on your local environment (for testing the dev datacenter) or uploaded into an EC2 instance with the appropriate permissions to access client data for backing up. Clone the repository into a machine you want to perform the task on and use node to run the script.
Run package.ps1 to bundle up the code into LambdaFunction.zip, which is ready to upload as a Lambda function to AWS. Set the handler to be index.logbuildList, give it a role with the appropriate permissions, and run it by giving it a test event in the form:
{
"Prefix": "ServiceFolder/TableFolder/ClientID",
"DestinationTableName": "Example Table Name" ,
"Bucket": "au-backup-bucket-name",
"restoreToPointInTime": "2020-11-14T23:50:32.000Z",
"region": "ap-southeast-2"
}##TODO
- Batch writes to Dynamo in blocks of 25, and handle unprocessed items
- Break process into three steps (Build Version List, Build Dynamo Requests, Send Requests to Dynamo)
- Parallelise retrieval from S3, and Batch Write Items
See lib/index.js
In lieu of a formal styleguide, take care to maintain the existing coding style. Add unit tests for any new or changed functionality. Lint and test your code using gulp.
Copyright (c) 2016 Abhaya Chauhan. Licensed under the MIT license.