serverless-plugin-dynamodb-autoscaling
Autoscaling configuration for DynamoDB tables
- Convention over configuration approach - Automatically discovers preconfigured tables and accompanies them with dedicated scaling resources. Configuration can be fine tuned addressing specific tables or indexes, or switched to white list approach
-
Resources savvy - Existing project's IAM role is reused for handling scaling target resources. It's only mandatory
ScalableTarget
andScalingPolicy
resources that are added to CloudFormation stack
Installation
npm install serverless-plugin-dynamodb-autoscaling
Configuration
Activate plugin in serverless.yml
plugins:
- serverless-plugin-dynamodb-autoscaling
In most cases it's all you need to have autoscaling resources configured. Still if you need to fine tune configuration for some reason see tables and IAM role configuration. Additionally if you approach any errors during CloudFormation deployment, refer to troubleshooting
Tables configuration
By default autoscaling configuration is automatically applied to all preconfigured DynamoDB tables and all its eventual global secondary indexes.
Still, we can exclude individual tables or tweak their configuration via configuration withinserverless.yml
config:
resources:
Resources:
SomeTable1:
Properties:
TableName: foo
SomeTable2:
Properties:
TableName: bar
SomeTable3:
Properties:
TableName:
Fn::Sub: ${AWS::Region}-test
custom:
dynamodbAutoscaling:
tablesConfig:
# Disable autoscaling for table referenced by "SomeTable1" resource name
SomeTable1: false
# Disable autoscaling just for indexes of "SomeTable2" table
SomeTable2:
indexes: false
# Tweak minCapacity setting for all tables of which resource names start with SomeTable
# (glob patterns can be used)
SomeTable*:
minCapacity: 10
SomeTable4:
# Tweak maxCapacity setting for table referenced by "SomeTable4" (just table)
table:
maxCapacity: 300
# Tweak targetUsage setting for SomeTable4 indexes
indexes:
targetUsage: 0.5
SomeTable5:
indexes:
# Do not autoscale index 'foo'
foo: false
SomeTable6:
indexes:
# Do not autoscale any indexes but 'someIndex1' and 'someIndex2'
"*": false
someIndex1: true
someIndex2:
# Tweaking any of the configuration option will also whitelist the index
minCapacity: 100
White list approach
If you prefer white list instead of black list approach you can handle configuration as below
custom:
dynamodbAutoscaling:
tablesConfig:
# Disable autoscaling for all
"*": false
# but enable for table referenced by resource name "SomeTable1"
SomeTable1: true
Configurable settings:
-
maxCapacity
(defaults to200
) refers toScalableTarget.MaxCapacity
-
minCapacity
(defaults to5
) refers toScalableTarget.MinCapacity
-
targetUsage
(defaults to0.75
) refers toScalingPolicy.TargetTrackingScalingPolicyConfiguration.TargetValue
(value is multiplied by100
when assigned toTargetValue
setting) -
scaleInCooldown
(defaults to60
) refers toScalingPolicy.TargetTrackingScalingPolicyConfiguration.ScaleInCooldown
-
scaleOutCooldown
(defaults to60
) refers toScalingPolicy.TargetTrackingScalingPolicyConfiguration.ScaleOutCooldown
ScalingPolicy
chaining
By default ScalingPolicy
resources are chained via DependsOn
property, so they're deployed sequentially and not in parallel. It's to avoid reaching eventual CloudWatch rate limits.
Still it has a downside of slowing down the deployment. If number of tables in your stack is not large, or you've lifted rate limits on AWS side, then you can safely turn off that option to ensure more robust deployments:
custom:
dynamodbAutoscaling:
chainScalingPolicies: false
IAM role configuration
By default existing lambda IAM role is reused (if recognized) or new dedicated IAM role is created.
Still it's possible to handle IAM configuration outside of this plugin, for that just pass ARN of externally configured IAM role via iamRoleArn
setting:
custom:
dynamodbAutoscaling:
iamRoleArn: "arn-of-iam-role-to-handle-tables"
Troubleshooting
Eventual IAM policy update race condition issue
If at first deployment you're faced with one of the following errors:
Unable to assume IAM role
Role is missing the following permissions
The security token included in the request is invalid
It can be result of a race condition issue described as following by AWS team:
It's a known situation and confirmed by the internal team that manages CloudFormation that the propagation of IAM policies and resources might take longer than CloudFormation to launch the dependent resources. This race condition happens now and then, and unfortunately CloudFormation team is not able to determine programmatically when a role is effectively available in a region.
To workaround it, the stack with just IAM polices update (and no autoscaling resources yet) needs to be deployed first, and then further deployment may carry the autoscaling resources update (unfortunately just relying on DependsOn
brings no rescue)
To make handling of that case easier this plugin enables the IAM only deployment via iamOnly
option, you may refer to this option as one-time mean
custom:
dynamodbAutoscaling:
iamOnly: true
Tests
npm test