DynamoDB Time-to-Live (TTL)

DynamoDB Time-to-Live (TTL) is a mechanism that lets you set a specific timestamp to delete expired items from your tables. Once the timestamp expires, the corresponding item is marked as expired and is subsequently deleted from the table. TTL can help you reduce storage usage and reduce the cost of storing data that is no longer relevant.

Use cases

There are two main scenarios where TTL can come in handy:

1. Deleting old data that is no longer relevant.

2. Sometimes you may want data to be kept in DynamoDB for a specified time period in order to comply with your data retention and management policies.

Enabling TTL

To enable TTL for a table, there should be a numeric DynamoDB table attribute populated with an epoch timestamp to specify the expiration criterion. Can set TTL on existing tables if it has an attribute that can be used as TTL for its items. If the table does not have such an attribute, you will have to create one.

To specify TTL, first enable the TTL setting on the table and specify the attribute to be used as the TTL value. This value is the expiry time, specified in epoch time format. TTL can be specified from the console or using the TTL API. If format is wrong, the value is ignored and the item won’t be deleted.

DynamoDB Console allows you to hover over a TTL attribute to see its value in human-readable local and UTC time. TTL attribute is like any other attribute. The TTL value can be read like other attributes. Can create indexes. Can be projected to an index. They are editable. Can set field level access control.

Expiring Items

DynamoDB runs a background scanner that monitors all the items. If the timestamp has expired, the process will mark the item as expired and queue it for subsequent deletion. TTL scans and deletes using background throughput available. The expired item may not be deleted immediately.

DynamoDB will aim to delete expired items within a two-day window on a best-effort basis. If you try to read items that have expired but haven’t yet been deleted, the returned result will include the expired items. You can filter these items out based on the TTL value if required.

The expiry of data in a table on account of the TTL value triggering a purge is recorded as a delete operation. Therefore, DynamoDB Streams will also have the delete operation recorded in it. The delete record will have an additional qualifier to distinguish between your deletes and deletes due to TTL.

Disabling / Changing TTL             

If a table already has TTL enabled and you want to specify a different TTL attribute, then you need to disable TTL for the table first, then you can re-enable TTL on the table with a new TTL attribute. Note that Disabling TTL can take up to one hour to apply across all partitions

TTL – Expected Usage and Limits

1. There is no option to automatically delete the entire table.

2. TTL takes affect only for those items that have a defined value in the TTL attribute. Other items in the table remain unaffected.

3. AWS currently do not support specifying an attribute in a JSON document as TTL attribute. Also, do not support deleting an item in a JSON document once it expires, but only whole document.

4. Updating items with an older TTL values is allowed. However, if the value is over 5 years old, DynamoDB will ignore the timestamp and not delete the item.

5. Expired items are not backed up before deletion. You can leverage the DynamoDB Streams to keep track deletions upto 24 hours since the time it is deleted.

6. You can get the status of TTL at any time by invoking the DescribeTable API or viewing the table details in the DynamoDB console.

TTL Pricing

Enabling TTL requires no additional fees.

The scan and delete operations needed for TTL are carried out by the system and does not count toward your provisioned throughput or usage. You are also not charged for the internal scan operations to monitor TTL expiry for items. Also these operations will not affect your throughput usage for the table.

Expired items accrue storage costs till they are actually deleted. If I query for an expired item, it will still use my read capacity. This behavior is the same as when you query for an item that does not exist in table.