DynamoDB Local Secondary Index (LSI)

1. an index that has the same partition key as the table, but a different sort key. E.g. tracks Orders for customers, where CustomerId is the partition key.

2. "local" in the sense that every partition is scoped to a table partition that has the same partition key.

3. When you query a LSI, DynamoDB can access any of the projected attributes, with the same performance characteristics as if those attributes were in a table of their own.

4. Each index may also specify that all non-key attributes from the primary index are projected. If you choose to project all attributes into index, the index replicates the same data as the table itself, but organized by the alternate sort key.

5. LSI can only be created during table creation; cannot be updated/deleted later.

   1. When you create a table with local secondary index, you may decide to create local secondary index for future use by defining a sort key element that is currently not used. Since local secondary index are sparse, this index costs nothing until you decide to use it.

6. In order to ensure that data in the table and the index are co-located on the same partition, LSIs limit the total size of all elements (tables and indexes) to 10 GB per partition key value. 

7. When a read query requests attributes that are not projected into the LSI, DynamoDB will fetch those attributes from the primary index.

8. LSIs are updated automatically when the primary index is updated (additions, updates and deletes). 

9. Similar to reads from a primary index, LSI supports both strong and eventually consistent read options. 

10. You don’t need to explicitly provision capacity for a LSI. It consumes provisioned capacity as part of the table with which it is associated. 

Local Secondary Indexes - Pricing

Querying non-projected attributes stored in the primary index by performing a table fetch operation will cost additional read capacity units.

Reads from LSIs and writes to tables with LSIs consume capacity by the standard formula of 1 unit per 1KB of data, with the following differences:

• When writes contain data that are relevant to one or more LSI, those writes are mirrored to the appropriate LSI. In these cases, write capacity will be consumed for the table itself, and additional write capacity will be consumed for each relevant LSI.

• Updates that overwrite an existing item can result in two operations– delete and insert – and thereby consume extra units of write capacity per 1KB of data.

• When a read query requests attributes that are not projected into the LSI, DynamoDB will fetch those attributes from the primary index. This implicit GetItem request consumes one read capacity unit per 4KB of item data fetched.

Note: LSI consume storage for the attribute name and value of each LSI’s primary and index keys, for all projected non-key attributes, plus 100 bytes per item for the overhead of indexing.

Local Secondary Indexe Item Collections

An item collection is any group of items that have the same partition key, across a table and all of its local secondary indexes. Traditional relational database systems call these as shards or partitions.

Item collections are automatically created and maintained for every table that includes LSI.

DynamoDB stores each item collection within a single disk partition and hence every item collection is subject to a maximum size limit of 10 gigabytes.

For any distinct partition key value, the sum of the item sizes in the table plus the sum of the item sizes across all of that table's LSIs must not exceed 10 GB.

The 10 GB limit for item collections does not apply to tables without LSIs. Also, the storage size of an overall table with LSIs is not limited.

DynamoDB’s write APIs (PutItem, UpdateItem, DeleteItem, and BatchWriteItem) include an option, which allows the API response to include an estimate of the relevant item collection’s size.

Amazon recommend that you instrument your application to monitor the sizes of your item collections. Should try to log an error message whenever an item collection exceeds a user-defined limit (8 GB, for example), to provide an early warning system.

If a particular item collection exceeds the 10GB limit, then you will not be able to write new items, or increase the size of existing items, for that particular partition key. Read and write operations that shrink the size of the item collection are still allowed. Other item collections in the table are not affected.

You can also remove items or reduce item sizes in the collection that has exceeded 10GB.

If your table includes historical data that is infrequently accessed, consider archiving the historical data to S3, Glacier etc.

Limitations of Local Secondary Indexes

1. Local secondary indexes are not appropriate for every application as they introduce some constraints on the volume of data you can store within a single partition.

2. Local secondary indexes can only be queried via the Query API.

   1. To query a local secondary index, explicitly reference the index in addition to the name of the table you query.

   2. You must specify the index partition attribute name and value. You can optionally specify a condition against the index key sort attribute.

3. Each table can have up to 20 projected non-key attributes, in total across all local secondary indexes within the table.