Introduction to Amazon Elasticsearch Service

Amazon Elasticsearch Service (Amazon ES) is an AWS managed Elasticsearch service on the AWS Cloud. Amazon ES delivers Elasticsearch’s APIs and real-time capabilities along with the availability, scalability, and security required by production workloads. Amazon Elasticsearch Service currently supports Elasticsearch versions upto 6.2.

Amazon ES can automatically detect and replace failed Elasticsearch nodes, reducing the overhead associated with self-managed infrastructure. You can easily scale your cluster via a single API call or a few clicks in the console. You can also get direct access to the Elasticsearch open-source API so that code and applications with existing Elasticsearch environments will work seamlessly.

Amazon ES also offers built-in integrations with Kibana, Logstash, and AWS services including Kinesis Firehose, Lambdas, and CloudWatch.

 If you are new to Elasticsearch you may first want to go through the notes available @ Elasticsearch Essentials. You should also be familiar with Essential AWS concepts.

Amazon ES domains

1. Amazon ES domains are Elasticsearch clusters in the cloud with the compute and storage resources you specify.

2. We can create Amazon ES domains through console, API, or CLI.

3. Amazon ES support configuring dedicated master nodes for each domain. 

4. Can create multiple Elasticsearch indices within a single Amazon ES domain.

5. Programs running on servers in my own data center can access my Amazon ES domains. 

6. Can migrate data from my existing Elasticsearch cluster to my new Amazon ES domain.

7. Amazon ES allows you to control the scaling of your Amazon ES domains using the console, API, and CLI. Scaling does not require downtime.

Amazon ES Snapshots

1. A snapshot is a copy of your Amazon ES domain at a moment in time.

2. Snapshot creation can be automated.

3. The daily snapshots retained by Amazon Elasticsearch Service will be deleted as part of domain deletion. Before deleting a domain, you should consider creating a snapshot of the domain in your own S3 buckets using the manual snapshot process.

Network configuration

Amazon ES support both internet or VPC access.

If we choose VPC access, Amazon ES will use private IP addresses from our VPC, which provides security by default. We can control network access within our VPC using security groups. We can optionally add an additional layer of security by applying a restrictive access policy.

Internet endpoints are publicly accessible. If we choose public access, we should secure our domain with an access policy that only allows specific users or IP addresses to access the domain.

Amazon ES Integrations with Other Services

1. Amazon ES is integrated with Amazon CloudWatch to produce metrics that provide information about the state of the domains.

2. Amazon ES can use CloudFormation Templates to provision Amazon ES domains.

3. Amazon ES supports three options for data ingestion:

   1. For large data volumes, Amazon recommend Amazon Kinesis Firehose. Can grant access using an IAM role that allows Firehose to perform the actions against a particular domain.

   2. Supports integration with Logstash.

   3. Use native Elasticsearch APIs, such as PUT and _bulk to load data into your domain.

4. Amazon ES support integration with Kibana. Can create custom reports with the Kibana installation included.

5. Customers will be able to choose between local on-instance storage available with the provisioned Amazon ES instances or EBS volumes to store their Elasticsearch indices.

6. Customers will be able to set up IAM policies to control access to their Amazon ES domains and sub resources like indices within the domains.

Amazon ES Pricing

1. You pay only for what you use, and there are no minimum or setup fees.

2. Amazon ES instance hours are billed for each hour your instance is running in an available state.

3. Partial Amazon ES instance hours consumed are billed as full hours.

4. Free tier provides free usage of up to 750 hours per month of a single-AZ t2.micro.elasticsearch or t2.small.elasticsearch instanceand 10GB per month of optional Amazon EBS storage (Magnetic or General Purpose).