[AWS Certificate]-Amazon Elasticsearch Service

Amazon Elasticsearch Service - Overview

• May be called Amazon ES at the exam
• Managed version of ElasticSearch (Open source project)
• The ELK Stack on AWS Cloud ( ElasticSearch + Logstash + Kibana)
  • ElasticSearch - provides search and indexing functionality
  • Logstash - provides log ingestion mechanism, alternative to CloudWatch Logs
  • Kibana - provides real-time dashboards for ES data, is a visualization tool
• Needs to provision servers (not a serverless offering)
• Use cases:
  • Log Analytics
  • Real time application monitoring
  • Seucrity analytics
  • Full text search
  • Clickstream analytics
  • Indexing

---

Elasticsearch + Logstash + Kibana (ELK)

 

• ElasticSearch - provides search and indexing capability
  • You send data in the form of JSON documents to Elasticsearch using the API / Logstash / Firehose
  • Elasticsearch automatically adds a searchable reference to these documents in the cluster's index
• Logstash
  • Log ingestion mechanism, uses the "Logstash agent"
  • Alternative to CloudWatch Logs (you decide on retention and granularity)
• Kibana
  • Provides real-time dashboards on top of the data that sits in ES
  • Alternative to CloudWatch dashboards (more advanced capabilities)

---

ElasticSearch Service patterns

 

---

Elasticsearch Service - Multi-AZ

 

• Supports multi-AZ - up to three-AZ deployments (1-, 2-, or 3-AZs)
• Can be deployed within VPC
• Uses dedicated master nodes to increase cluster stability
• Dedicated master nodes are distributed across 3-AZs, even if you select 2-AZ deployment
• Recommended to use 3-AZs for production
• For multi-AZ, create at least one replica for each index in the cluster
• Without replicas, cross-AZ replication doesn't happen which largely defeats the purpose of Multi-AZ

 

 

 

---

Loggin in Elasticsearch Service

 

• Amazon ES provides three types of Elasticsearch logs
  • error logs
  • search slow logs
  • index slow logs
• Accessible through CloudWatch Logs
• Can be enabled from within ES Console (disabled by default)

---

ElasticSearch Service Pricing

 

• Instance pricing (priced per hour)
  • On-Demand instance
  • Reserved instance - discounted pricing over 1- or 3-year term
  • UltraWarm instance - On-Demand or Managed storage (new tier type, cost-effective way to store large amounts of read-only data)
• EBS volume pricing (magnetic, general purpose, and provisioned IOPS)
• Standard data transfer charges
• Automated snapshots - free with 14-day retention
• Manual snapshots - per GB

---

Demo

---