The best way to centralize your Amazon CloudWatch Logs
What are CloudWatch Logs (CWL)?
Nearly every service publishes logs to CloudWatch Logs — they’re stored in Groups
with multiple Streams
in each group. Over time Streams are aged out and logs are no longer delivered to them. New streams will be created by CloudWatch and written to. Services can publish logs to multiple Streams at a time and logs generally are delivered to the groups in a few seconds. For example, here’s what a Lambda Log Group with multiple Streams looks like:
Subscription Filters automate log delivery
Instead of polling for logs, just enable a subscription filter to automate the delivery of your logs. You can choose from these four delivery targets:
- Amazon Kinesis
- Amazon Kinesis Firehose
- A custom AWS Lambda function — CW will batch Stream entries and invoke your function with up to 10 log records.
- Amazon OpenSearch — this utilizes a Lambda function that is created on your behalf to deliver logs to ES. CW will also batch and deliver entries to Lambda. Do not modify this function!
Just a few caveats to keep in mind:
- While the Subscription is free, the delivery targets are not. All 4…