The best way to centralize your Amazon CloudWatch Logs

What are CloudWatch Logs (CWL)?

Nearly every service publishes logs to CloudWatch Logs — they’re stored in Groups with multiple Streams in each group. Over time Streams are aged out and logs are no longer delivered to them. New streams will be created by CloudWatch and written to. Services can publish logs to multiple Streams at a time and logs generally are delivered to the groups in a few seconds. For example, here’s what a Lambda Log Group with multiple Streams looks like:

Subscription Filters automate log delivery

Instead of polling for logs, just enable a subscription filter to automate the delivery of your logs. You can choose from these four delivery targets:

1. Amazon Kinesis
2. Amazon Kinesis Firehose
3. A custom AWS Lambda function — CW will batch Stream entries and invoke your function with up to 10 log records.
4. Amazon OpenSearch — this utilizes a Lambda function that is created on your behalf to deliver logs to ES. CW will also batch and deliver entries to Lambda. Do not modify this function!

Just a few caveats to keep in mind:

• While the Subscription is free, the delivery targets are not. All 4…