New Function - Use Amazon S3 event notifications on Amazon Eventbridge

Amazon Eventbridge was released in mid -2019 so that all -scale powerful event -driven applications could be easily built.Since its launch, some important features have been added, such as Schema Registry, Archive and Replay Events, Cross-Region Event Bus Targets, API Destinations, and have been able to send events to any HTTP API.Eventbridge is a very powerful and flexible architectural component that supports a very long destination list and has the function of performing pattern matching, filtering, and routing.

From November 29, 2021 (US time), it will be easier to build applications that will respond quickly and efficiently to change S3 objects from November 29, 2021 (US time).This is a new "direct wiring" model that is high -speed, reliable, and friendly for developers.In order to process the event, it is no longer necessary to add an object copy or write a dedicated code.

At this point, you already have the ability to respond to changes in S3 objects, and you may be wondering what is going on here. In 2014, he launched his S3 event notification to SNS topics, SQS queues, and Lambda functions. This was a very powerful feature (and still), but to use on an enterprise scale, it would be necessary to cooperate with other independent teams with other independent teams that share interest in the same objects and events. Sometimes. Eventbridge can already extract S3 API calls from his CloudTrail log and use it to perform pattern matching and filtering. Again, we always want to do something better (focusing on audit and log record), which is perfect for a very powerful and various types of apps.

Net-net can now set the S3 event notification directly to Eventbridge!This new model has some advantages:

Advanced filtering -can be filtered in many other metadata fields, such as object size, key name, time range, etc.This is more efficient than using the Lambda function that needs to call back to S3 and get additional metadata to determine the appropriate action policy.S3 issues only events that match the rules, so you can save costs by paying fees only for the event you are interested in.

Multiple transmission destinations -The same event notification can be routed via the API Destations service, including Step Functions, Kinesis Firehose, Kinesis Data Streams, and HTTP targets.This is much easier than creating a unique fan out mechanism, and it can also help independent teams to deal with enterprise conditions that you want to perform your own event processing.

High -speed, highly reliable call -patterns are quick and directly matched, and the target is called.S3 can distribute events at least once to Eventbridge, increasing the reliability of the application.

Other features of Eventbridge, such as the function of archiving and replaying events, can be used.This allows you to reprocess the event if an error occurs or if a new target is added to the event bus.

It takes only a few minutes to start the start method.First, one of his S3 buckets (in this case, Jbarrr-Public), enable her Eventbridge notification.Open the S3 console, search for your bucket, open the Properties tab, scroll down to the event notification and click Edit.

Select On on and click Save Change to be ready to roll as follows.

Next, create the rules using the Eventbridge console.Start by entering your name and explanation as usual.

Next, define a pattern that matches the bucket and an interesting event.

One pattern matches one or more buckets and one or more events.The following events are supported.

Next, select the default event bus and set the target to the SNS topic that issues a message to your Amazon email address.

Click Create to complete the preparation.To test, upload some files to your bucket and wait for the following message.

The message contains all interesting related information about the event, and (when you remove the quote and format):

{"version": "0","id": "2d4eba74-fd51-3966-4bfa-b013c9da8ff1","detail-type": "Object Created","source": "aws.s3","account": "348414629041","time": "2021-11-13T00:00:59Z","region": "us-east-1","resources": ["arn:aws:s3:::jbarr-public"],"detail": {"version": "0","bucket": {"name": "jbarr-public"},"object": {"key": "eb_create_rule_mid_1.png","size": 99797,"etag": "7a72374e1238761aca7778318b363232","version-id": "a7diKodKIlW3mHIvhGvVphz5N_ZcL3RG","sequencer": "00618F003B7286F496"},"request-id": "4Z2S00BKW2P1AQK8","requester": "348414629041","source-ip-address": "72.21.198.68","reason": "PutObject"}

The first event pattern was very simple and matched only the bucket name.Use content -based filtering to describe more complex and interesting patterns.For example, you can use a numerical matching to set a pattern that matches object events less than 1 megabett.

{"source": ["aws.s3"],"detail-type": ["Object Created","Object Deleted","Object Tags Added","Object Tags Deleted"],"detail": {"bucket": {"name": ["jbarr-public"]},"object" : {"size": [{"numeric" :["<=", 1048576 ] }]}}}

Alternatively, you can use the prefick to find an uploaded object to the "subfolder" (not actually exist) in the bucket.

"object": {"key" : [{"prefix" : "uploads/"}]}]}

All of these can be used in combination with all the existing Eventbridge functions, including archives and replays.You can also access the CloudWatch Metrics of each rule as follows:

Available now This function can be used right now, and you can start using all commercial AWS regions today.Pay 1 USD for each event that matches the rules.For more information, see the Eventbridge price page.

- Jeff;

The original text is here.