Skip to content

Logging, Events, and AWS Messaging⚓︎

AWS CloudTrail⚓︎

AWS CloudTrail is a service that monitors and records account activity across AWS infrastructure, providing a history of events and API calls. Key features include:

  • Governance and Compliance: Ensures governance, compliance, and audit capabilities for AWS accounts.
  • Default Activation: CloudTrail is enabled by default for all AWS accounts.
  • Regional Flexibility: Trails can be applied globally across all regions (default) or limited to a single region.

CloudTrail Events⚓︎

CloudTrail events can be categorized into read and write events, further divided into management events (default on) and data events (default off due to volume, but can be enabled to trigger actions).

CloudTrail Insights⚓︎

CloudTrail Insights is used to detect unusual activity within an account. It analyzes normal management events to create a baseline and continuously evaluates write events for anomalies. Detected patterns include inaccurate resource provisioning, hitting service limits, bursts of AWS IAM actions, and gaps in periodic maintenance. CloudTrail events are stored for 90 days and can be sent to S3 for further analysis by Athena.

Amazon EventBridge (aka Cloudwatch Events)⚓︎

Amazon EventBridge provides connectivity between events and associated services. It supports various use cases, such as CRON job triggering, Lambda triggering SNS/SQS messages, and reacting to specific service actions through Event Patterns. Key features include:

  • Event Patterns: Rules specified in AWS configs react to certain service actions.
  • Permissions: EventBridge rules require permissions on the target resource or IAM roles allowing EventBridge.
  • Schema Inference: Analyzes events and infers an associated schema, allowing code generation for applications.
  • Event Buses: Supports default, partner, and custom event buses, accessible across AWS accounts and regions.

SQS⚓︎

Amazon Simple Queue Service (SQS) is a messaging service with key features, including:

  • IAM Role Permissions: IAM role permissions are required for writing to SQS.
  • Access Policy: Access policies are needed for receiving messages from SNS.
  • Message Retention: Messages are retained from 1 minute to 14 days, with a default of 4 days.
  • Message Limits: Messages must be 256k or less, and up to 10 messages can be received at a time.
  • Consumer Flexibility: Can have multiple consumers, and ordering is guaranteed only on FIFO Queues.
  • Batch Processing: Batch mode is available, supporting up to 3,000 messages per second.
  • Queue Conversion: To switch between standard and FIFO queues, new queues must be created.
  • Visibility Timeout: Determines how long a message is invisible in the queue after being picked by a reader.
  • Decoupling Applications: SQS is ideal for decoupling applications and follows a pull-based data model.

SNS⚓︎

Amazon Simple Notification Service (SNS) is a messaging service with key features, including:

  • Filter Policies: Allows the creation of filter policies for certain SQS consumers to receive specific messages.
  • FIFO Support: FIFO queues can be used with SNS for ordering and deduplication.
  • Push-Based Data: Operates on a push-based data model.
  • Subscription Variety: Subscribers can include Platform Application Endpoint, SQS, HTTP(s) endpoints, Email/Email-JSON, AWS Lambda, Amazon Kinesis Data Firehose, and SMS.
  • Encryption and Access Control: Supports encryption in flight and at rest, and access is controlled via IAM or SNS access policies.

For more advanced use cases, SNS integrates with SQS for fanout (requires SNS to be FIFO). It is suitable for pub/sub scenarios with up to 100,000 topics and 12,500,000 subscribers. SNS is not meant for data persistence, and data is lost if not delivered. It also supports cross-region delivery, and topic events can be archived, filtered, and replayed.