Skip to content

Storage Configurations on AWS⚓︎

Amazon Elastic Block Store (EBS)⚓︎

Amazon EBS offers versatile storage options with features tailored to various use cases:

  • Volume Types: EBS volumes function as virtual hard disks, supporting different types such as General Purpose (gp2), Provisioned IOPS (io1/io2), and more.

  • Snapshot Management: Snapshots, point-in-time copies of disks, reside on Amazon S3 and are incremental, capturing only changed blocks. The initial snapshot may take longer.

  • Multi-Attach Support: EBS volumes can be attached to multiple EC2 instances within the same Availability Zone, making it suitable for database workloads.

  • Dynamic Adjustments: Volumes allow dynamic resizing and storage type changes, providing flexibility in adapting to evolving needs.

  • Region and AZ Specific: EBS volumes are specific to a region but can be migrated to other AZs through snapshots.

  • Snapshot Archiving: EBS snapshot archives offer cost-effective storage, although restoration may take 24-72 hours.

Amazon Machine Image (AMI) Types (EBS vs Instance Store)⚓︎

Understanding AMI types aids in selecting suitable storage solutions for instances:

  • Instance Store: Provides high random I/O performance and is ephemeral. Data is lost if the host fails, and these instances can't be stopped.

  • EBS Backed Instances: Support stopping without data loss. Users can choose to retain root volumes on termination, offering more flexibility.

  • Boot Volumes: Various types like gp2, gp3, io1, io2, and magnetic (std), with EBS volumes specific to an AZ but migratable via snapshots.

Encrypted vs Unencrypted AMIs⚓︎

Differentiating between encrypted and unencrypted AMIs is crucial for security considerations:

  • Cross-Region Copy: Copying an AMI to another region automatically generates a snapshot in the destination region.

  • Encryption Cascade: Snapshots of encrypted volumes and volumes from encrypted snapshots are both encrypted.

  • Sharing Restrictions: Unencrypted snapshots can be shared, while encrypted ones can only be shared with other AWS accounts or made public.

  • Root Device Encryption: Users can encrypt root device volumes upon EC2 instance creation or convert unencrypted volumes by creating encrypted snapshots.

Amazon Elastic File System (EFS)⚓︎

Amazon EFS, designed for Linux-based systems, provides scalable and flexible file storage:

  • Multiple Mounting: EFS can be mounted on multiple EC2 instances using Security Groups for access control.

  • Performance Modes: General Purpose for latency-sensitive use cases and Max I/O for high throughput and parallelism.

  • Throughput Modes: Choose between Bursting (variable throughput) and Provisioned (consistent throughput) options.

  • Storage Tiers: Standard for frequently accessed files and Infrequent Access (EFS-IA) for cost-effective storage with retrieval costs.

  • Availability and Durability: Standard offers multi-AZ support, while One Zone is suitable for development with built-in backup.

Amazon FSx⚓︎

Amazon FSx provides managed file systems with various types catering to specific needs:

  • Types: FSx for Windows File Server, FSx for Lustre, and FSx for NetApp ONTAP supporting NFS, SMB, and iSCSI protocols.

  • Features: FSx for Windows offers SMB and Windows NTFS support, Active Directory integration, user quotas, and daily S3 backups.

  • FSx for Lustre: Designed for high-performance computing with seamless integration with S3 for data processing.

AWS Storage Gateway⚓︎

AWS Storage Gateway facilitates hybrid cloud storage with support for different protocols:

  • Volume and Tape Backups: Volumes backed by EBS snapshots, tapes backed by S3 or S3 Glacier, supporting access to AWS storage classes.

  • SMB/NTFS Integration: Integrates with Windows Active Directory for seamless SMB/NTFS access.

  • Versatile Protocols: Supports S3, FSx (SMB/NTFS), tape (iSCSI), and volume (iSCSI) interfaces.

Instance/DB Snapshots⚓︎

  • Storage Location: Stored in an S3 bucket within the same AWS region as the instance.

  • Snapshot Access: Direct access to snapshots in S3 is not provided, but sharing options are available.

AWS Snow Family⚓︎

AWS Snow Family includes various devices for offline data migration and edge computing:

  • Snowball Edge: Physical transport for TBs/PBs of data, offering block and S3-compatible object storage. Suitable for large data migrations.

  • Snowcone: Portable, lightweight, and rugged device for edge computing, with 8TB storage. Ideal for space-constrained environments.

  • Snowmobile: Transfers exabytes of data with 100PB capacity, suitable for massive data transfers.

  • Edge Computing Options: Snowcone, Snowball Edge (Compute and Storage Optimized), offering CPU and storage options for various use cases.

AWS OpsHub⚓︎

AWS OpsHub simplifies the management of Snow Family devices:

  • Device Management: Unlocking, configuring, and monitoring single or clustered devices.

  • File Transfer: Facilitates file transfers and launching compatible AWS services on devices.

  • Monitoring: Provides metrics monitoring for storage capacity and active instances.

  • User Interface: Offers a user-friendly interface, replacing the need for a CLI tool.

S3 Advanced Features⚓︎

S3 Batch Replication⚓︎

  • Object Replication: Replicates objects existing before configuring replication, ensuring consistency across regions.

  • Configurational Differences: Differs from live replication and requires special considerations for existing objects.

S3 Sync Command⚓︎

  • Efficient Copying: Uses copy object APIs for efficient copying between S3 buckets, considering versioning and last modified dates.

  • Error Recovery: In case of failures, the command can be rerun without duplicating previous copies, facilitating large data transfers.

Origin Access Control (OAC)⚓︎

  • Access Restriction: Restricts access to S3, ensuring access through intended CloudFront distributions. Replaces the previous Origin Access Identity (OAI).

  • Secure Configuration: Supports AWS KMS (SSE-KMS) for enhanced security, allowing dynamic requests to S3.

Origin Access Identity (OAI)⚓︎

  • Access Restriction: Similar to OAC, it restricts S3 access, ensuring access through intended CloudFront distributions.

  • Authentication Control: Can be used to allow only authenticated access through CloudFront configurations.