Storage Configurations on AWS⚓︎
Amazon Elastic Block Store (EBS)⚓︎
Amazon EBS offers versatile storage options with features tailored to various use cases:
-
Volume Types: EBS volumes function as virtual hard disks, supporting different types such as General Purpose (gp2), Provisioned IOPS (io1/io2), and more.
-
Snapshot Management: Snapshots, point-in-time copies of disks, reside on Amazon S3 and are incremental, capturing only changed blocks. The initial snapshot may take longer.
-
Multi-Attach Support: EBS volumes can be attached to multiple EC2 instances within the same Availability Zone, making it suitable for database workloads.
-
Dynamic Adjustments: Volumes allow dynamic resizing and storage type changes, providing flexibility in adapting to evolving needs.
-
Region and AZ Specific: EBS volumes are specific to a region but can be migrated to other AZs through snapshots.
-
Snapshot Archiving: EBS snapshot archives offer cost-effective storage, although restoration may take 24-72 hours.
Amazon Machine Image (AMI) Types (EBS vs Instance Store)⚓︎
Understanding AMI types aids in selecting suitable storage solutions for instances:
-
Instance Store: Provides high random I/O performance and is ephemeral. Data is lost if the host fails, and these instances can't be stopped.
-
EBS Backed Instances: Support stopping without data loss. Users can choose to retain root volumes on termination, offering more flexibility.
-
Boot Volumes: Various types like gp2, gp3, io1, io2, and magnetic (std), with EBS volumes specific to an AZ but migratable via snapshots.
Encrypted vs Unencrypted AMIs⚓︎
Differentiating between encrypted and unencrypted AMIs is crucial for security considerations:
-
Cross-Region Copy: Copying an AMI to another region automatically generates a snapshot in the destination region.
-
Encryption Cascade: Snapshots of encrypted volumes and volumes from encrypted snapshots are both encrypted.
-
Sharing Restrictions: Unencrypted snapshots can be shared, while encrypted ones can only be shared with other AWS accounts or made public.
-
Root Device Encryption: Users can encrypt root device volumes upon EC2 instance creation or convert unencrypted volumes by creating encrypted snapshots.
Amazon Elastic File System (EFS)⚓︎
Amazon EFS, designed for Linux-based systems, provides scalable and flexible file storage:
-
Multiple Mounting: EFS can be mounted on multiple EC2 instances using Security Groups for access control.
-
Performance Modes: General Purpose for latency-sensitive use cases and Max I/O for high throughput and parallelism.
-
Throughput Modes: Choose between Bursting (variable throughput) and Provisioned (consistent throughput) options.
-
Storage Tiers: Standard for frequently accessed files and Infrequent Access (EFS-IA) for cost-effective storage with retrieval costs.
-
Availability and Durability: Standard offers multi-AZ support, while One Zone is suitable for development with built-in backup.
Amazon FSx⚓︎
Amazon FSx provides managed file systems with various types catering to specific needs:
-
Types: FSx for Windows File Server, FSx for Lustre, and FSx for NetApp ONTAP supporting NFS, SMB, and iSCSI protocols.
-
Features: FSx for Windows offers SMB and Windows NTFS support, Active Directory integration, user quotas, and daily S3 backups.
-
FSx for Lustre: Designed for high-performance computing with seamless integration with S3 for data processing.
AWS Storage Gateway⚓︎
AWS Storage Gateway facilitates hybrid cloud storage with support for different protocols:
-
Volume and Tape Backups: Volumes backed by EBS snapshots, tapes backed by S3 or S3 Glacier, supporting access to AWS storage classes.
-
SMB/NTFS Integration: Integrates with Windows Active Directory for seamless SMB/NTFS access.
-
Versatile Protocols: Supports S3, FSx (SMB/NTFS), tape (iSCSI), and volume (iSCSI) interfaces.
Instance/DB Snapshots⚓︎
-
Storage Location: Stored in an S3 bucket within the same AWS region as the instance.
-
Snapshot Access: Direct access to snapshots in S3 is not provided, but sharing options are available.
AWS Snow Family⚓︎
AWS Snow Family includes various devices for offline data migration and edge computing:
-
Snowball Edge: Physical transport for TBs/PBs of data, offering block and S3-compatible object storage. Suitable for large data migrations.
-
Snowcone: Portable, lightweight, and rugged device for edge computing, with 8TB storage. Ideal for space-constrained environments.
-
Snowmobile: Transfers exabytes of data with 100PB capacity, suitable for massive data transfers.
-
Edge Computing Options: Snowcone, Snowball Edge (Compute and Storage Optimized), offering CPU and storage options for various use cases.
AWS OpsHub⚓︎
AWS OpsHub simplifies the management of Snow Family devices:
-
Device Management: Unlocking, configuring, and monitoring single or clustered devices.
-
File Transfer: Facilitates file transfers and launching compatible AWS services on devices.
-
Monitoring: Provides metrics monitoring for storage capacity and active instances.
-
User Interface: Offers a user-friendly interface, replacing the need for a CLI tool.
S3 Advanced Features⚓︎
S3 Batch Replication⚓︎
-
Object Replication: Replicates objects existing before configuring replication, ensuring consistency across regions.
-
Configurational Differences: Differs from live replication and requires special considerations for existing objects.
S3 Sync Command⚓︎
-
Efficient Copying: Uses copy object APIs for efficient copying between S3 buckets, considering versioning and last modified dates.
-
Error Recovery: In case of failures, the command can be rerun without duplicating previous copies, facilitating large data transfers.
Origin Access Control (OAC)⚓︎
-
Access Restriction: Restricts access to S3, ensuring access through intended CloudFront distributions. Replaces the previous Origin Access Identity (OAI).
-
Secure Configuration: Supports AWS KMS (SSE-KMS) for enhanced security, allowing dynamic requests to S3.
Origin Access Identity (OAI)⚓︎
-
Access Restriction: Similar to OAC, it restricts S3 access, ensuring access through intended CloudFront distributions.
-
Authentication Control: Can be used to allow only authenticated access through CloudFront configurations.