AWS
Amazon Web Service (AWS) for Data Engineering
Throughout the different posts, we will work with services such as Amazon S3, IAM, AWS Glue, Amazon Athena, and other key components of the AWS ecosystem. Each article will include practical examples, code snippets, and clear explanations, allowing you to reproduce the exercises in your own AWS account.
The goal is that, by following this step-by-step approach, you will be able to design, implement, and maintain scalable and secure data solutions on AWS, both for personal projects and enterprise environments.
At the end of this post, you will find instructions for releasing the resources used throughout the exercises, in order to avoid additional charges to your AWS account.
In this example, we use a free dataset called Brazilian E-Commerce Public Dataset by Olist, wich is available in Kaggle.com.
Amazon S3 as Data Lake
1. After signing up for Amazon Web service, type 'S3' into the search bar and select S3 service and click 'create a bucket'
2. The bucket name is proyecto-olis-datos-2026, but you can use any other name you prefer. Keep all the default setting and click 'Create bucket'
Once the bucket has been created, the following table should appear.
3. Creating Folders in the proyecto-olis-datos-2026 Bucket Using CloudShell
| # Definir la variable del bucket |
| BUCKET_NAME="proyecto-olist-datos-2026" |
| # Crear la carpeta procesada |
| aws s3api put-object --bucket $BUCKET_NAME --key processed/ |
| # Crear la carpeta raw y sus subcarpetas |
| aws s3api put-object --bucket $BUCKET_NAME --key raw/ |
| aws s3api put-object --bucket $BUCKET_NAME --key raw/orders/ |
| aws s3api put-object --bucket $BUCKET_NAME --key raw/customers/ |
The command has created two folder called raw and processed. Inside the raw folder , It has creates two subfolder called orders and customers.
4. Verifying folder creation inside proyecto-olist-datos-2026 bucket
| ~ $ aws s3 ls s3://$BUCKET_NAME/ --recursive |
| 2026-08-30 04:32:47 0 processed/ |
| 2026-08-30 04:33:11 0 raw/ |
| 2026-08-30 04:33:45 0 raw/customers/ |
| 2026-08-30 04:33:32 0 raw/orders/ |
5.Upload the files olist_orders_dataset.csv to raw/orders/ and olist_customers_dataset.csv to raw/customers/
Creating IAM security roles for AWS Glue
1.Type 'IAM' into the search bar and select IAM service and select roles option. Select trusted entity and select Glue as service or use case and Glue
2. Select the permission policies. In this case, we will select:
- AWSGlueServiceRole
- AmazonS3FullAccess
3.Name and create the role: GlueServiceRoleProject
Selecting the appropriate region is essential. In this example, the US East (N. Virginia) region will be used.
