AWS STS Integration

AWS STS enables setting up temporary, limited-privilege credentials for IAM users, allowing authenticated third-party users to access data.

To set up STS in AWS and connect it to Dataloop, follow these instructions:

1. Log in to the AWS Management Console
2. Go to the S3 service
3. Click on the "Create bucket" button
4. Provide a unique bucket name - related to DataLoop
5. Make sure Block Public Access settings for this bucket - Block all public access
6. For all other optional settings use default values
7. Click on the "Create bucket" button

For a step-by-step guide on creating an S3 bucket in AWS, read the AWS documentation.

Create an IAM Policy

1. Go to the IAM service
2. Click on “Policies” in the left navigation panel
3. Click on the "Create policy" button
4. Select the JSON tab
5. Define policy document in JSON format, like:
   TextText

   {
       "Version": "2012-10-17",
       "Statement": [
           {
               "Sid": "AllowListBucket",
               "Effect": "Allow",
               "Action": [
                   "s3:ListBucket",
                   "s3:PutObject",
                   "s3:DeleteObject",
                   "s3:GetObject"
               ],
               "Resource": [
   	            "arn:aws:s3:::<DataLoopBucketName>",
                   "arn:aws:s3:::<DataLoopBucketName>/*"
               ]
           }
       ]
   }

   
   
   Note: Replace <DataLoopBucketName> with the name of the desired S3 bucket.
   
   6. Click on the next button to Review the policy
   7. Provide a meaningful name for the policy and an optional description
   8. Click the “Create policy” button to finish

Notes:
a. “s3:DeleteObject” action allows DataLoop platform to delete dataset items - please see downstream
b. The resource arn:aws:s3:::<DataLoopBucketName> is required for the ListBucket action

For a step-by-step guide on creating an IAM policy in AWS, read AWS documentation.

Create an IAM User without permissions

1. Go to the IAM service
2. Click on “Users” in the left navigation panel
3. Click on the "Add users" button
4. Enter a user name for the new user
5. Select "Programmatic access" as the type of access you want to grant the user - Or leave as default
   Note: Use the default access type and don't choose console access
6. Click on the next button 
7. Don’t choose any permission - the user is not supposed to have any access permissions 
8. Click on the next button 
9. Click on the "Create user" button to create the user

For a step-by-step guide on creating an IAM user in AWS, read AWS docs.

Create an Access Key for the IAM User

1. Go to the IAM service
2. Click on “Users” in the left navigation panel
3. Find and choose the desired user whom you want to create an access key and click on the user's name
   NOTE: The DataLoop user you created
4. Click on the “Security credentials” tab
5. In the “Access keys” section, click on “Create access key”
6. Under “Access key best practices & alternatives” choose “Application running outside AWS”
7. Click “Next”
8. Add an informative and meaningful description that include DataLoop in it
9. Click on “Create access key”
10. Copy and save the access key ID and secret access key in a secure location OR download the CSV file containing the keys
    NOTE: You’ll need the access key ID and secret access key for the integration phase
11. Click on “Done” button

Create an IAM Role

1. Go to the IAM service
2. Click on “Roles” in the left navigation panel
3. Click on the "Create role" button
4. Choose "AWS service" as the trusted entity type
5. Choose "EC2" as Use case
6. Click “Next”
7. Search and select the related policy you created for accessing the S3 bucket
8. Click “Next”
9. Give the role a meaningful name, and optional description.
10. Click on the "Create role" button to create the IAM role.
11. NOTE: You’ll need the Role ARN for the integration phase

Update the Trust relationship of the Role to allow the IAM user to “Assume Role”

1. Go to the IAM service
2. Click on “Roles” in the left navigation panel
3. Search and select the IAM role whose trust relationship you want to change - The one you created
4. Click on the "Trust relationships" tab
5. Click on the "Edit trust relationship" button 
6. Replace the current trust policy JSON document with : 

{    "Version": "2012-10-17",

    "Statement": [

        {

            "Effect": "Allow",

            "Principal": {

     "AWS":"arn:aws:iam::<AWSAccountId>:user/<IAMUserName>"

            },

            "Action": "sts:AssumeRole"

        }

    ]

}

Note:
A. Replace <AWSAccountId> with your AWS account id
B. Replace <IAMUserName> with the name of the desired IAM User

7. Click on the "Update Policy" button to save the changes

Create an AWS STS Integration on DataLoop platform

1. On the Dataloop platform navigate to the left-side panel and choose integrations
2. Click on the “CREATE NEW INTEGRATION” button
3. Enter a meaningful name for the integration
4. Under Type, choose “AWS STS”
5. Under Key, enter the IAM user’s access key ID from earlier 
6. Under Secret, enter the IAM user’s secret access key from earlier 
7. Under Role ARN, enter the Role ARN from earlier 
8. Click on Create 

Create an AWS STS Storage Driver on DataLoop platform

1. On the Dataloop platform navigate to the left-side panel, choose “Data Management” and then click on “Cloud Storage” 
2. Click on the “CREATE DRIVER” button
3. Enter a meaningful name for the storage driver
4. Under Integration choose your relevant AWS STS Integration
5. Enter your AWS Bucket name (The one that is relevant to the integration)
6. Select the region your S3 bucket is located in 
7. (Optional) Allow delete items
8. Click “TEST” to test if the connection is successful 
9. Click “Create”