-
Print
-
DarkLight
-
PDF
AWS Cross Account Integration
-
Print
-
DarkLight
-
PDF
Important
Organization can create up to 15 Cross Account integration.
If you would like to increase the number of Cross Account integrations for your organization, please contact our Customer support team
Create an S3 Bucket
- Log in to the AWS Management Console
- Go to the S3 service
- Click on the "Create bucket" button
- Provide a unique bucket name - related to DataLoop
- Make sure to block public access settings for this bucket - Block all public access
- For all other optional settings use default values
- Click on the "Create bucket" button
For a step-by-step guide on creating an S3 bucket in AWS, read AWS documentation
Create an IAM Policy
- Go to the IAM service
- Click on “Policies” in the left navigation panel
- Click on the "Create policy" button
- Select the JSON tab
- Define policy document in JSON format, like:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowListBucket",
"Effect": "Allow",
"Action": [
"s3:ListBucket",
"s3:PutObject",
"s3:DeleteObject",
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::<DataLoopBucketName>",
"arn:aws:s3:::<DataLoopBucketName>/*"
]
}
]
}
Note:
Replace <DataLoopBucketName>
with the name of the desired S3 bucket.
- Click on the next button to Review the policy
- Provide a meaningful name for the policy and an optional description
- Click the “Create policy” button to finish
Note:
a. “s3:DeleteObject” action allows DataLoop platform to delete dataset items - please see downstream
b. The resource arn:aws:s3:::<DataLoopBucketName>
is required for the ListBucket
action
For a step-by-step guide on creating an IAM policy in AWS, read AWS documentation.
Start the AWS Cross Account Integration on DataLoop platform
- On the Dataloop platform navigate to the left-side panel and choose integrations
- Click on the “Create New Integration” button
- Enter a meaningful name for the integration
- Under Type, choose “Cross Account”
- Click on the “Get IAM user”
Note: In a case you already created one you can choose from a list of created IAM users that have not been assigned to an integration - Copy the IAM user ARN
- Open a new tab and go to your AWS console
Grant Dataloop IAM user access to an IAM role
- Go to the IAM service
- Click on “Roles” in the left navigation panel
- Click on the "Create role" button
- Under "Select trust entity" choose "Customer trust policy"
- Define the Trust policy document in JSON format, like:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::<Dataloop_Account_Id>:user/<Dataloop_IAM_User>"
},
"Action": "sts:AssumeRole"
}
]
}
- Replace the AWS principal with the IAM user ARN you copied from Dataloop platform
- Click Next
- Under Add permissions choose the IAM policy you created earlier
- Click Next
- Choose a name for the role
- (Optional) add description
- Click the “Create role” button to finish
Note: You can also create an IAM role in advance and edit his trust relationship
For a step-by-step guide on editing an IAM role trust relationship in AWS, read AWS documentation.
Finish the AWS Cross Account Integration on DataLoop platform:
- Go back to the Dataloop platform
- Under Role ARN, enter the IAM role ARN that you edited his trust relationship
- Click on Create
Create an AWS S3 Storage Driver on DataLoop platform
- On the Dataloop platform navigate to the left-side panel, choose “Data Management” and then click on “Cloud Storage”
- Click on the “CREATE DRIVER” button
- Enter a meaningful name for the storage driver
- Under Integration choose your relevant AWS Cross Account Integration
- Enter your AWS Bucket name (The one that is relevant to the integration)
- Select the region your S3 bucket is located in
- (Optional) Allow delete items
- Click “TEST” to test if the connection is successful
- Click “Create”