- 31 Oct 2023
- DarkLight
- PDF
Cross Account Integration
- Updated On 31 Oct 2023
- DarkLight
- PDF
Organizations can create up to 15 Cross Account integrations.
To increase the number of Cross Account integrations for your organization, please contact our Customer support team.
The AWS cross-account integration process involves:
Create an S3 Bucket
- Log in to the AWS Management Console.
- Go to Services > Storage and click the S3 service.
- Click Create bucket. The Create Bucket page is displayed.
- Provide a Bucket name.
- Select your AWS region from the list.
- Ensure you block public access settings for this bucket - Block all public access.
- For all other optional settings, use the default values.
- Click Create bucket. A confirmation message is displayed.
For a step-by-step guide on creating an S3 bucket in AWS, see Creating a bucket.
Create an IAM Policy
- Log in to the AWS Management Console.
- Go to Services and click All services. A list of services is displayed.
- Select the IAM service from the list.
- Click on Policies from the left navigation panel.
- Click Create policy. The Create Policy page is displayed.
- Select the JSON tab and define the policy document in the following JSON format:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowListBucket",
"Effect": "Allow",
"Action": [
"s3:ListBucket",
"s3:PutObject",
"s3:DeleteObject",
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::<DataloopBucketName>",
"arn:aws:s3:::<DataloopBucketName>/*"
]
}
]
}
Replace <DataloopBucketName>
with the name of the your S3 bucket.
- Click Next.
- Enter a Name for the policy and an optional Description.
- Click Create policy. A confirmation message is displayed.
s3:DeleteObject
action allows Dataloop platform to delete dataset items, see downstream.- The resource
arn:aws:s3:::<DataloopBucketName>
is required for theListBucket
action.
For a step-by-step guide on creating an IAM policy in AWS, see Creating IAM policies.
In a case you want to restrict your IAM policy further please take a look at S3 Restricted Folder Access.
Create an IAM Role
- Log in to the AWS Management Console.
- Go to Services and click All services. A list of services is displayed.
- Click the IAM service from the list.
- From the left portal menu, click Roles.
- Click Create role.
- Choose AWS service as the trusted entity type.
- Choose EC2 as the use case.
- Click Next.
- Search and select the policy that you created for accessing the S3 bucket.
- Click Next.
- Enter a name and an optional description for the role.
- Click Create role. A confirmation message is displayed.
- Click the Role that you created from the list.
- Copy the ARN value, which is required during the integration phase.
- For a step-by-step guide on creating an S3 bucket in AWS, see Creating IAM roles.
Start the AWS Cross Account Integration on Dataloop platform
- Log in to the Dataloop platform.
- From the left portal menu, select Integrations.
- Click Create New Integration. A pop-up window is displayed.
- Enter a Name for the integration.
- Select Cross Account from the Integration Type list.
- Click Get IAM user.
Note: If you have created one already, you can choose it from a list of IAM users that have not been assigned to an integration. - Copy the IAM user's ARN.
- Open a new tab and go to your AWS console.
Grant Dataloop IAM user access to an IAM role
- Log in to the AWS Management Console.
- Go to Services and click All services. A list of services is displayed.
- Select the IAM service from the list.
- Click Roles from the left navigation panel.
- Choose and click on the role which you recently created.
- Click on Trust relationship tab.
- Click Edit trust policy.
- Define the Trust relationship document in JSON format as follows:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::<Dataloop_Account_Id>:user/<Dataloop_IAM_User>"
},
"Action": "sts:AssumeRole"
}
]
}
- Replace the AWS principal with the IAM user ARN you copied from the Dataloop platform.
- Click Update policy.
For a step-by-step guide on editing an IAM role trust relationship in AWS, see Editing the trust relationship for an existing role.
Complete the AWS Cross Account Integration on Dataloop Platform
- Log in to the Dataloop platform.
- Under the Role ARN field, enter the ARN of the IAM role for which you edited the Trust relationship.
- Click Create. A confirmation message is displayed.
Create S3 Storage Service on the Dataloop Platform
For more information, see the Create AWS S3 Storage Driver on the Dataloop Platform topic.