Contents x
    Cross Account Integration
    • 31 Oct 2023
    • Dark
      Light
    • PDF
    Contents

    Cross Account Integration

    • Updated On 31 Oct 2023
    • Dark
      Light
    • PDF
    Article Summary

    Important

    Organizations can create up to 15 Cross Account integrations.
    To increase the number of Cross Account integrations for your organization, please contact our Customer support team.

    The AWS cross-account integration process involves:

    Create an S3 Bucket

    1. Log in to the AWS Management Console.
    2. Go to Services > Storage and click the S3 service.
    3. Click Create bucket. The Create Bucket page is displayed.
    4. Provide a Bucket name.
    5. Select your AWS region from the list.
    6. Ensure you block public access settings for this bucket - Block all public access.
    7. For all other optional settings, use the default values.
    8. Click Create bucket. A confirmation message is displayed.

    For a step-by-step guide on creating an S3 bucket in AWS, see Creating a bucket.

    Create an IAM Policy

    1. Log in to the AWS Management Console.
    2. Go to Services and click All services. A list of services is displayed.
    3. Select the IAM service from the list.
    4. Click on Policies from the left navigation panel.
    5. Click Create policy. The Create Policy page is displayed.
    6. Select the JSON tab and define the policy document in the following JSON format:
    {
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowListBucket",
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:PutObject",
                "s3:DeleteObject",
                "s3:GetObject"
            ],
            "Resource": [
                "arn:aws:s3:::<DataloopBucketName>",
                "arn:aws:s3:::<DataloopBucketName>/*"
            ]

        }
    ]
}
    Note:

    Replace <DataloopBucketName> with the name of the your S3 bucket.

    1. Click Next.
    2. Enter a Name for the policy and an optional Description.
    3. Click Create policy. A confirmation message is displayed.
    Note:
    • s3:DeleteObject action allows Dataloop platform to delete dataset items, see downstream.
    • The resource arn:aws:s3:::<DataloopBucketName> is required for the ListBucket action.

    For a step-by-step guide on creating an IAM policy in AWS, see Creating IAM policies.

    S3 Restricted Folder Access

    In a case you want to restrict your IAM policy further please take a look at S3 Restricted Folder Access.

    Create an IAM Role

    1. Log in to the AWS Management Console.
    2. Go to Services and click All services. A list of services is displayed.
    3. Click the IAM service from the list.
    4. From the left portal menu, click Roles.
    5. Click Create role.
    6. Choose AWS service as the trusted entity type.
    7. Choose EC2 as the use case.
    8. Click Next.
    9. Search and select the policy that you created for accessing the S3 bucket.
    10. Click Next.
    11. Enter a name and an optional description for the role.
    12. Click Create role. A confirmation message is displayed.
    13. Click the Role that you created from the list.
    Note:
    • Copy the ARN value, which is required during the integration phase.
    • For a step-by-step guide on creating an S3 bucket in AWS, see Creating IAM roles.

    Start the AWS Cross Account Integration on Dataloop platform

    1. Log in to the Dataloop platform.
    2. From the left portal menu, select Integrations.
    3. Click Create New Integration. A pop-up window is displayed.
    4. Enter a Name for the integration.
    5. Select Cross Account from the Integration Type list.
    6. Click Get IAM user.
      Note: If you have created one already, you can choose it from a list of IAM users that have not been assigned to an integration.
    7. Copy the IAM user's ARN.
    8. Open a new tab and go to your AWS console.

    Grant Dataloop IAM user access to an IAM role

    1. Log in to the AWS Management Console.
    2. Go to Services and click All services. A list of services is displayed.
    3. Select the IAM service from the list.
    4. Click Roles from the left navigation panel.
    5. Choose and click on the role which you recently created.
    6. Click on Trust relationship tab.
    7. Click Edit trust policy.
    8. Define the Trust relationship document in JSON format as follows:
    {
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::<Dataloop_Account_Id>:user/<Dataloop_IAM_User>"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}
    1. Replace the AWS principal with the IAM user ARN you copied from the Dataloop platform.
    2. Click Update policy.

    For a step-by-step guide on editing an IAM role trust relationship in AWS, see Editing the trust relationship for an existing role.

    Complete the AWS Cross Account Integration on Dataloop Platform

    1. Log in to the Dataloop platform.
    2. Under the Role ARN field, enter the ARN of the IAM role for which you edited the Trust relationship.
    3. Click Create. A confirmation message is displayed.

    Create S3 Storage Service on the Dataloop Platform

    For more information, see the Create AWS S3 Storage Driver on the Dataloop Platform topic.

    What's Next