Integrations (Secrets)
  • Dark
  • PDF

Integrations (Secrets)

  • Dark
  • PDF

Cloud Integrations/Secrets

Integrations are created and saved at the organization-level, making it accessible to all projects in the organization.

Integrations enables Dataloop organizations to define secrets for accessing cloud resources, including cloud storage such as GCS/S3, Secure Token Service (STS), container registry services (ECR/GCR) and others.

  1. Storage -
    1. AWS (Amazon S3)
    2. Google Cloud (GCS)
    3. Azure (Blob)
  2. Private container registry services (ECR/GCR)
  3. Secure Token Service (STS) - Amazon

Once an integration is defined for cloud storage, a storage-driver must be created with storage details such as bucket, folder etc. See full guides for each storage type: S3/GCS/Azure. Additional infromation is provided here.

Adding Integration/Secret

To add a new integration or secret:

  1. From the left-side menu, locate the Organization related area at the bottom and select ‘Integrations’

If you do not see the integrations menu, make sure you are a member of the organization, with sufficient permissions, and that it is set as your active Organization

  1. Click on create new integration icon(1) icon

  2. Name the integration and select the integration type from the drop-down list;

  3. Amazon S3 - Provide key and secret information.

  4. Google cloud storage - Provide Private key (JSON) information.

  5. Azure - Provide Account Name, Application ID, Directory ID and Client secret information.

  6. AWS-STS - Provide Key, Secret and Role ARN information

  7. Private container registry - Select between:

    1. Google Container Registry (GCR) - provide server and password information.
    2. Elastic Container Registry (ECR) - provide account, region, access key id and secret key id information.

Connecting Cloud Storage

After creating an integration with your cloud storage provider (S3 / AWS / Azure), continue to create a storage-drive in the Dataloop system, and connect it with a Dataset. Read here for detailed instructions.

What's Next