Integrations (Secrets)
-
Print
-
DarkLight
-
PDF
Integrations (Secrets)
-
Print
-
DarkLight
-
PDF
Cloud Integrations/Secrets
Integrations are created and saved at the organization-level, making it accessible to all projects in the organization.
Integrations enables Dataloop organizations to define secrets for accessing cloud resources, including cloud storage such as GCS/S3, Secure Token Service (STS), container registry services (ECR/GCR) and others.
- Storage -
- AWS (Amazon S3)
- Google Cloud (GCS)
- Azure (Blob)
- Private container registry services (ECR/GCR)
- Secure Token Service (STS) - Amazon
Once an integration is defined for cloud storage, a storage-driver must be created with storage details such as bucket, folder etc. See full guides for each storage type: S3/GCS/Azure. Additional infromation is provided here.
Adding Integration/Secret
To add a new integration or secret:
-
From the left-side menu, locate the Organization related area at the bottom and select ‘Integrations’
If you do not see the integrations menu, make sure you are a member of the organization, with sufficient permissions, and that it is set as your active Organization
-
Click on
icon -
Name the integration and select the integration type from the drop-down list;
-
Amazon S3 - Provide key and secret information.
-
Google cloud storage - Provide Private key (JSON) information.
-
Azure - Provide Account Name, Application ID, Directory ID and Client secret information.
-
AWS-STS - Provide Key, Secret and Role ARN information
-
Private container registry - Select between:
- Google Container Registry (GCR) - provide server and password information.
- Elastic Container Registry (ECR) - provide account, region, access key id and secret key id information.
Connecting Cloud Storage
After creating an integration with your cloud storage provider (S3 / AWS / Azure), continue to create a storage-drive in the Dataloop system, and connect it with a Dataset. Read here for detailed instructions.