Contents x
    Private Key Integration
    • 18 Sep 2023
    • Dark
      Light
    • PDF
    Contents

    Private Key Integration

    • Updated On 18 Sep 2023
    • Dark
      Light
    • PDF
    Article Summary

    To connect your data from GCS bucket as a Dataloop dataset, follow the setup instructions.

    Create an IAM Role

    1. Log in to the Google Cloud Console.
    2. From the left portal menu, select the IAM & admin > Roles.
    3. Click +Create Role.
    4. Enter a role Title.
    5. (Optional) Enter a Description.
    6. (Optional) Enter a role ID. By default, the role id is generated.
    7. Click +Add Permissions, then search for and add the following permissions:
      1. storage.objects.create
      2. storage.objects.delete
      3. storage.objects.get
      4. storage.objects.list
      5. storage.buckets.get
    Note: The storage.objects.delete permission allows the Dataloop platform to delete dataset items. For more information, see the Downstream section.
    1. Click Create.
    Tip
    • See Create and manage custom roles for more information on creating an IAM role in GCP.
    • To display Roles information, you must select a project in the Google Cloud Console. If not available, create a project.

    Create a Service Account

    1. Log in to the Google Cloud Console.
    2. From the left portal menu, select the IAM & admin > Service accounts.
    3. Click +Create service account.
    4. (Optional) Enter a Name and Description for the account.
    5. Click Create and continue.
    6. Click Done to create the service account.
    Tip: See Create service accounts for more information on creating a service account in GCP.

    Create a Private Key for the Service Account

    1. Log in to the Google Cloud Console.
    2. From the left portal menu, select the IAM & admin > Service accounts.
    3. Find the desired service account for which you want to add a private key, and click on its name to open its details.
      Note: The Service Account you recently created.
    4. Select the Keys tab.
    5. Click Add key.
    6. Click Create new key from the list. A popup window is displayed.
    7. Ensure the Key type is in JSON format.
    8. Click Create. The "Private key saved to your computer" message is displayed.
    Note
    • Use the key to authenticate as the service account on the Dataloop platform.
    • Save the JSON file in a convenient location, it is required at the integration phase.

    Create a Cloud Storage Bucket

    1. Log in to Google Cloud Console.
    2. From the left portal menu, go to the Cloud Storage > Buckets.
    3. Click Create bucket.
    4. Enter a Name for the storage bucket.
    5. Click Continue.
    6. In the Choose where to store your data, select a location for the bucket.
    7. Click Continue.
    8. Click Create bucket.
    Note: For all other optional settings, use the default values.

    Grant Permissions to the Cloud Storage Bucket

    1. Log in to Google Cloud Console.
    2. From the left portal menu, go to the Cloud Storage > Buckets.
    3. Select the Storage bucket for which you want to add permissions, and then click on it.
    4. Select the Permissions tab.
    5. Click Grant access. The Add principals dialog box appears.
    6. Under the Add principals, add your Service account principal.
    7. Under the Assign roles, choose custom and choose the role you recently created.
    8. Click Save. A confirmation message is displayed.
    Tip: See Use IAM with buckets for more information on adding permissions to a storage bucket in GCP.

    Create a GCS Integration on the Dataloop platform

    1. Log in to the Dataloop platform.
    2. From the left portal menu, select Integrations.
    3. Click Create New Integration.
    4. Enter a Name for the integration.
    5. Select Google Cloud Storage from the Integration Type list.
    6. Click Import JSON file, and import the GCS Private Key JSON file that you downloaded.
    7. Click Create. A confirmation message is displayed.

    Create a GCS Storage Driver on the Dataloop Platform

    For more information, see the Create a GCS Storage Driver on the Dataloop Platform topic.

    What's Next