- 17 Feb 2025
- Print
- DarkLight
- PDF
Private Key Integration
- Updated On 17 Feb 2025
- Print
- DarkLight
- PDF
To connect your data from GCS bucket as a Dataloop dataset, follow the setup instructions.
1. Create an IAM role
Log in to the Google Cloud Console.
From the left portal menu, select the IAM & admin > Roles.
Click +Create Role.
Enter a role Title.
(Optional) Enter a Description.
(Optional) Enter a role ID. By default, the role ID is generated.
Click +Add Permissions, then search for and add the following permissions:
storage.objects.create
storage.objects.delete
storage.objects.get
storage.objects.list
storage.buckets.get
Note: The
storage.objects.delete
permission allows the Dataloop platform to delete dataset items. For more information, see the Downstream section.Click Create.
- See Create and manage custom roles for more information on creating an IAM role in GCP.
- To display Roles information, you must select a project in the Google Cloud Console. If not available, create a project.
2. Create a service account
- Log in to the Google Cloud Console.
- From the left portal menu, select the IAM & admin > Service accounts.
- Click +Create service account.
- (Optional) Enter a Name and Description for the account.
- Click Create and continue.
- Click Done to create the service account.
3. Create a private key for the service account
- Log in to the Google Cloud Console.
- From the left portal menu, select the IAM & admin > Service accounts.
- Find the desired service account for which you want to add a private key, and click on its name to open its details.Note: The Service Account you recently created.
- Select the Keys tab.
- Click Add key.
- Click Create new key from the list. A popup window is displayed.
- Ensure the Key type is in JSON format.
- Click Create. The "Private key saved to your computer" message is displayed.
- Use the key to authenticate as the service account on the Dataloop platform.
- Save the JSON file in a convenient location, it is required at the integration phase.
4. Create a cloud storage bucket
- Log in to Google Cloud Console.
- From the left portal menu, go to the Cloud Storage > Buckets.
- Click Create bucket.
- Enter a Name for the storage bucket.
- Click Continue.
- In the Choose where to store your data, select a location for the bucket.
- Click Continue.
- Click Create bucket.
5. Grant permissions to the cloud storage bucket
- Log in to Google Cloud Console.
- From the left portal menu, go to the Cloud Storage > Buckets.
- Select the Storage bucket for which you want to add permissions, and then click on it.
- Select the Permissions tab.
- Click Grant access. The Add principals dialog box appears.
- Under the Add principals, add your Service account principal.
- Under the Assign roles, choose custom and choose the role you recently created.
- Click Save. A confirmation message is displayed.
6. Create a GCP private key integration
- Log in to the Dataloop platform.
- From the left-side panel, select Data Governance.
- Click Create Integration -> Create Storage Integration. A pop-up window is displayed on the right-side.
- Integration Name: Enter a Name for the integration.
- Provider: Choose GCP from the list.
- Integration Type: Select the Private Key from the list.
- Click Import JSON file, and import the GCS Private Key JSON file that you downloaded.
- Click Create Integration. A confirmation message is displayed.
7. Create a GCS Storage Driver
For more information, see the Create a GCS Storage Driver on the Dataloop Platform topic.