Container Registry

Overview

Container registry is a repository for storing and accessing container images, supporting container-based application development, often as part of DevOps workflows.

Dataloop integrates with Google Container Registry (GCR), a GCP service for storing private container images. While GCR offers robust features, Artifact Registry is the recommended GCP service for managing container images and other artifacts. Learn more

To integrate Google Container Registry (GCR) with the Dataloop platform, follow these steps:

1. Create and Configure a GCP Google Container Registry
2. Configure GCR for Use with Dataloop Platform

Create and Configure a GCP Google Container Registry

Step 1: Set Up a GCP Project

If you don’t have an existing project:

1. Go to the Project Selector.
2. Click New Project.
3. Provide a Project Name and select the Billing Account.
4. Click Create.
5. Set the project as active:

gcloud config set project [PROJECT_ID]

Step 2: Enable the Google Container Registry API

1. Go to the GCP Console: https://console.cloud.google.com/.
2. Navigate to APIs & Services > Library.
3. Search for Google Container Registry API.
4. Click Enable.
5. Enable billing for your GCP project by following the necessary steps.

Step 3: Configure Docker Authentication

1. Install the Google Cloud CLI if you haven't already.
2. Download gcloud CLI.
3. Authenticate and configure Docker:

gcloud auth login
gcloud auth configure-docker

Step 4: Create and Push a Container Image

1. Tag your Docker image:

docker tag [IMAGE_NAME] gcr.io/[PROJECT_ID]/[IMAGE_NAME]:[TAG]

2. Push the image to GCR:

docker push gcr.io/[PROJECT_ID]/[IMAGE_NAME]:[TAG]

3. Verify the image:
   1. Navigate to Container Registry in the GCP Console: https://console.cloud.google.com/gcr.
   2. Confirm that the image is visible in the registry.

Configure GCR for Use with Dataloop Platform

Step 1: Provide Access to Dataloop

1. In GCP, go to IAM & Admin > IAM.
2. Add the Dataloop service account (provided in Dataloop documentation or platform settings) as a member:
   1. Assign roles:
      1. Container Registry Viewer (to view and pull images).
      2. Storage Object Viewer (to access image layers).
3. If the Dataloop service account details are unclear: Check Dataloop's documentation or contact their support team for the correct service account information.

Step 2: Generate a Service Account Key

1. Create a service account:
   1. Navigate to IAM & Admin > Service Accounts.
   2. Click Create Service Account.
   3. Provide a Name (e.g., dataloop-access) and click Create.
   4. Assign the following roles:
      1. Storage Object Viewer.
      2. Container Registry Viewer.
   5. Click Done.
2. Generate a key for the service account:
   1. Find your service account in the list.
   2. Click the three-dot menu and select Manage Keys.
   3. Click Add Key > Create New Key.
   4. Select JSON and click Create
   5. Save the JSON key file securely.

Step 3: Upload the Service Account Key to Dataloop

1. Log in to your Dataloop Platform.
2. Navigate to the Data Governance page.
3. Click Create Integration.
4. Name: Provide a name for the integration.
5. Provider: Select GCP from the list.
6. Integration Type: Select Private Container Registry from the list.
7. Registry Name: Select Google Container Registry (GCP) from the list.
8. Password: Provide the JSON key (base64).

{
    "type": "service_account",
    "project_id": "",
    "private_key_id": "",
    "private_key": "",
    "client_email": "",
    "client_id": "",
    "auth_uri": "",
    "token_uri": "",
    "auth_provider_x509_cert_url": "",
    "client_x509_cert_url": ""
}

Step 4: Test the Configuration

• In the Dataloop platform, try pulling or referencing a Docker image hosted in your GCR.
• Ensure the integration is working without any errors.