Documentation Index

Fetch the complete documentation index at: https://docs.dataloop.ai/llms.txt

Use this file to discover all available pages before exploring further.

NFS

Prev Next

What is NFS?

NFS is a network-based file sharing protocol that allows multiple machines to access files stored on a central server as if those files were located on their own local disk.

Think of it as a shared network drive.

Example

Suppose you have a storage server:

Server IP: 10.172.164.34
Shared Folder: /exports/images

Multiple Kubernetes nodes can mount this folder:

/mnt/images

and access the same files simultaneously.


How NFS Works in DDOE

When DDOE is configured with an NFS (Network File System) Storage Driver, the dataset does not store copies of the files inside DDOE. Instead, it accesses the files directly from the NFS server through the compute cluster.

NFS Server → Compute Cluster → Storage Driver → Dataset

  1. An NFS server stores the files: The NFS server is the actual location where your data resides.

  2. The Compute Cluster mounts the NFS share: The DDOE Compute Cluster connects to the NFS server and mounts the exported directory.

  3. The Storage Driver exposes the mounted path to DDOE: The Storage Driver acts as the connection layer between DDOE and the mounted storage.

  4. Datasets access data directly from the NFS storage: A dataset is then created using the storage driver.


Example Configuration

{
  "name": "nfs-storage",
  "type": "nfs",
  "config": {
    "server": "10.172.164.34",
    "path": "/exports/images",
    "mountPath": "/mnt/images"
  }
}

Advantages and Limitations

Advantages

Limitations

  • Shared Storage: All pods and nodes can see the same files.

  • Centralized Management: Data is stored in one location.

  • Easy Data Access: No need to copy data into DDOE.

  • Suitable for Large Files

Commonly used for:

  • Images

  • Videos

  • Documents

  • Medical datasets

  • Network Dependent: If the NFS server is unavailable, data becomes unavailable.

  • Performance Limitations: Can become slower when many users access the same storage simultaneously.


Create an On-Premises NFS Integration

Prerequisites

  • Roles and Permissions: Only users with the Organization Admin or Owner roles are authorized to create integrations. Learn more

  • NFS Server Address: The IP address or the hostname of your NFS server.

  • Export Path: The directory path on the NFS server that is exported or shared.

  1. Log in to the DDOE platform.

  2. From the left-side panel, select Integrations.

  3. Click Create Integration -> Create Storage Integration. A pop-up window is displayed on the right-side.

  4. Integration Name: Provide a name for your NFS Integration.

  5. Provider: Select On-Prem from the list.

  6. Integration Type: Select NFS from the list.

  7. NFS Server Address: Enter the IP address or the hostname of your NFS server.

  8. Export Path: Enter the directory path on the NFS server that is exported or shared.

  9. Click Create Integration. The NFS type integration will be created and listed in the Integration page.