> ## Documentation Index > Fetch the complete documentation index at: https://mintlify.com/treeverse/dvc/llms.txt > Use this file to discover all available pages before exploring further. # Remote Storage Configuration > Complete guide to configuring remote storage backends in DVC DVC supports various remote storage backends for storing and sharing data. This guide covers all available storage options and their configuration parameters. ## Adding a Remote Add a remote storage location: ```bash theme={null} dvc remote add -d myremote s3://mybucket/path ``` The `-d` flag sets it as the default remote. Configuration is stored in `.dvc/config`: ```ini theme={null} [core] remote = myremote ['remote "myremote"'] url = s3://mybucket/path ``` Use `dvc remote add` to configure remotes, or edit `.dvc/config` directly. ## Common Configuration These options apply to all remote types: Remote storage URL. Format depends on the storage type: * S3: `s3://bucket/path` * GCS: `gs://bucket/path` * Azure: `azure://container/path` * SSH: `ssh://user@host:/path` * Local: `/path/to/storage` or `file:///path/to/storage` Number of parallel jobs for upload/download operations. Higher values speed up transfers but use more resources. Number of parallel jobs for checksum calculation. Enable version-aware operations for supported cloud storage. Useful for S3, GCS, and Azure with versioning enabled. Enable worktree mode for the remote. ## Local & File Remotes Local filesystem or network-mounted storage. ### Configuration ```bash theme={null} # Local directory dvc remote add storage /mnt/shared/dvc-storage # Network share (Linux/Mac) dvc remote add storage /mnt/nas/dvc-storage # Windows network share dvc remote add storage \\server\share\dvc-storage # Relative path (relative to .dvc directory) dvc remote add storage ../../shared-storage ``` ### Parameters Cache link type: `reflink`, `hardlink`, `symlink`, or `copy`. * `reflink`: Copy-on-write (fastest, limited support) * `hardlink`: Hard links (fast, same filesystem required) * `symlink`: Symbolic links * `copy`: Full copy (slowest, most compatible) Can specify multiple as comma-separated list (tried in order). Set to `group` to make cache group-writable. Useful for shared storage accessed by multiple users. Warn when using slow link types (copy). Verify checksums after transfer. ### Example ```ini theme={null} ['remote "local"'] url = /mnt/shared/dvc-cache type = hardlink,copy shared = group ``` ## Amazon S3 Amazon S3 and S3-compatible storage (MinIO, DigitalOcean Spaces, etc.). ### Basic Setup ```bash theme={null} dvc remote add s3remote s3://mybucket/path dvc remote modify s3remote region us-east-1 ``` ### Authentication Parameters AWS access key ID. Alternatively, set `AWS_ACCESS_KEY_ID` environment variable. AWS secret access key. Alternatively, set `AWS_SECRET_ACCESS_KEY` environment variable. Store credentials in `.dvc/config.local` (git-ignored) or use environment variables. AWS session token for temporary credentials. AWS profile name from `~/.aws/credentials`. Path to custom AWS credentials file. Path to custom AWS config file. Allow anonymous access to public buckets. ### Region & Endpoint AWS region (e.g., `us-west-2`, `eu-central-1`). Custom S3 endpoint URL for S3-compatible services: * MinIO: `http://localhost:9000` * DigitalOcean: `https://nyc3.digitaloceanspaces.com` ### Connection Settings Use HTTPS for connections. Verify SSL certificates. Set to `false` to disable or path to CA bundle. Read timeout in seconds. Connection timeout in seconds. ### Encryption Server-side encryption algorithm: `AES256` or `aws:kms`. KMS key ID for KMS encryption. Customer-provided encryption algorithm. Customer-provided encryption key. ### Access Control Canned ACL: `private`, `public-read`, `public-read-write`, `authenticated-read`, etc. Grant read permissions (grantee format: `id=xxx` or `emailAddress=xxx`). Grant read ACP permissions. Grant write ACP permissions. Grant full control permissions. ### Performance Cache region information for faster bucket access. ### Complete Example ```bash theme={null} # Add remote dvc remote add s3storage s3://my-dvc-bucket/project # Configure region and encryption dvc remote modify s3storage region us-west-2 dvc remote modify s3storage sse AES256 dvc remote modify s3storage acl bucket-owner-full-control # Set credentials (local config, git-ignored) dvc config --local remote.s3storage.access_key_id AKIAIOSFODNN7EXAMPLE dvc config --local remote.s3storage.secret_access_key wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY ``` Resulting `.dvc/config`: ```ini theme={null} ['remote "s3storage"'] url = s3://my-dvc-bucket/project region = us-west-2 sse = AES256 acl = bucket-owner-full-control ``` ```bash theme={null} dvc remote add minio s3://my-bucket/dvc dvc remote modify minio endpointurl http://localhost:9000 dvc remote modify minio access_key_id minioadmin dvc remote modify minio secret_access_key minioadmin ``` ## Google Cloud Storage (GCS) Google Cloud Storage backend. ### Basic Setup ```bash theme={null} dvc remote add gsremote gs://mybucket/path ``` ### Authentication Path to service account JSON key file. Alternatively, set `GOOGLE_APPLICATION_CREDENTIALS` environment variable. Google Cloud project name. Allow anonymous access to public buckets. ### Endpoint Custom GCS endpoint URL (for emulators or compatible services). ### Example ```bash theme={null} dvc remote add gcs gs://my-dvc-bucket/data dvc config --local remote.gcs.credentialpath /path/to/service-account.json dvc remote modify gcs projectname my-gcp-project ``` ## Microsoft Azure Blob Storage Azure Blob Storage backend. ### Basic Setup ```bash theme={null} dvc remote add azure azure://mycontainer/path ``` ### Authentication Methods Azure storage connection string. Format: `DefaultEndpointsProtocol=https;AccountName=...;AccountKey=...` Storage account name. Storage account key. Shared Access Signature token. Azure AD tenant ID for service principal authentication. Azure AD client ID. Azure AD client secret. Allow anonymous access. ### Credential Chain Control Exclude environment variables from credential chain. Exclude VS Code credentials. Exclude shared token cache. Exclude managed identity credentials. ### Connection Settings General timeout in seconds. Read timeout in seconds. Connection timeout in seconds. ### Example ```bash theme={null} dvc remote add azure azure://mycontainer dvc config --local remote.azure.connection_string "DefaultEndpointsProtocol=https;AccountName=...;AccountKey=..." ``` ```bash theme={null} dvc remote add azure azure://mycontainer/dvc dvc remote modify azure account_name mystorageaccount dvc config --local remote.azure.tenant_id YOUR_TENANT_ID dvc config --local remote.azure.client_id YOUR_CLIENT_ID dvc config --local remote.azure.client_secret YOUR_CLIENT_SECRET ``` ## SSH / SFTP Remote storage over SSH/SFTP. ### Basic Setup ```bash theme={null} dvc remote add sshremote ssh://user@example.com:/path/to/storage ``` ### Authentication SSH username. SSH password (not recommended, use key-based auth). Prompt for password interactively. Path to SSH private key file. Passphrase for encrypted private key. Prompt for passphrase interactively. Use GSS-API authentication. Allow SSH agent for authentication. ### Connection Settings SSH port number. Connection timeout in seconds. Maximum number of concurrent SSH sessions. ### Cache Settings Cache link type on remote: `reflink`, `hardlink`, `symlink`, or `copy`. ### Example ```bash theme={null} dvc remote add ssh ssh://user@server.com:/data/dvc-storage dvc remote modify ssh port 2222 dvc remote modify ssh keyfile ~/.ssh/id_rsa_dvc ``` ## HDFS Hadoop Distributed File System. ### Basic Setup ```bash theme={null} dvc remote add hdfsremote hdfs://namenode:8020/path/to/storage ``` ### Parameters HDFS username. Path to Kerberos ticket cache file. HDFS replication factor. ## WebHDFS Web-based HDFS access. ### Basic Setup ```bash theme={null} dvc remote add webhdfs webhdfs://namenode:50070/path/to/storage ``` ### Authentication WebHDFS username. WebHDFS password. Use Kerberos authentication. Kerberos principal name. Delegation token. ### Settings Use HTTPS instead of HTTP. Verify SSL certificates. Proxy to specific DataNode. Target for data proxy operations. ## Alibaba OSS Alibaba Cloud Object Storage Service. ### Basic Setup ```bash theme={null} dvc remote add oss oss://mybucket/path ``` ### Authentication OSS access key ID. OSS access key secret. OSS endpoint URL (e.g., `oss-cn-hangzhou.aliyuncs.com`). ### Example ```bash theme={null} dvc remote add oss oss://my-bucket/dvc-storage dvc remote modify oss oss_endpoint oss-cn-beijing.aliyuncs.com dvc config --local remote.oss.oss_key_id YOUR_KEY_ID dvc config --local remote.oss.oss_key_secret YOUR_KEY_SECRET ``` ## Google Drive Google Drive backend (experimental). ### Basic Setup ```bash theme={null} dvc remote add gdrive gdrive://folder-id ``` ### Authentication Google OAuth client ID. Google OAuth client secret. Path to user credentials file. ### Service Account Use service account for authentication. Path to service account JSON file. Email for service account impersonation. ### Settings Move files to trash instead of permanent deletion. Acknowledge abuse risk when downloading flagged files. ## HTTP / HTTPS Read-only remote over HTTP(S). ### Basic Setup ```bash theme={null} dvc remote add httpremote https://example.com/data ``` ### Authentication Authentication method: `basic`, `digest`, or `custom`. Username for basic/digest auth. Password for authentication. Prompt for password interactively. Custom authentication header value. Example: `Bearer YOUR_TOKEN` ### Connection Settings Verify SSL certificates. Can be `true`, `false`, or path to CA bundle. HTTP method to use (default: GET). Connection timeout in seconds. Read timeout in seconds. ## WebDAV / WebDAVs WebDAV protocol support. ### Basic Setup ```bash theme={null} dvc remote add webdav webdavs://example.com/dvc-storage ``` ### Authentication Username. Password. Prompt for password. Authentication token. Command to generate bearer token dynamically. Custom authentication header. ### SSL/TLS Path to client certificate. Path to client private key. Verify SSL certificates. Connection timeout in seconds. ## Best Practices Use `.dvc/config.local` for sensitive data: ```bash theme={null} # Add remote (tracked by Git) dvc remote add storage s3://bucket/path # Add credentials (git-ignored) dvc config --local remote.storage.access_key_id YOUR_KEY dvc config --local remote.storage.secret_access_key YOUR_SECRET ``` Or use environment variables: ```bash theme={null} export AWS_ACCESS_KEY_ID=YOUR_KEY export AWS_SECRET_ACCESS_KEY=YOUR_SECRET ``` Configure different remotes for different purposes: ```bash theme={null} # Production storage dvc remote add -d production s3://prod-bucket/data # Backup storage dvc remote add backup gs://backup-bucket/data # Local cache for fast access dvc remote add local /mnt/fast-storage # Push to specific remote dvc push -r backup ``` When running on cloud instances, use IAM roles instead of access keys: **AWS EC2:** ```bash theme={null} # No credentials needed - uses instance profile dvc remote add storage s3://bucket/path ``` **Azure VM:** ```bash theme={null} # Uses managed identity dvc remote add storage azure://container/path dvc remote modify storage account_name myaccount ``` Adjust job counts based on your network and CPU: ```bash theme={null} # Increase for fast networks dvc remote modify storage jobs 16 dvc remote modify storage checksum_jobs 16 # Reduce for slow connections or limited CPU dvc remote modify storage jobs 2 dvc remote modify storage checksum_jobs 2 ``` For important datasets, enable post-transfer verification: ```bash theme={null} dvc remote modify storage verify true ``` Verification doubles the time needed for transfers but ensures data integrity. ## Remote Management Commands ```bash theme={null} # List remotes dvc remote list # Add remote dvc remote add myremote s3://bucket/path # Set as default dvc remote default myremote # Modify remote dvc remote modify myremote region us-west-2 # Rename remote dvc remote rename myremote newname # Remove remote dvc remote remove myremote # Push to specific remote dvc push -r myremote # Pull from specific remote dvc pull -r myremote ``` ## Troubleshooting **Check credentials:** ```bash theme={null} # Verify config dvc remote list dvc config remote.storage.url # Test connection dvc remote list storage ``` **Common issues:** * Expired credentials * Wrong region/endpoint * Missing permissions * Credentials in wrong config level **Optimize settings:** ```bash theme={null} # Increase parallelism dvc remote modify storage jobs 16 # Check network dvc pull -v # verbose output ``` **Considerations:** * Network bandwidth * Remote storage throughput limits * Number of files vs. file sizes **Disable verification (not recommended for production):** ```bash theme={null} dvc remote modify storage ssl_verify false ``` **Or provide CA bundle:** ```bash theme={null} dvc remote modify storage ssl_verify /path/to/ca-bundle.crt ``` ## Next Steps Learn about DVC configuration system Understand DVC file formats