Azure.Storage.DataMovement
12.0.0
Prefix Reserved
dotnet add package Azure.Storage.DataMovement --version 12.0.0
NuGet\Install-Package Azure.Storage.DataMovement -Version 12.0.0
<PackageReference Include="Azure.Storage.DataMovement" Version="12.0.0" />
paket add Azure.Storage.DataMovement --version 12.0.0
#r "nuget: Azure.Storage.DataMovement, 12.0.0"
// Install Azure.Storage.DataMovement as a Cake Addin #addin nuget:?package=Azure.Storage.DataMovement&version=12.0.0 // Install Azure.Storage.DataMovement as a Cake Tool #tool nuget:?package=Azure.Storage.DataMovement&version=12.0.0
Azure Storage Data Movement Common client library for .NET
Azure Storage is a Microsoft-managed service providing cloud storage that is highly available, secure, durable, scalable, and redundant.
The Azure Storage Data Movement library is optimized for uploading, downloading and copying customer data.
Currently this version of the Data Movement library only supports Blobs and File Shares.
Source code | API reference documentation | REST API documentation | Product documentation
Getting started
Install the package
Install the Azure Storage client library for .NET you'd like to use with
NuGet and the Azure.Storage.DataMovement
client library will be included:
dotnet add package Azure.Storage.DataMovement
Prerequisites
You need an Azure subscription and a Storage Account to use this package.
To create a new Storage Account, you can use the Azure Portal, Azure PowerShell, or the Azure CLI. Here's an example using the Azure CLI:
az storage account create --name MyStorageAccount --resource-group MyResourceGroup --location westus --sku Standard_LRS
Authenticate the client
Authentication is specific to the targeted storage service. Please see documentation for the individual services:
Permissions
Data Movement must have appropriate permissions to the storage resources. Permissions are specific to the type of storage Data Movement is connected to.
Key concepts
The Azure Storage DataMovement client library contains shared infrastructure like TokenCredential, TransferManager and RequestFailedException.
Thread safety
We guarantee that all client instance methods are thread-safe and independent of each other (guideline). This ensures that the recommendation of reusing client instances is always safe, even across threads.
Additional concepts
Client options | Accessing the response | Long-running operations | Handling failures | Diagnostics | Mocking | Client lifetime
Examples
This section demonstrates usage of Data Movement regardless of extension package. Package-specific information and usage samples can be found in that package's documentation. These examples will use local disk and Azure Blob Storage when specific resources are needed for demonstration purposes, but the topics here will apply to other packages.
Setup the TransferManager
Singleton usage of TransferManager
is recommended. Providing TransferManagerOptions
is optional.
TransferManager transferManager = new TransferManager(new TransferManagerOptions());
Starting New Transfers
Transfers are defined by a source and destination StorageResource
. There are two kinds of StorageResource
: StorageResourceSingle
and StorageResourceContainer
. Source and destination of a given transfer must be of the same kind.
StorageResource
instances are obtained from StorageResourceProvider
instances. See Initializing Local File StorageResource(s) for more information on the resource provider for local files and directories. See the Next Steps for our DataMovement extension packages for more info on their respective StorageResourceProvider
types.
The sample below demonstrates StorageResourceProvider
use to start transfers by uploading a file to Azure Blob Storage, using the Azure.Storage.DataMovement.Blobs package. It uses an Azure.Core TokenCredential
generated from Azure.Identity DefaultAzureCredential()
with permission to write to the blob.
TokenCredential defaultTokenCredential = new DefaultAzureCredential();
BlobsStorageResourceProvider blobs = new BlobsStorageResourceProvider(defaultTokenCredential);
// Create simple transfer single blob upload job
TransferOperation transferOperation = await transferManager.StartTransferAsync(
sourceResource: LocalFilesStorageResourceProvider.FromFile(sourceLocalPath),
destinationResource: await blobs.FromBlobAsync(destinationBlobUri));
await transferOperation.WaitForCompletionAsync();
The sample below demonstrates StorageResourceProvider
use to start transfers by uploading a file to Azure Share File Storage, using the Azure.Storage.DataMovement.Files.Shares package. It uses an Azure.Core TokenCredential
generated from Azure.Identity DefaultAzureCredential()
with permission to write to the blob.
TokenCredential tokenCredential = new DefaultAzureCredential();
ShareFilesStorageResourceProvider shares = new(tokenCredential);
TransferManager transferManager = new TransferManager(new TransferManagerOptions());
TransferOperation fileTransfer = await transferManager.StartTransferAsync(
sourceResource: LocalFilesStorageResourceProvider.FromFile(sourceLocalFile),
destinationResource: await shares.FromFileAsync(destinationFileUri));
await fileTransfer.WaitForCompletionAsync();
See more examples of starting a transfer respective to each storage service:
Initializing Local File or Directory StorageResource
Local filesystem resources are provided by LocalFilesStorageResourceProvider
. This provider requires no setup to produce storage resources.
LocalFilesStorageResourceProvider files = new();
StorageResource fileResource = files.FromFile("C:/path/to/file.txt");
StorageResource directoryResource = files.FromDirectory("C:/path/to/dir");
Troubleshooting
See Handling Failed Transfers and Enabling Logging to assist with any troubleshooting.
See Known Issues for detailed information.
Next steps
Get started with our Blob DataMovement samples.
Get started with our Share File DataMovement samples.
For advanced scenarios regarding the TransferManager
see TransferManager Samples.
For advanced scenarios regarding the TransferManager.StartTransfer()
operation, see Start Transfer Samples.
For advanced scenarios and information regarding Resume, Pause and/or checkpointing see Pause and Resume, Checkpointing Samples.
Contributing
See the Storage CONTRIBUTING.md for details on building, testing, and contributing to these libraries.
This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit cla.microsoft.com.
This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.
Product | Versions Compatible and additional computed target framework versions. |
---|---|
.NET | net5.0 was computed. net5.0-windows was computed. net6.0 is compatible. net6.0-android was computed. net6.0-ios was computed. net6.0-maccatalyst was computed. net6.0-macos was computed. net6.0-tvos was computed. net6.0-windows was computed. net7.0 was computed. net7.0-android was computed. net7.0-ios was computed. net7.0-maccatalyst was computed. net7.0-macos was computed. net7.0-tvos was computed. net7.0-windows was computed. net8.0 is compatible. net8.0-android was computed. net8.0-browser was computed. net8.0-ios was computed. net8.0-maccatalyst was computed. net8.0-macos was computed. net8.0-tvos was computed. net8.0-windows was computed. net9.0 was computed. net9.0-android was computed. net9.0-browser was computed. net9.0-ios was computed. net9.0-maccatalyst was computed. net9.0-macos was computed. net9.0-tvos was computed. net9.0-windows was computed. |
.NET Core | netcoreapp2.0 was computed. netcoreapp2.1 was computed. netcoreapp2.2 was computed. netcoreapp3.0 was computed. netcoreapp3.1 was computed. |
.NET Standard | netstandard2.0 is compatible. netstandard2.1 was computed. |
.NET Framework | net461 was computed. net462 was computed. net463 was computed. net47 was computed. net471 was computed. net472 was computed. net48 was computed. net481 was computed. |
MonoAndroid | monoandroid was computed. |
MonoMac | monomac was computed. |
MonoTouch | monotouch was computed. |
Tizen | tizen40 was computed. tizen60 was computed. |
Xamarin.iOS | xamarinios was computed. |
Xamarin.Mac | xamarinmac was computed. |
Xamarin.TVOS | xamarintvos was computed. |
Xamarin.WatchOS | xamarinwatchos was computed. |
-
.NETStandard 2.0
- Azure.Core (>= 1.44.1)
- Azure.Storage.Common (>= 12.22.0)
- System.Threading.Channels (>= 6.0.0)
-
net6.0
- Azure.Core (>= 1.44.1)
- Azure.Storage.Common (>= 12.22.0)
-
net8.0
- Azure.Core (>= 1.44.1)
- Azure.Storage.Common (>= 12.22.0)
NuGet packages (3)
Showing the top 3 NuGet packages that depend on Azure.Storage.DataMovement:
Package | Downloads |
---|---|
Azure.Storage.DataMovement.Blobs
This client library enables high-performance, multi-threaded uploading, downloading and copying Azure Storage blobs and containers. The package depends on the Azure.Storage.DataMovement package and can be used in conjuction with other Data Movement sub-packages, such as Azure.Storage.DataMovement.Files.Shares to transfer data between services. For this release see notes - https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/storage/Azure.Storage.DataMovement.Blobs/README.md and https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/storage/Azure.Storage.DataMovement.Blobs/CHANGELOG.md |
|
Azure.Storage.DataMovement.Files.Shares
This client library enables high-performance, multi-threaded uploading, downloading and copying Azure File Share files and directories. The package depends on the Azure.Storage.DataMovement package and can be used in conjuction with other Data Movement sub-packages, such as Azure.Storage.DataMovement.Blobs to transfer data between services. For this release see notes - https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/storage/Azure.Storage.DataMovement.Files.Shares/README.md and https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/storage/Azure.Storage.DataMovement.Files.Shares/CHANGELOG.md |
|
HHDev.CanopyApi
Library that provides functionality for users of the Canopy Simulations portal to submit and download studies directly through the HH DM client. |
GitHub repositories
This package is not used by any popular GitHub repositories.
Version | Downloads | Last updated |
---|---|---|
12.0.0 | 62 | 2/11/2025 |
12.0.0-beta.6 | 20,323 | 10/14/2024 |
12.0.0-beta.5 | 4,897 | 7/17/2024 |
12.0.0-beta.4 | 9,618 | 12/5/2023 |
12.0.0-beta.3 | 24,003 | 7/11/2023 |
12.0.0-beta.2 | 3,716 | 4/27/2023 |
12.0.0-beta.1 | 618 | 12/16/2022 |