Getting Started with Cloud Storage

Cloud Storage provides easy-to-use components for accessing various cloud storage services including:

Amazon S3
OneDrive
Google Drive
Azure Blobs
ShareFile
Custom S3 providers

Dropbox
SharePoint
IBM Cloud Storage
Akamai (Linode) Object Storage
Cloudflare R2
Wasabi

Box.com
Backblaze B2
DigitalOcean Spaces
Google Cloud Storage
Hadoop HDFS

In addition, the CloudFiles component is designed to provide a unified interface for basic functionality regardless of the storage provider and may be used for any supported provider.

S3 compatible providers are supported in the S3 component by specifying the appropriate ServiceProvider. For select providers a service specific component is also available that provides more advanced functionality specific to the service.

S3 Compatible Providers

The S3 component provides access to any S3-compatible services in one convenient location. While some services contain unique features that can only be accessed via that service's specific component, all generic functionality can be accessed using this component. Supported providers include:

Amazon S3
Backblaze B2
DigitalOcean Spaces
Google Storage
IBM Cloud Storage

Wasabi

Cloudflare R2

Oracle

Seagate Lyve Cloud

Akamai (Linode)

Custom Providers (MinIO, IDrive, etc.)

To begin, set the ServiceProvider property to the desired service provider and create an account with that service. Consult the service-specific documentation for instructions on completing this process.

Authentication

Authentication is performed by setting the AccessKey and SecretKey properties associated with the service account.

Managing Buckets

The ListBuckets method will return all the buckets for a provided account. When called the following fields will be populated:

BucketName
CreationDate
OwnerId
OwnerName


s3.ListBuckets();
for (int i = 0; i < s3.Buckets.Count; i++) {
  Console.WriteLine(s3.Buckets[i].Name);
  Console.WriteLine(s3.Buckets[i].CreationDate);
  Console.WriteLine(s3.Buckets[i].OwnerDisplayName);
}

The S3 component allows for deletion of buckets by calling the DeleteBucket method and allows for the creation of a new bucket by calling the CreateBucket method.

Other features for managing buckets include:

An UpdateBucketACL method that will update the access policy of the bucket.
A GetBucketLocation method that will return the location value of the bucket.

Managing Objects

The ListObjects method will return all the objects within a given bucket. The bucket specified is set by the Bucket property. ObjectPrefix, ObjectDelimiter, and ObjectMarker can be used to filter or control the objects listed from the ListObjects method.

When called the following fields will be populated:

ETag
LastModified
LatestVersion
Name
OwnerDisplayName
OwnerId
Size
StorageClass


s3.Bucket = "TEST_BUCKET";

// Set a prefix.
s3.ObjectPrefix = "photos/2016/";

s3.ListObjects();
for (int i = 0; i < s3.Objects.Count; i++) {
  Console.WriteLine(s3.Objects[i].Name);
  Console.WriteLine(s3.Objects[i].LastModified);
  Console.WriteLine(s3.Objects[i].Size);
}

The S3 component allows for deletion of objects by calling the DeleteObject method and allows for the creation of a new object by calling the CreateObject method.

In order to retrieve a desired object, call the GetObject method. This will store the object in the file specified by the LocalFile property. A bucket name must also be set using the Bucket property.


s3.Bucket = "TEST_BUCKET";
s3.LocalFile = "C:\\testFile.txt";
s3.GetObject("testObject");

Other features for managing objects include:

An UpdateObjectACL method that updates the access policy of an object.
A GetObjectInfo method that stores the metadata in the ParsedHeaders property.
An AddMetadata method that allows for custom metadata to be associated with an object. Before calling the CreateObject method, use the AddMetadata method to add up to 2K of user meta data.
A GetLink method that creates an authenticated link to allow access to objects.

Managing Uploads

The StartMultipartUpload method begins a multipart upload and returns the UploadId associated with the upload. The upload does not expire and must be completed by calling the CompleteMultipartUpload method or aborted by calling the AbortMultipartUpload method.

The UploadId returned by this method is used to reference the upload when uploading parts via the UploadPart method as well as other methods such as AbortMultipartUpload, CompleteMultipartUpload, and ListParts.


// Start a multipart upload.
s3.Bucket = "TEST_BUCKET";
string uploadId = s3.StartMultipartUpload("test_file.dat");

// List the current multipart uploads.
s3.ListMultipartUploads();
for (int i = 0; i < s3.Objects.Count; i++) {
  Console.WriteLine(s3.Objects[i].Name);
  Console.WriteLine(s3.Objects[i].UploadId);
  Console.WriteLine(s3.Objects[i].Size);
}
s3.CompleteMultipartUpload("test_file.dat", uploadId);

The ListParts method lists all the parts in a current multipart upload. The PartList event will fire once for each part in the current upload as reported by the server. The Parts collection will also be populated. Parts may be inspected to determine various pieces of information, such as ETag, PartNumber, ObjectName, and more.

By default, the number of parts returned per call to ListParts is determined by the service provider, but this can be adjusted via the MaxParts configuration setting. The PartMarker property will be populated if more parts exist, and further calls to ListParts will retrieve more sets of parts until PageMarker is empty.


s3.Bucket = "TEST_BUCKET";
string uploadId = s3.StartMultipartUpload("test_file.dat");

// Upload parts.
s3.UploadPart("test_file.dat", 1, uploadId);
s3.UploadPart("test_file.dat", 2, uploadId);

// List all the parts.
s3.ListParts("test_file.dat", uploadId);
for (int i = 0; i < s3.Parts.Count; i++) {
  Console.WriteLine(s3.Parts[i].ObjectName);
  Console.WriteLine(s3.Parts[i].PartNumber);
  Console.WriteLine(s3.Parts[i].Size);
}
s3.CompleteMultipartUpload("test_file.dat", uploadId);

Additional Functionality

The S3 component offers advanced functionality, including:

The ability to encrypt and decrypt files using the EncryptionAlgorithm and EncryptionPassword properties.
The ability to manage bucket and object ACLs with the UpdateBucketACL and UpdateObjectACL methods.
A CopyObject method to copy objects on the server.
And more!

CloudFiles

The CloudFiles component provides a single interface that can be used to work with a variety of services. Capabilities include uploading and downloading files, strong encryption support, creating folders, and more.

By supporting multiple providers with a single API code may be written once and used to support multiple services. Some of the providers currently supported by this component include:

Amazon S3
Box.com
Dropbox
Google Drive
Microsoft OneDrive
Azure Blob
Wasabi

DigitalOcean Spaces

Google Cloud Storage

Hadoop DFS

Azure File

Backblaze B2

IBM Cloud Object Storage

Akamai (Linode) Object Storage

To begin, first create an account and register your application with the desired provider(s). Consult each provider's service documentation for instructions on this process.

Authentication

Depending on the provider, authentication is either handled by setting the Authorization property or by configuring various fields on the Account property. Please refer to the Cloud Storage component's documentation for more information on how to authenticate for each provider.

Selecting a Provider

To specify the provider simply set ServiceProvider. This tells the component to which service requests will be made.

Listing Files and Folders

ListDirectory lists files and folders in the path specified by RemotePath. A file mask may optionally be supplied in RemoteFile.

The directory entries are provided through the DirList event and also via the DirList property.


cloudfiles.RemotePath = "MyFolder";
cloudfiles.ListDirectory();
for (int i = 0; i < cloudfiles.DirList.Count; i++) {
  Console.WriteLine(cloudfiles.DirList[i].FileName);
  Console.WriteLine(cloudfiles.DirList[i].FileSize);
  Console.WriteLine(cloudfiles.DirList[i].FileTime);
  Console.WriteLine(cloudfiles.DirList[i].IsDir);
}

Optionally set RemoteFile to a file mask to list only specific files. For instance:


cloudfiles.RemoteFile = "*.txt";
cloudfiles.ListDirectory();

Downloading Files

The Download method downloads a specific file.

Set RemoteFile to the name the file to download before calling this method. If RemoteFile only specifies a filename it will be downloaded from the path specified by RemotePath. RemoteFile may also be set to an absolute path.

The file will be downloaded to the stream specified (if any) by SetDownloadStream. If a stream is not specified and LocalFile is set the file will be saved to the specified location. If a stream is not specified and LocalFile is not set the file data will be held by ResourceData.

To decrypt an encrypted file set EncryptionAlgorithm and EncryptionPassword before calling this method.


cloudfiles.RemotePath = "My Folder";
cloudfiles.RemoteFile = "MyFile.zip";
cloudfiles.LocalFile = "../MyFile.zip";
cloudfiles.Download();

Resuming Downloads

The component also supports resuming failed downloads by using the StartByte property. If the download was interrupted, set StartByte to the appropriate offset before calling this method to resume the download.


cloudfiles.RemotePath = myRemoteFolder;
cloudfiles.RemoteFile = myRemoteFile;
cloudfiles.LocalFile = downloadFile;
cloudfiles.Download();

// The transfer is interrupted and Download() above fails. Later, resume the download:

// Get the size of the partially downloaded file.
cloudfiles.StartByte = new FileInfo(downloadFile).Length; 
cloudfiles.RemotePath = myRemoteFolder;
cloudfiles.RemoteFile = myRemoteFile;
cloudfiles.LocalFile = downloadFile;
cloudfiles.Download();

Resuming Encrypted File Downloads

Resuming encrypted file downloads is only supported when LocalFile was set in the initial download attempt. When beginning an encrypted download if LocalFile is set the component will create a temporary file in TempPath to hold the encrypted data until it is complete.

If the download is interrupted DownloadTempFile will be populated with the temporary file holding the partial data. When resuming, DownloadTempFile must be populated along with StartByte to allow the remainder of the encrypted data to be downloaded. Once the encrypted data is downloaded it will be decrypted and written to LocalFile.


cloudfiles.RemotePath = myRemoteFolder;
cloudfiles.RemoteFile = myRemoteFile;
cloudfiles.LocalFile = downloadFile;
cloudfiles.EncryptionPassword = "password";
cloudfiles.Download();

// The transfer is interrupted and Download() above fails. Later, resume the download:

// Get the size of the partially downloaded temp file.
cloudfiles.StartByte = new FileInfo(cloudfiles.Config("DownloadTempFile")).Length; 
cloudfiles.RemotePath = myRemoteFolder;
cloudfiles.RemoteFile = myRemoteFile;
cloudfiles.LocalFile = downloadFile;
cloudfiles.EncryptionPassword = "password";
cloudfiles.Download();

Uploading Files

The Upload method is used to upload files. If SetUploadStream is used to set an upload stream the data to upload is taken from the stream instead.

RemoteFile should be set to either a relative or absolute path. If RemoteFile is not an absolute path it will be uploaded relative to RemotePath.

To encrypt a file before uploading set EncryptionAlgorithm and EncryptionPassword.

Note: Resuming uploads is not currently supported.


// Upload with a relative path.
cloudfiles.LocalFile = "C:\localfile.txt"
cloudfiles.RemoteFile = "remotefile.txt"
cloudfiles.Upload()

// Upload with an absolute path.
cloudfiles.LocalFile = "C:\localfile2.txt"
cloudfiles.RemoteFile = "/folder/remotefile2.txt"
cloudfiles.Upload()

Additional Functionality

The CloudFiles component offers advanced functionality beyond simple uploads and downloads. For instance:

Encrypt and decrypt files using EncryptionAlgorithm and EncryptionPassword.
DeleteFile provides a way to delete files.
MakeDirectory and RemoveDirectory support creating and deleting folders.
RenameFile allows renaming of existing files on the server.
And more!

Azure Blob

The AzureBlob component provides an easy to use interface to Microsoft's Azure Blob Service, which allows you to store text and binary data. The Blob service offers the following three resources: the storage account, containers, and blobs. Within your storage account, containers provide a way to organize sets of blobs, of which there are three kinds: block blobs, page blobs, and append blobs.

To begin, first sign up for the Azure Blob Service. Consult the Microsoft Azure documentation for instructions on this process.

Authentication

Authentication is performed using the Account and AccessKey provided by Microsoft Azure.


blob = new Azureblob();
blob.Account = BLOB_ACCOUNT;
blob.AccessKey = BLOB_ACCESS_KEY;

Managing Containers

The ListContainers() method will return all the containers for the provided account. The Prefix property can be used to filter the containers listed by this method.

If there are more than MaxResults results, the ContainerMarker property will be populated with a marker identifying the position in the results. Subsequent ListContainers() calls will return the next portion of results. If ContainerMarker is an empty string, the end of the list has been reached.


do {
  blob.ListContainers();
} while (!string.IsNullOrEmpty(blob.ContainerMarker));
for (int i = 0; i < blob.Containers.Count; i++) {
  Console.WriteLine(blob.Containers[i].Name);
  Console.WriteLine(blob.Containers[i].ModifiedTime);
}

To select a container, set the Container property to its name. If the selected container doesn't yet exist, the CreateContainer() method can be called to create it. The selected container can also be deleted using the DeleteContainer() method.

Other methods for managing containers include:

GetContainerACL() and SetContainerACL() for working with container access policies.
GetContainerInfo() and UpdateMetadata() for retrieving container information and metadata, and setting container metadata.
Lease() to work with container leases.

Managing Blobs

The ListBlobs() method will list the blobs within the container currently selected by the Container property. The Prefix and BlobDelimiter properties can be used to filter the blobs listed by this method; refer to their documentation for more information about how to use them in tandem.

If there are more than MaxResults results, the BlobMarker will be populated with a marker identifying the position in the results. Subsequent ListBlobs() calls will return the next portion of results. If BlobMarker is an empty string, the end of the list has been reached.


blob.ContainerName = "testcontainer";
do {
  blob.ListBlobs();
} while (!string.IsNullOrEmpty(blob.BlobMarker));
for (int i = 0; i < blob.Blobs.Count; i++) {
  Console.WriteLine(blob.Blobs[i].Name);
  Console.WriteLine(blob.Blobs[i].ModifiedTime);
  Console.WriteLine(blob.Blobs[i].ContentLength);
}

Blobs can be created and deleted in the selected container using the CreateBlob() and DeleteBlob() methods. To download an existing blob's content, call the GetBlob() method. This will store the blob in the file specified by LocalFile, unless it is set to an empty string, in which case it will be stored in the BlobData property. Alternatively, in certain editions, the data can be downloaded to a given stream.

When calling the CreateBlob() method, you can specify whether you want to create a block blob, a page blob, or an append blob. When creating a block blob, the component will upload any data that has been supplied immediately (this is not allowed for page or append blobs). When creating a page blob, you must also specify the blob's initial capacity. Refer to the CreateBlob() method's documentation for more information.

Other general methods for managing blobs include:

CopyBlob() and AbortCopy() to copy a source blob to a destination blob, or abort an ongoing copy operation.
GetBlobInfo(), UpdateBlobInfo(), and UpdateMetadata() for retrieving and setting blob information and metadata.
Lease() to work with blob leases.

Managing Block Blobs

Block blobs track data as a series of blocks, each of which may be up to 100MB in size. The ListBlocks() method can be used to list the blocks that have been uploaded as part of a specific block blob. There are two types of block lists maintained for a blob, committed and uncommitted. When calling ListBlocks(), you can specify whether the server should return the committed block list, the uncommitted block list, or both.

The committed block list contains a list of blocks that have been successfully committed to the blob using the PutBlockList() method. The uncommitted block list contains a list of blocks that have been uploaded using the PutBlock() method, but not yet committed. When GetBlob() is called on a block blob, only the committed blocks' data is returned.


blob.ListBlocks("TestBlockBlob", 2); // 2 means "return both lists".
for (int i = 0; i < blob.Blocks.Count; i++) {
  Console.WriteLine(blob.Blocks[i].Type);
  Console.WriteLine(blob.Blocks[i].Id);
  Console.WriteLine(blob.Blocks[i].Size);
}

To commit a new block list, add items to the Blocks collection property in the order you wish for them to appear in the blob, and then call PutBlockList() to commit the block list. For each block in the new block list, you can specify whether the server should search for a block with the given Id in the current committed block list, the uncommitted block list, or both (uncommitted, then committed). Refer to the PutBlockList() method's documentation for more information.

Managing Page Blobs

Page blobs track data in 512-byte chunks called pages. The ListPageRanges() method can be used to list the page ranges that contain data. To upload data to a page blob, use the PutPages() method; the amount of data supplied must be a multiple of 512 bytes no larger than 4MB. To clear a range of pages in a page blob, call the ClearPages() method. Finally, to resize a page blob and/or change its sequence number, use the UpdatePageBlob() method.


blob.ListPageRanges("TestPageBlob");
for (int i = 0; i < blob.PageRanges.Count; i++) {
  Console.WriteLine("Page Range " + i + ": " + blob.PageRanges[i].First + "-" + blob.PageRanges[i].Last);
}

Managing Append Blobs

Append blobs can be thought of as a very specific kind of block blob. The only way to write data to an append blob is to use the AppendBlock() method to append it; up to 4MB of data can be appended at a time. Unlike block blobs, the appended block is immediately committed to the end of the append blob, and you cannot list the appended blocks using ListBlocks().

Additional Functionality

The AzureBlob component also offers additional functionality, such as:

The CreateSnapshot() method, and other related API members, to create and interact with blob snapshots.
The GetLink() method, which allows you to create a shared access signature (SAS) that can be used to access a container, blob, or blob snapshot.
And more!

Box.com

The Box component provides a simple interface to working with Box.com. Capabilities include uploading and downloading files, strong encryption support, creating folders, moving and copying resources, and more.

Authentication

This component supports authentication via OAuth 2.0. First, perform OAuth authentication using the OAuth component or a separate process. Once complete you should have an authorization string which looks like: Bearer ya29.AHES6ZSZEJzATdZYjeihDn5W-VrXSsxEZu5p0pclxGdKKQ

Assign this value to the Authorization property before attempting any operations. Consult the documentation for the service for more information about supported scope values and more details on OAuth authentication.

Listing Resources

ListResources() lists resources within the specified folder. Calling this method will fire the ResourceList event once for each resource, and will also populate the Resources collection.

If there are still more resources available to list when this method returns, the ResourceMarker property will be populated. Continue to call this method until ResourceMarker is empty to accumulate all pages of results in the Resources collection.


// ResourceList event handler.
box.OnResourceList += (s, e) => {
  Console.WriteLine(e.Name);
};

do {
  box.ListResources("d:123456");

  for (int i = 0; i < box.Resources.Count; i++) {
    // Process resources here.
  }
} while (!string.IsNullOrEmpty(box.ResourceMarker));

Downloading Files

The DownloadFile() method downloads file resources.

If a stream has been specified using SetDownloadStream(), the file data will be sent through it. If a stream is not specified, and LocalFile is set, the file will be saved to the specified location; otherwise, the file data will be held by ResourceData.

To download and decrypt an encrypted file, set EncryptionAlgorithm and EncryptionPassword before calling this method.

In the simplest use-case, downloading a file looks like this:


box.LocalFile = "../MyFile.zip";
box.DownloadFile(box.Resources[0].Id);

Resuming Downloads
The component also supports resuming failed downloads by using the StartByte property. If a download is interrupted, set StartByte to the appropriate offset before calling this method to resume the download.


string downloadFile = "../MyFile.zip";
box.LocalFile = downloadFile;
box.DownloadFile(box.Resources[0].Id);

// The transfer is interrupted and DownloadFile() above fails. Later, resume the download:

// Get the size of the partially downloaded file.
box.StartByte = new FileInfo(downloadFile).Length;
box.DownloadFile(box.Resources[0].Id);

Resuming Encrypted File Downloads
Resuming encrypted file downloads is only supported when LocalFile was set in the initial download attempt.

If LocalFile is set when beginning an encrypted download, the component creates a temporary file in TempPath to hold the encrypted data until the download is complete. If the download is interrupted, DownloadTempFile will be populated with the path of the temporary file that holds the partial data.

To resume, DownloadTempFile must be populated, along with StartByte, to allow the remainder of the encrypted data to be downloaded. Once the encrypted data is downloaded it will be decrypted and written to LocalFile.


box.LocalFile = "../MyFile.zip";
box.EncryptionPassword = "password";
box.DownloadFile(box.Resources[0].Id);

// The transfer is interrupted and DownloadFile() above fails. Later, resume the download:

// Get the size of the partially downloaded temp file.
box.StartByte = new FileInfo(box.Config("DownloadTempFile")).Length;
box.DownloadFile(box.Resources[0].Id);

Uploading Files

The UploadFile() method uploads new file resources.

If SetUploadStream() has been used to set an upload stream, it will take priority as the file data source. If LocalFile is set the file will be uploaded from the specified path. If LocalFile is not set the data in ResourceData will be used.

To encrypt the file before uploading it, set EncryptionAlgorithm and EncryptionPassword.

Box offers two ways to upload a file. For smaller files a simple upload option is provided to upload data in one request. This is the default option. For larger files (must be larger than 20 MB), uploads can be fragmented into multiple pieces, allowing resuming of uploads that may be interrupted.

Simple
By default the component uses the simple upload mechanism. When doing a simple upload, the HashSimpleUploads setting is applicable.


box.LocalFile = "../MyFile.zip";
box.UploadFile("MyFile.zip", "");

Resumable
To enable resumable uploads set UseResumableUpload to True. This is recommended for large files (must be larger than 20 MB). The component will automatically fragment the specified file into smaller pieces and upload each individually.

When UseResumableUpload is set to True and UploadFile() is called, a resumable upload session is started by the component. ResumeURL is populated with a URL identifying the session (this value may be needed for additional operations if the upload does not complete normally).

During a resumable upload, the FragmentComplete event fires after each fragment is uploaded to indicate overall progress. The component also updates StartByte as necessary to indicate the current offset in the file.

If the upload is interrupted for any reason, resuming it is easy. First, verify that ResumeURL is populated (if the same instance of the component is used, it should already be populated, and no special action should be needed). If uploading from a stream, be sure to reset its position to where it was the first time the upload was started (typically the beginning). Call PollUploadStatus() to populate the correct values for StartByte and UploadFragmentSize. Then call UploadFile() again to resume the upload at the specified StartByte offset.

Note that if the upload is not resumed after some time the upload session will expire. PollUploadStatus() may be used to check the status of a resumable upload, including when it will expire (which is stored in the UploadExpDate configuration setting). An interrupted upload can be aborted explicitly using the AbortUpload() method.


box.LocalFile = "../MyFile.zip";
box.UploadFile("MyFile.zip", "");

// The transfer is interrupted and UploadFile() above fails. Later, resume the download:
// Using the same instance, StartByte and ResumeURL are already populated from the previous
// upload attempt.
box.UploadFile("MyFile.zip", "");


MemoryStream uploadStream = new MemoryStream(File.ReadAllBytes("../MyFile.zip"));
box.SetUploadStream(uploadStream);
box.UploadFile("MyFile.zip", "");

// The transfer is interrupted and UploadFile() above fails. Later, resume the download:
// Using the same instance, StartByte and ResumeURL are already populated from the previous 
// upload attempt.
// You MUST reset the stream's position to where it was when you first started the upload!
uploadStream.Position = 0;
box.UploadFile("MyFile.zip", "");

Additional Functionality

The Box component offers advanced functionality beyond simple uploads and downloads. For instance:

Encrypt and decrypt files using the EncryptionAlgorithm and EncryptionPassword properties.
Basic file and folder manipulation and organization using methods such as CopyResource(), CreateFolder(), DeleteResource(), MoveResource(), and RestoreResource().
Support for resource sharing using CreateLink() and RevokeLink().
Resource metadata management with CreateMetadata(), ListMetadata(), UpdateMetadata(), and DeleteMetadata().
Advanced resource listing using the Search() method.
Retrieval of account and space usage details using GetAccountInfo().
File version handling with ListVersions() and PromoteVersion().
And more!

Dropbox

The Dropbox component provides a simple interface to working with Dropbox. Capabilities include uploading and downloading files, strong encryption support, creating folders, moving and copying resources, and more.

Authentication

Addressing Resources

Dropbox typically allows resources to be addressed in multiple ways:

Using a path (e.g. /path/to/resource.txt).
Using a resource Id (e.g. id:xxxxx).
Using an Id-based relative path (e.g. id:xxxxx/relative/path/test.txt, where the Id is that of a folder resource).
For certain methods, using a revision Id (e.g. rev:xxxxx).

The documentation for this component's methods will always note which of the above options are acceptable for each applicable method parameter.

Listing Resources

ListResources() lists resources within the specified folder. Calling this method will fire the ResourceList event once for each resource, and will also populate the Resources collection.


// ResourceList event handler.
dropbox.OnResourceList += (s, e) => {
  Console.WriteLine(e.Name);
};

do {
  dropbox.ListResources("/work_files/serious_business/cats");

  for (int i = 0; i < dropbox.Resources.Count; i++) {
    // Process resources here.
  }
} while (!string.IsNullOrEmpty(dropbox.ResourceMarker));

Downloading Files

The DownloadFile() method downloads file resources.

To download and decrypt an encrypted file, set EncryptionAlgorithm and EncryptionPassword before calling this method.

In the simplest use-case, downloading a file looks like this:


dropbox.LocalFile = "../MyFile.zip";
dropbox.DownloadFile(dropbox.Resources[0].Id);


string downloadFile = "../MyFile.zip";
dropbox.LocalFile = downloadFile;
dropbox.DownloadFile(dropbox.Resources[0].Id);

// The transfer is interrupted and DownloadFile() above fails. Later, resume the download:

// Get the size of the partially downloaded file.
dropbox.StartByte = new FileInfo(downloadFile).Length;
dropbox.DownloadFile(dropbox.Resources[0].Id);

Resuming Encrypted File Downloads
Resuming encrypted file downloads is only supported when LocalFile was set in the initial download attempt.


dropbox.LocalFile = "../MyFile.zip";
dropbox.EncryptionPassword = "password";
dropbox.DownloadFile(dropbox.Resources[0].Id);

// The transfer is interrupted and DownloadFile() above fails. Later, resume the download:

// Get the size of the partially downloaded temp file.
dropbox.StartByte = new FileInfo(dropbox.Config("DownloadTempFile")).Length;
dropbox.DownloadFile(dropbox.Resources[0].Id);

Uploading Files

The UploadFile() method uploads new file resources.

To encrypt the file before uploading it, set EncryptionAlgorithm and EncryptionPassword.

Dropbox offers two ways to upload a file. For smaller files a simple upload option is provided to upload data in one request. This is the default option. For larger files, uploads can be fragmented into multiple pieces, allowing resuming of uploads that may be interrupted.

Simple
By default the component uses the simple upload mechanism.


dropbox.LocalFile = "../MyFile.zip";
dropbox.UploadFile("/MyFile.zip");

Resumable
To enable resumable uploads set UseResumableUpload to True. This is recommended for large files. The component will automatically fragment the specified file into smaller pieces and upload each individually. FragmentSize may be set to specify the size of the fragment if desired. The default fragment size is 10 MB.

When UseResumableUpload is set to True and UploadFile() is called, a resumable upload session is started by the component. UploadSessionId is populated with a resumable upload session Id identifying the session (this value may be needed for additional operations if the upload does not complete normally).

If the upload is interrupted for any reason, resuming it is easy. First, verify that UploadSessionId and StartByte are populated (if the same instance of the component is used, they should already be populated, and no special action should be needed). If uploading from a stream, be sure to reset its position to where it was the first time the upload was started (typically the beginning). Then call UploadFile() again to resume the upload at the specified StartByte offset.

Note that if the upload is not resumed after some time the upload session will expire.


dropbox.LocalFile = "../MyFile.zip";
dropbox.UploadFile("MyFile.zip");

// The transfer is interrupted and UploadFile() above fails. Later, resume the download:
// Using the same instance, StartByte and ResumeURL are already populated from the previous
// upload attempt.
dropbox.UploadFile("MyFile.zip");


MemoryStream uploadStream = new MemoryStream(File.ReadAllBytes("../MyFile.zip"));
dropbox.SetUploadStream(uploadStream);
dropbox.UploadFile("MyFile.zip");

// The transfer is interrupted and UploadFile() above fails. Later, resume the download:
// Using the same instance, StartByte and ResumeURL are already populated from the previous 
// upload attempt.
// You MUST reset the stream's position to where it was when you first started the upload!
uploadStream.Position = 0;
dropbox.UploadFile("MyFile.zip");

Additional Functionality

The Dropbox component offers advanced functionality beyond simple uploads and downloads. For instance:

Encrypt and decrypt files using the EncryptionAlgorithm and EncryptionPassword properties.
Basic file and folder manipulation and organization using methods such as CopyResource(), CreateFolder(), DeleteResource(), and MoveResource().
Support for resource sharing using CreateLink(), ListSharedLinks(), and RevokeLink().
Change tracking with ListChanges() and WaitForChanges().
Advanced resource listing using the Search() method.
Retrieval of account and space usage details using GetAccountInfo().
File revision handling with ListRevisions() and RestoreResource().
And more!

Google Drive

The GoogleDrive component provides an easy-to-use interface for Google Drive. Capabilities include uploading and downloading files, file and folder manipulation and organization, Google Shared Drive support, strong client-side file encryption functionality, and more.

Authentication

Listing Resources

The ListResources() method is used to list resources within the scope specified by the ListResourcesScope property. The ListChildren(), ListParents(), and GetResourceInfo() methods are also available for finer-grained control.


// ResourceList event handler.
googledrive.OnResourceList += (s, e) => {
  Console.WriteLine(e.Name);
};

// List all of the current user's resources.
googledrive.ListResourcesScope = GoogledriveListResourcesScopes.lrsUser;
do {
  googledrive.ListResources();

  for (int i = 0; i < googledrive.Resources.Count; i++) {
    // Process resources here.
  }
} while (!string.IsNullOrEmpty(googledrive.ResourceMarker));

// List all of the resources in the specified shared drive.
googledrive.SharedDrive = "qwerty1234567890";
googledrive.ListResourcesScope = GoogledriveListResourcesScopes.lrsSharedDrive;
do {
  googledrive.ListResources();

  for (int i = 0; i < googledrive.Resources.Count; i++) {
    // Process resources here.
  }
} while (!string.IsNullOrEmpty(googledrive.ResourceMarker));

Downloading Files

The DownloadFile() method is used to download files.

Downloading an Encrypted File
To decrypt an encrypted file set EncryptionAlgorithm and EncryptionPassword before calling this method.


googledrive.LocalFile = "../MyFile.zip";
googledrive.DownloadFile(googledrive.Resources[0].Id, "");


string downloadFile = "../MyFile.zip";
googledrive.LocalFile = downloadFile;
googledrive.DownloadFile(googledrive.Resources[0].Id, "");

// The transfer is interrupted and DownloadFile() above fails. Later, resume the download:

// Get the size of the partially downloaded file.
googledrive.StartByte = new FileInfo(downloadFile).Length;
googledrive.DownloadFile(googledrive.Resources[0].Id, "");

Resuming Encrypted File Downloads
Resuming encrypted file downloads is only supported when LocalFile was set in the initial download attempt.


googledrive.LocalFile = "../MyFile.zip";
googledrive.EncryptionPassword = "password";
googledrive.DownloadFile(googledrive.Resource[0].Id, "");

// The transfer is interrupted and DownloadFile() above fails. Later, resume the download:

// Get the size of the partially downloaded temp file.
googledrive.StartByte = new FileInfo(googledrive.Config("DownloadTempFile")).Length;
googledrive.DownloadFile(googledrive.Resource[0].Id, "");

Uploading Files

The UploadFile() method is used to upload files.

Google Drive offers two ways to upload a file. For smaller files a simple upload option is provided to upload data in one request. This is the default option. For larger files, uploads can be fragmented into multiple pieces, allowing resuming of uploads that may be interrupted.

Simple
By default the component uses the simple upload mechanism.


googledrive.LocalFile = "../MyFile.zip";
googledrive.UploadFile("MyFile.zip", "");

If the upload is interrupted for any reason, resuming it is easy. First, verify that ResumeURL and StartByte are populated (if the same instance of the component is used, they should already be populated, and no special action should be needed). If uploading from a stream, be sure to reset its position to where it was the first time the upload was started (typically the beginning). Then call UploadFile() again to resume the upload at the specified StartByte offset.

Note that if the upload is not resumed after some time the upload session will expire. GetUploadStatus() may be used to check the status of a resumable upload.


googledrive.LocalFile = "../MyFile.zip";
googledrive.UploadFile("MyFile.zip", "");

// The transfer is interrupted and UploadFile() above fails. Later, resume the download:
// Using the same instance, StartByte and ResumeURL are already populated from the previous
// upload attempt.
googledrive.UploadFile("MyFile.zip", "");


MemoryStream uploadStream = new MemoryStream(File.ReadAllBytes("../MyFile.zip"));
googledrive.SetUploadStream(uploadStream);
googledrive.UploadFile("MyFile.zip", "");

// The transfer is interrupted and UploadFile() above fails. Later, resume the download:
// Using the same instance, StartByte and ResumeURL are already populated from the previous 
// upload attempt.
// You MUST reset the stream's position to where it was when you first started the upload!
uploadStream.Position = 0;
googledrive.UploadFile("MyFile.zip", "");

Shared Drive Support

The GoogleDrive component has full support for Google Shared Drives. For the most common use-cases (such as those described above), there is very little difference when using a shared drive versus a personal Google Drive ("My Drive").

For more information about how to use the component with Google Shared Drives, refer to both the shared-drive-specific documentation sections for commonly-used methods like ListResources(), MoveResource(), UpdatePermissions(), etc.; and browse through the documentation for shared-drive-specific methods and properties such as the AddSharedDriveMember() and ListSharedDrives() methods, the SharedDrive property, etc.

Additional Functionality

The GoogleDrive component offers advanced functionality beyond simple uploads and downloads. For instance:

Encrypt and decrypt files using the EncryptionAlgorithm and EncryptionPassword properties.
Basic file and folder manipulation and organization using methods such as CopyResource(), CreateFolder(), DeleteResource(), MoveResource(), and UpdateResource().
Enumeration and manipulation of parent-child relationships using the AddParents(), ListChildren(), ListParents(), and RemoveParents() methods.
Resourcing trashing and deletion: DeleteResource(), TrashResource(), RestoreResource().
Control over permissions using ListPermissions() and UpdatePermissions().
Change tracking with ListChanges().
And more!

HadoopDFS

The HadoopDFS component offers an easy-to-use API compatible with any Hadoop distributed file system (HDFS) cluster that exposes Hadoop's standard WebHDFS REST API. Capabilities include uploading and downloading files, strong encryption support, creating folders, file manipulation and organization, and more.

Authentication

First, set the URL property to the base WebHDFS URL of the server (see the URL property's documentation for more details).

Depending on how the server is configured, there are a few different authentication methods that might be used; or, the server might not require authentication at all). Refer to the AuthScheme property's documentation for more information about configuring the component to authenticate correctly.

Addressing Resources

HDFS addresses resources (files, directories, and symlinks) using Linux-style absolute paths. Unless otherwise specified, the component always works in terms of absolute paths, and will always prepend a forward slash (/) to any path passed to it that does not already start with one.

Listing Directory Contents

ListResources() lists resources (files, directories, and symlinks) within the specified directory. Calling this method will fire the ResourceList event once for each resource, and will also populate the Resources collection.


// ResourceList event handler.
hdfs.OnResourceList += (s, e) => {
  Console.WriteLine(e.Name);
};

hdfs.ListResources("/work_files/serious_business/cats");

for (int i = 0; i < hdfs.Resources.Count; i++) {
  // Process resources here.
}

Downloading Files

The DownloadFile() method downloads files.

To download and decrypt an encrypted file, set EncryptionAlgorithm and EncryptionPassword before calling this method.

In the simplest use-case, downloading a file looks like this:


hdfs.LocalFile = "../MyFile.zip";
hdfs.DownloadFile(hdfs.Resources[0].Path);


string downloadFile = "../MyFile.zip";
hdfs.LocalFile = downloadFile;
hdfs.DownloadFile(hdfs.Resources[0].Path);

// The transfer is interrupted and DownloadFile() above fails. Later, resume the download:

// Get the size of the partially downloaded file.
hdfs.StartByte = new FileInfo(downloadFile).Length;
hdfs.DownloadFile(hdfs.Resources[0].Path);

Resuming Encrypted File Downloads
Resuming encrypted file downloads is only supported when LocalFile was set in the initial download attempt.


hdfs.LocalFile = "../MyFile.zip";
hdfs.EncryptionPassword = "password";
hdfs.DownloadFile(hdfs.Resources[0].Path);

// The transfer is interrupted and DownloadFile() above fails. Later, resume the download:

// Get the size of the partially downloaded temp file.
hdfs.StartByte = new FileInfo(hdfs.Config("DownloadTempFile")).Length;
hdfs.DownloadFile(hdfs.Resources[0].Path);

Uploading Files

The UploadFile() method uploads new files.

To encrypt the file before uploading it, set EncryptionAlgorithm and EncryptionPassword.


hdfs.LocalFile = "../MyFile.zip";
hdfs.UploadFile("/MyFile.zip");

Additional Functionality

The HadoopDFS component offers advanced functionality beyond simple uploads and downloads. For instance:

Encrypt and decrypt files using the EncryptionAlgorithm and EncryptionPassword properties.
Basic file and folder manipulation and organization using methods such as AppendFile(), DeleteResource(), MakeDirectory(), MoveResource(), and TruncateFile().
Advanced file and directory manipulation with SetFileReplication(), SetOwner(), SetPermission(), and SetTimes().
Retrieval of both general file/directory information, as well as directory quota information, using GetResourceInfo() and GetDirSummary().
Execute any arbitrary WebHDFS operation with ease using the DoCustomOp() method.
And more!

OneDrive

The OneDrive component provides a simple interface to working with Microsoft OneDrive. Capabilities include uploading and downloading files, strong encryption support, creating folders, moving and copying resources, OneDrive for Business and SharePoint Online support, and more.

Authentication

Note: There are a couple of extra factors to consider when doing OAuth for OneDrive; please refer to the Authorization property documentation for more information.

Listing Resources

ListResources() lists resources within the folder resource currently selected by RemoteId or RemotePath. Calling this method will fire the ResourceList event once for each resource, and will also populate the Resources collection.


// ResourceList event handler.
onedrive.OnResourceList += (s, e) => {
  Console.WriteLine(e.Name);
};

// (Assume that the RemoteId property isn't set here; it takes precedence if it is.)
onedrive.RemotePath = "/work_files/serious_business/cats";
do {
  onedrive.ListResources();

  for (int i = 0; i < onedrive.Resources.Count; i++) {
    // Process resources here.
  }
} while (!string.IsNullOrEmpty(onedrive.ResourceMarker));

Downloading Files

The DownloadFile() method downloads the file resource currently selected by RemoteId or RemotePath.

To download and decrypt an encrypted file, set EncryptionAlgorithm and EncryptionPassword before calling this method.

In the simplest use-case, downloading a file looks like this:


onedrive.LocalFile = "../MyFile.zip";
onedrive.RemoteId = onedrive.Resources[0].Id;
onedrive.DownloadFile();


string downloadFile = "../MyFile.zip";
onedrive.LocalFile = downloadFile;
onedrive.RemoteId = onedrive.Resources[0].Id;
onedrive.DownloadFile();

// The transfer is interrupted and DownloadFile() above fails. Later, resume the download:

// Get the size of the partially downloaded file.
onedrive.StartByte = new FileInfo(downloadFile).Length;
onedrive.DownloadFile();

Resuming Encrypted File Downloads
Resuming encrypted file downloads is only supported when LocalFile was set in the initial download attempt.


onedrive.LocalFile = "../MyFile.zip";
onedrive.EncryptionPassword = "password";
onedrive.RemoteId = onedrive.Resources[0].Id;
onedrive.DownloadFile();

// The transfer is interrupted and DownloadFile() above fails. Later, resume the download:

// Get the size of the partially downloaded temp file.
onedrive.StartByte = new FileInfo(onedrive.Config("DownloadTempFile")).Length;
onedrive.DownloadFile();

Uploading Files

The UploadFile() method uploads new file resources to the folder resource currently selected by RemoteId or RemotePath.

To encrypt the file before uploading it, set EncryptionAlgorithm and EncryptionPassword.

OneDrive offers two ways to upload a file. For smaller files a simple upload option is provided to upload data in one request. This is the default option. For larger files, uploads can be fragmented into multiple pieces, allowing resuming of uploads that may be interrupted.

Simple
By default the component uses the simple upload mechanism.


onedrive.LocalFile = "../MyFile.zip";
onedrive.UploadFile("MyFile.zip");


onedrive.LocalFile = "../MyFile.zip";
onedrive.UploadFile("MyFile.zip");

// The transfer is interrupted and UploadFile() above fails. Later, resume the download:
// Using the same instance, StartByte and ResumeURL are already populated from the previous
// upload attempt.
onedrive.UploadFile("MyFile.zip");


MemoryStream uploadStream = new MemoryStream(File.ReadAllBytes("../MyFile.zip"));
onedrive.SetUploadStream(uploadStream);
onedrive.UploadFile("MyFile.zip");

// The transfer is interrupted and UploadFile() above fails. Later, resume the download:
// Using the same instance, StartByte and ResumeURL are already populated from the previous 
// upload attempt.
// You MUST reset the stream's position to where it was when you first started the upload!
uploadStream.Position = 0;
onedrive.UploadFile("MyFile.zip");

Additional Functionality

The OneDrive component offers advanced functionality beyond simple uploads and downloads. For instance:

Encrypt and decrypt files using the EncryptionAlgorithm and EncryptionPassword properties.
Basic file and folder manipulation and organization using methods such as CopyResource(), CreateFolder(), DeleteResource(), MoveResource(), and UpdateResource().
Creation of resource sharing links using CreateLink().
Change tracking with ListChanges().
Advanced resource listing using the Search() method.
Support for OneDrive for Business and SharePoint Online functionality, including drive selection using ListDrives(), Drive, and other API members.
And more!

ShareFile

The ShareFile component provides a simple interface for working with ShareFile. Capabilities include uploading and downloading files, strong encryption support, creating folders, moving and copying items, creating request and send links, and more.

Authentication


    Bearer ya29.AHES6ZSZEJzATdZYjeihDn5W-VrXSsxEZu5p0pclxGdKKQ

Referencing and Creating Items, Links, Permissions, and Users

ShareFile typically uses specified ids to reference its objects. When an object is created, through methods such as CreateClient() or CreateFolder() or uploaded with the UploadFile() method; the component will return with the objects ShareFile Id in the form of a string. In certain cases, a path (/parentFolder/ChildFolder) can be used to reference an item.

When a link is created using methods like CreateAndEmailLink() , CreateAndEmailRequestLink(), CreateLink(), or CreateRequestLink() then their URL is returned rather than their ShareFile id. To get the newly created id, the component also clears and populates the Links collection with the new link. When creating a link, you can use the CreateLinkOptions property to set certain options for the link.

When a permission is created using the CreatePermission() method, the component will not return anything as ShareFile uses a combination of a UserId and the ItemId of a folder to reference permissions. When creating a permission, you can use the CreatePermissionOptions property to set certain options for the permission.

Listing and Getting Items, Links, Permissions, and Users

When listing out the links and users currently available to the authenticated user you will use the corresponding ListLinks() and ListUsers() methods. Both these methods take no parameters and will populates their corresponding collection. For links the collection is the Links collection and for users it is the Users collection.

When listing the Items within a Folder or Permissions, the corresponding methods will take a folders ItemId. The ListItems() will list the items in the folder to the Items collection. The method will not recursively list out items found in child folders. The ListPermissions method will list all the permissions for the specified folder to the Permissions collection.

The component can also list out the items associated with a specified link. The ListLinkItems() method will take a LinkId and populate Items collection.

The component also offers the ability to get the specific information about a certain item, link, permission, or user. GetItemInfo(), GetLinkInfo() and GetUserInfo() each take a corresponding id. For permissions, GetPermissionInfo() will take a FolderId and UserId rather than a specific id. Once called, they each clear and populate the corresponding collection.

For all list* and get* methods, there is a corresponding event that will fire for each item in the list. The ItemList event will fire when the ListItems(), ListLinkItems(), or GetItemInfo() methods are called; the LinkList event will fire when the ListLinks() or GetLinkInfo() methods are called; the PermissionList event will fire when the ListPermissions() or GetPermissionInfo() methods are called; and the UserList event will fire when the ListUsers() or GetUserInfo() methods are called.

Downloading Files

The DownloadFile method downloads file or folder items. If a stream has been specified using SetDownloadStream(), the file data will be sent through it. If a stream is not specified, and LocalFile is set, the file will be saved to the specified location; otherwise, the file data will be held by ItemData. To download and decrypt an encrypted file, set EncryptionAlgorithm and EncryptionPassword before calling this method.

Simple Download
A simple download is consistent with setting the LocalFile to the destination of the file when it is downloaded and then calling the method with the item's id. For example:


    shareFile.LocalFile = "../MyFile.zip";
    shareFile.DownloadFile(shareFile.Items[0].Id);

Streaming Download
To download to a stream, first call the SetDownloadStream() method and then call the DownloadFile() method. For example:


    shareFile.LocalFile = "";
    using (FileStream fs = new FileStream("../../temp/temp_data1.zip", FileMode.OpenOrCreate)) {
    shareFile.SetDownloadStream(fs);
    shareFile.DownloadFile(id);
    }

Uploading Files
The UploadFile() method uploads new file items. If SetUploadStream() has been used to set an upload stream, it will take priority as the file data source. If LocalFile is set the file will be uploaded from the specified path. If LocalFile is not set the data in ItemData will be used. To encrypt the file before uploading it, set EncryptionAlgorithm and EncryptionPassword.

ShareFile offers two ways to upload a file. For smaller files a simple upload option is provided to upload data in one request. This is the default option. For larger files, uploads can be fragmented into multiple pieces, allowing resuming of uploads that may be interrupted.

Simple
By default the component uses the simple upload mechanism.


    ShareFile.LocalFile = "../MyFile.zip";
    ShareFile.UploadFile("/MyFile.zip");

Resumable
To enable resumable uploads set UseResumableUpload to true. This is recommended for large files. The component will automatically fragment the specified file into smaller pieces and upload each individually.

When UseResumableUpload is set to true and UploadFile() is called, a resumable upload session is started by the component. Once called and the component fragments the file, the ResumeURL is populated. This URL needs to be set so that the component can resume the upload if the upload is interrupted.

Note that if the upload is not resumed after some time the upload session will expire.


    shareFile.UseResumableUpload = true;
    shareFile.LocalFile = "../MyFile.zip";
    shareFile.UploadFile("MyFile.zip");
     
    // The transfer is interrupted and UploadFile() above fails. Later, resume the download.
    // Using the same instance, StartByte and ResumeURL are already populated from the previous
    // upload attempt.
    shareFile.UploadFile("MyFile.zip");


    MemoryStream uploadStream = new MemoryStream(File.ReadAllBytes("../MyFile.zip"));
    shareFile.UseResumableUpload = true;
    shareFile.SetUploadStream(uploadStream);
    shareFile.UploadFile("MyFile.zip");
     
    // The transfer is interrupted and UploadFile() above fails. Later, resume the download.
    // Using the same instance, StartByte and ResumeURL are already populated from the previous
    // upload attempt.
    // You MUST reset the stream's position to where it was when you first started the upload!
    uploadStream.Position = 0;
    shareFile.UploadFile("MyFile.zip");

Additional Functionality

The ShareFile component offers advanced functionality beyond simple uploads and downloads. For instance:

Encrypt and decrypt files using the EncryptionAlgorithm and EncryptionPassword properties.
Basic file and folder manipulation and organization using methods such as CopyItem and MoveItem.
And more!

We appreciate your feedback. If you have any questions, comments, or suggestions about this article please contact our support team at kb@nsoftware.com.