What's New in IPWorks Cloud 2020

Requirements: IPWorks Cloud 2020

IPWorks Cloud 2020 is the latest version of /n software's popular cloud storage product, and represents a major upgrade from Cloud Storage Integrator 2016. The IPWorks Cloud 2020 toolkit introduces support for additional cloud storage providers, adds new functionality to existing providers, and offers refreshed APIs across the board.

This article is intended to provide both a high level overview of new features and functionality, as well as details about how to migrate existing code from the Cloud Storage Integrator 2016 components to the IPWorks Cloud 2020 components. Please refer to the IPWorks Cloud 2020 documentation for more information, and don't forget to check out the Getting Started with IPWorks Cloud article as well.

Contents

• New Components
• Updated Components
  • AmazonS3, DigitalOcean, and Wasabi Components
  • AzureBlob Component
  • Box Component
  • Dropbox Component
  • GoogleDrive Component
  • OneDrive Component
• Other Updates

New Components

IPWorks Cloud 2020 now provides support for even more cloud storage services with the addition of the following components:

• HadoopDFS: Any Hadoop Distributed File System (HDFS) cluster, via its WebHDFS endpoint
• GoogleStorage: Google Cloud Storage
• AzureFile: Microsoft Azure File Storage

Support for these services has also been added to the existing CloudStorage component.

Updated Components

Many of the cloud storage services already supported have continued to evolve over time, adding both quality-of-life improvements and new major features since the initial release of Cloud Storage Integrator 2016. In IPWorks Cloud 2020, we've updated a number of components, augmenting them with additional functionality based on the current offerings of each cloud storage service. Along the way, we meticulously redesigned these components' APIs to provide a uniform and intuitive developer experience.

For each updated component, you'll find a section below that provides an overview of the additions and changes that have been made. Each section also includes a detailed API migration table to assist you in upgrading from Cloud Storage Integrator 2016 to IPWorks Cloud 2020. For more information, please refer to the IPWorks Cloud 2020 documentation, which provides in-depth details.

AmazonS3, DigitalOcean, and Wasabi Components

Overview of Changes

• Support for versioning has been added. New API members include:
  • The VersioningEnabled property, which can be queried to determine if the bucket selected by the Bucket property has versioning enabled, or set to enable or disable versioning for that bucket.
  • The ListVersions() method and VersionMarker property, for listing object versions in a bucket.
  • New fields on the S3Object (and for Wasabi, WasabiObject) types: VersionId, LatestVersion, Deleted. These are also now available as parameters in the ObjectList event.
  • The VersionId property, to specify the object version that the component should make requests against when calling the following methods: CopyObject(), DeleteObject(), GetLink(), GetObject(), GetObjectInfo(), SendCustomRequest(), and UpdateObjectACL().
• The CreateObject() method will now automatically perform a multi-part upload when attempting to create an object above a certain size. The following API members have been added to support this feature:
  • StartByte property: Tracks how much data has been successfully uploaded during an automatic multi-part upload, allowing it to be resumed if it's interrupted.
  • ResumableUploadState configuration setting: Tracks the state of the automatic multi-part upload, allowing it to be resumed if it's interrupted.
  • SimpleUploadLimit configuration setting: Specifies the maximum data size the component should upload directly; if more than this amount of data is present when CreateObject() is called, a multi-part upload will be performed.
  • FragmentSize configuration setting: Specifies the part size the component should use when an automatic multi-part upload is performed.
• Object metadata management has been improved:
  • A Metadata collection property has been added.
  • The GetObjectInfo() method now repopulates the Objects and Metadata collections with the information and metadata retrieved.
  • The AddUserMetaData() method has been renamed to AddMetadata().
• It is no longer necessary to call the ListParts() method before calling the CompleteMultipartUpload() method. The latter will now automatically collect the information required to complete the multipart upload internally.
• An interrupted download can now be resumed by setting the new StartByte property before calling the GetBlob() method.
• The SendCustomRequest() method is now more flexible in its usage.
• AmazonS3 Only: The UseVirtualHosting property is now True by default. This change was made in response to Amazon's announcement that buckets created after September 30, 2020 will only support virtual-hosted-style requests.
• A handful of unused configuration settings have been removed. Notably, the Region and LocationConstraint configuration settings have been removed (their functionality duplicated the still-available Region property).

API Migration Table

AzureBlob Component

Overview of Changes

The AzureBlob component's API has been greatly expanded in order to align it with the current Azure Blob Storage REST API. For brevity, this overview does not include the names of specific component API members; please refer to the component's documentation for more information.

• The component now targets version 2019-02-02 of the Blob Storage REST API (the old component targeted version 2013-08-15). In addition to the other items in this list, the following notable changes occurred as a result:
  • Full support has been added for page blobs and append blobs, and support for block blobs has been expanded and refined.
  • The component can now take advantage of the higher data size limits supported by newer versions of the Blob Storage REST API (e.g., a maximum block size of 100MB instead of 4MB).
  • Container leasing support has been added, and blob leasing support has been updated.
  • Soft-delete functionality is now supported, including listing of soft-deleted blobs and blob snapshots, and the ability to undelete soft-deleted items.
  • The component now supports Azure AD OAuth authentication, which may be used in place of access-key-based authentication.
• Numerous container and blob information manipulation improvements have been made:
  • Additional pieces of container and blob information (i.e. "properties") are now returned.
  • Blob information can now be updated more easily.
  • Metadata management, for both containers and blobs, has been revised.
• Additional support for delegating data access via shared access signatures (SAS), including:
  • The ability to manage a container's public access level and stored access policies.
  • Creation of SASs for containers, blobs, and blob snapshots.
  • Greater control over a SAS's permissions, validity period, and other parameters.
• Copying functionality has been updated to support:
  • Monitoring the progress of an asynchronous copy operation.
  • Aborting an asynchronous copy operation.
  • Copying from other Blob Storage accounts, or even from Azure File Shares, using the CopySourceURL configuration setting.

API Migration Table

Box Component

Overview of Changes

• Collections are now used in the .NET/Java editions (these are the Resource* and Metadata* properties in other editions).
• Methods now accept resource Ids directly as parameters. The ResourceId and ResourceParentId properties no longer exist in .NET/Java, and in other editions they are no longer used to determine what resource (or parent resource) a method should operate on.
• The component now exposes resource Ids prefixed with either f: (for files) or d: (for folders), and expects those prefixes to be present when resource Ids are passed in so that it knows ahead-of-time whether the associated resource is a file or a folder.
  • The ResourceType property no longer exists in .NET/Java, and in other editions it is no longer used to control whether to operate against a file or folder resource when calling methods.
• The component now supports Box's resumable uploads API in addition to the simple upload API. The following API members have been added to support this functionality:
  • UseResumableUpload and ResumeURL properties.
  • AbortUpload() and PollUploadStatus() methods.
  • FragmentComplete event.
  • UploadFragmentSize and UploadExpDate configuration settings.
• Numerous improvements to metadata manipulation have been made, including:
  • The component can now manipulate metadata items in any metadata container on any resource (the previous version was limited to manipulating metadata in the /global/properties container, and could only do so on file resources).
  • Additional metadata modification operations are now possible.
  • A MetadataList event has been added.
  • The Metadata collection property was added (and other API areas have been similarly updated).
• Shared resource handling has been updated to align with Box design concepts. Shared link information is now exposed directly on each resource item.
  • The SharedResourceInfo object property (SharedResource* properties in non-.NET/Java editions) has been removed.
• File resource "revisions" are now known as "versions", to match the Box terminology.
  • Additionally, there is now a VersionMarker property, used with ListVersions().
• The MoveResource() method has been added.
• The ability to download file thumbnails has been rolled into DownloadFile(). Use the new DownloadType property to control what gets downloaded.
  • Additionally, the ThumbnailFormat and ThumbnailSize properties are now configuration settings.
• A number of configuration settings have been added to expose more of the Box API's functionality. Here are a few examples (refer to the documentation for the full list):
  • Shared links can now be configured when created using the CreateLinkAccess, CreateLinkAllowDownloads, CreateLinkExpiration, and CreateLinkPassword settings.
  • The VersionId setting can be used to modify the behavior of the CopyResource() and DownloadFile() methods.
  • The MetadataUpdateOrder setting can be used to specify an explicit order for metadata items to be updated in when the UpdateMetadata() method is called.

API Migration Table

Dropbox Component

Overview of Changes

• Collections are now used in the .NET/Java editions (these are the Resource* and SharedLink* properties in other editions).
• Methods now accept one or more types of "resource identifiers" directly as parameters. The ResourcePath property no longer exists in .NET/Java, and in other editions it is no longer used to determine what resource a method should operate on.
  • Depending on the method, a "resource identifier" is either a resource path, a resource Id, an Id-based relative resource path, or a revision Id.
• Resource-Id-based operations are now supported.
  • The CopyResource(), CreateFolder(), RestoreResource(), and UploadFile() methods all return a resource Id now. (Previously, UploadFile() returned a resource path, and the others didn't return a value.)
• It is no longer necessary to call ListResources() before using ListChanges(); refer to the documentation for more information.
• The RenameIfExists configuration setting now applies to the following methods: CopyResource(), CreateFolder(), and MoveResource().
• The ability to download file previews and image thumbnails has been rolled into DownloadFile(). Use the new DownloadType property to control what gets downloaded.
• A number of configuration settings have been added to expose more of the Dropbox API's functionality. Here are a few examples (refer to the documentation for the full list):
  • DownloadFile() can now download folders as ZIP files if the AllowFolderDownloads setting is enabled.
  • Shared links can now be configured when created using the CreateLinkExpiration, CreateLinkVisibility, and CreateLinkPassword settings.
  • The IfRevision setting can be used to modify the behavior of the DeleteResource() and UploadFile() methods.

API Migration Table

GoogleDrive Component

Overview of Changes

• The new GetAccountInfo() method allows you to retrieve information about the currently-authenticated user's Google Drive account. Calling this method populates the new Account object property.
• Support for shared drives has been added, including:
  • Listing shared drives with ListSharedDrives(), which fires the SharedDriveList event.
  • Listing shared drive members by passing a shared drive Id to ListPermissions().
  • The ListResourcesScope and SharedDrive properties have been added to modify the behavior of the component when listing changes and resources when desired. Refer to the documentation for ListResources() and ListChanges() for more information.
  • Checking the CanCreateSharedDrives field of the Account object property to determine whether you have permission to create shared drives.
  • Creating and deleting shared drives with the CreateSharedDrive() and DeleteSharedDrive() methods.
  • Adding, updating, and removing shared drive members with the AddSharedDriveMember(), UpdateSharedDriveMember(), and RemoveSharedDriveMember() methods.
    • Note that since shared drive members are defined by manipulating permissions on a shared drive itself, you may also use the general process for manipulating permissions (see documentation for the UpdatePermissions() method).
  • The ability to make certain requests using your domain administrator privileges with the UseDomainAdminAccess configuration setting.
• Collections are now used in the .NET/Java editions (these are the Change*, Permission*, Resource*, and SharedDrive* properties in other editions).
• Methods now accept resource/shared drive/permission Ids directly as parameters. The ResourceId property no longer exists in .NET/Java, and in other editions it is no longer used to determine what resource a method should operate on.
• Resource child and parent information handling has been streamlined.
  • ListResource(), ListParents(), and GetResourceInfo() all use the Resources collection property and the ResourceList event.
  • ListChildren() uses the Children collection property and the ResourceList event.
  • Full resource metadata is available after calling the ListChildren() and ListParents() methods.
  • Ids of a resource's parents are now available via the ParentIds field on the items in the Resources and Children collection properties.
• Manipulating permissions is now accomplished by queuing create/update/delete operations for one or more permissions, and then applying them with the new UpdatePermissions() method. The previous version's AddPermission(), UpdatePermission(), and DeletePermission() methods have been removed. Refer to the documentation for more information.
• The EmptyTrash() and MoveResource() methods have been added.
• Many methods now allow batching requests, and/or now have parameters for more common metadata:
  • AddParents(), RemoveParents(), ListChildren(), CreateFolder(), CopyResource(), and MoveResource() method all accept a comma-separated list of parent folder Ids.
  • GetResourceInfo() accepts a list of comma-separated file and folder Ids.
• The AddChild() and DeleteChild() methods have been removed.

API Migration Table

OneDrive Component

Overview of Changes

• The Microsoft Graph API is now fully supported. New functionality is now available in addition to all previously supported functionality.
  • Note: Support for the legacy OneDrive API (https://api.onedrive.com/v1.0) has been discontinued. Associated configuration settings have been removed.
  • This may impact your OAuth authorization process. Refer to the documentation for the component's Authorization property for more information.
• Collections are now used in the .NET/Java editions (these are the Resource* and Drive* properties in other editions).
• The new RemoteId and RemotePath properties are now used to select a remote resource for methods to operate against. The ResourceId, ResourcePath, ResourceParentId, and ResourceParentPath properties no longer exist in .NET/Java, and in other editions they are no longer used to determine what resource (or parent resource) a method should operate on.
• The new Drive property is now used to select the drive to operate against. The DriveId property no longer exists in .NET/Java, and in other editions it is no longer used to determine what drive the component should operate on.
  • The DefaultDrive configuration setting has also been removed; when the Drive property is empty, the component automatically selects a default drive.
• The component now has improved support for OneDrive for Business/SharePoint Online drives, since the Microsoft Graph API treats them the same as personal OneDrive drives. Component API additions/changes include:
  • The GroupId configuration setting's behavior has been overhauled.
  • A SiteId configuration setting has been added.
  • The ListGroupDrives configuration setting has been removed in favor of a new ListDrivesScope configuration setting. This new setting provides finer control, and can be used in tandem with the GroupId or SiteId settings.
  • The UsersGroups and SubSites configuration settings have been added. These are convenience settings which return Ids for groups the currently-authenticated user is a part of, and Ids of subsites of the site selected by SiteId (respectively).
  • The CopyToDrive configuration setting has been added, allowing resources to be copied between OneDrive for Business/SharePoint Online drives.
  • The CreateLinkScope configuration setting has been added, allowing organization-only sharing links to be created.
• The ListChanges() and Search() methods now populate their own marker properties (ChangeMarker/NextChangeMarker, and SearchMarker) instead of reusing the ResourceMarker property (now only used by the ListResources() method).
  • All still use the Resources collection property (Resource* properties in non-.NET/Java editions).
• A DriveMarker property has been added; the ListDrives() method will use it when necessary.
• When retrieving the status of a copy operation that has been completed, the Id of the newly-copied resource is now exposed by the CopiedResourceId property.
• For use-cases that require it, the RelativePath configuration setting has been added. When the RemoteId property is set to the Id of a folder resource, it can be used to select a resource using a path relative to that folder.
• The usage of ListChanges() has been changed slightly; refer to the documentation for more information.
• Resource paths are now exposed without the leading /drive/root: prefix so as to work correctly with the new RemotePath property.

API Migration Table

Other Updates

IPWorks Cloud 2020, like any major release, also contains numerous minor improvements and bug fixes. Here are a few we felt were particularly notable, in no particular order:

• Components that support both plaintext and SSL/TLS connection now use SSL/TLS by default.
• Support for additional encryption algorithms has been added to all components. Refer to any component's EncryptionAlgorithm property for more information.
• A Progress event, and related configuration settings ProgressAbsolute and ProgressStep, have been added to all components to allow for easier tracking of upload and download progress.
• Support for streams is now available in the C++ (Windows and Linux) and Delphi editions.
• Error codes have been consolidated and renumbered. Common error codes are now used across all components in nearly all cases.

For more information about any of the components, please refer to the IPWorks Cloud 2020 documentation, which has been heavily revised and greatly expanded for this release. We hope that you'll enjoy using IPWorks Cloud 2020.