Kloudless Blog

Tutorials, case studies and how-tos from our experts

Integrating with File Storage vs. Object Storage

There are two broad categories of cloud storage for unstructured data:

  • File Storage: A personal cloud that users use to synchronize and collaborate on files. Examples include Dropbox and Google Drive.
  • Object Storage: A component of an application’s stack that stores unstructured binary data as files or “blobs”. Examples include Amazon S3 and Google Cloud Storage.

When consumers mention “Cloud Storage”, they almost always mean data sync and collaboration tools like Dropbox and Google Drive that let users store files remotely. Both categories of services above store file data and offer APIs to access them, but the primary user of file storage is usually an individual, whereas the primary user of object storage is usually an application.

In this post, we’ll consider the benefits of integrating with each type of storage service. Kloudless offers a unified cloud storage API that provides the ability to connect to both file and object storage services with a single implementation. However, the predominant use cases of each greatly differ.

Why integrate?

First, apps must decide which set of cloud storage providers to integrate with and why.

File storage platforms offer access to end-user data. Applications that enable end-user workflows, such as productivity tools, find this data valuable to transfer into or out of their application. User-generated content is rarely in a single app, but collectively represents a source of truth. Hence, apps must access each others’ APIs to enable the workflows that users demand.

On the other hand, users rarely use object storage services such as Amazon S3 directly to create, sync, and collaborate on content. However, an application’s user-base may collectively generate a large quantity of content, such as new versions of images, annotated PDFs, or their own files uploaded to the app. An application can integrate its backend with an object storage service to abstract away scaling concerns when storing this vast quantity of data.

The Kloudless File Explorer highlights the contrast described above. Developers can use it to prompt users to select files to import into their application for any purpose. Via the Kloudless portal, developers can configure the File Explorer to automatically send content uploaded by users from their machine, or selected from cloud storage, to the app’s own object storage, such as S3.

This key difference manifests itself when gaining access to the remote storage system in the first place. Apps prompt users to grant access to their data in cloud storage by guiding them through an OAuth 2.0 flow. However, apps access object storage using just a secret key provided by the storage service.

Features and Performance

User-facing cloud storage services offer a myriad of features to differentiate themselves from each other. Most of these features center around boosting end-user productivity. They enable users to view, edit, and collaborate on content flexibly across all their platforms. With some exceptions such as Box, storage services do not prioritize exposing this functionality for applications built on their platform to white-label.

However, object storage services’ primary value proposition centers around performance and cost at scale. These services bill on quantity of data stored and accessed, rather than the number of users accessing them. Their APIs offer higher rate limits and lower latency, especially within an app’s own hosting provider. They offer flexible approaches to store and transfer content, including which data-center stores an app’s data, data durability guarantees, CDNs, custom headers for caching, and more.

Alternate perspectives

Despite the contrasts described above, we’ve seen applications use cloud storage providers such as Google Drive as a backend for application data uploaded by users. Apps might want their own staff or other users to require direct access to end-user content. In fact, the File Explorer mentioned earlier supports all storage providers as backends for user content.

We’ve also seen apps access their users’ object storage accounts directly. Kloudless abstracts away object storage providers’ key-based authentication schemes behind a consistent OAuth 2.0 flow to enable easy access to a user’s own S3, Azure, Google, or other S3-compatible service. Tech-savvy customers realize the cost-savings gained by directly using the infrastructure that powers file sharing platforms. Users access object storage directly using open source file browsers like Cyberduck.

Check out our customers Octiv, 1947 Partition Archive, and Blackboard to guess which path each of them took! We’d love to learn about other use cases we didn’t cover above. You can reach us at hello@kloudless.com to discuss your strategy for storage integrations anytime!

Published By

Vinod Chandru