← All workCase study 13

cloudstorage-py

A Python SDK giving one consistent storage interface across AWS S3, Google Cloud Storage and Cloudflare R2 — swap providers by changing a single string.

Type

SDK

Status

Shipped

Context

Open source · packaged at v1.0.0

Period

2026

Source

Public

3

providers, 1 interface

1

string to switch backend

0

credentials needed to test

(01) Overview

Every cloud provider ships a different SDK with different method names, different exception trees and different configuration. Switching providers means touching every call site. cloudstorage-py puts an abstract provider contract in the middle: the client never branches on which cloud it is talking to, every provider returns the same typed models, and three unrelated exception hierarchies are normalised at the boundary into one. Upload, download, delete, list, batch operations and presigned URLs work identically across all three. The test suite is fully mocked, so it runs in CI and on a contributor’s laptop without live cloud credentials.

(02) Architecture

Abstract BaseProvider contract with three concrete implementations · typed result models identical across providers · unified exception hierarchy normalised at the boundary · environment-resolved per-provider config · batch operations and presigned URLs · fully mocked test suite · packaged with per-provider optional extras

(03) What was built
  1. 01One consistent interface across AWS S3, Google Cloud Storage and Cloudflare R2, with providers swappable by changing a single config string — no call-site changes.
  2. 02An abstract provider contract is the whole design: the client never branches on provider, so Cloudflare’s S3-compatibility quirks stay absorbed inside R2Provider instead of leaking upward.
  3. 03Typed data models mean an upload result or a listed object has the same shape whichever cloud produced it.
  4. 04A unified exception hierarchy normalises three unrelated error trees at the boundary, so application code catches ObjectNotFoundError rather than botocore, google.api_core and Cloudflare errors separately.
  5. 05Batch operations and presigned URLs supported uniformly across all three providers.
  6. 06Fully mocked test suite — no live cloud credentials needed, which keeps the tests runnable in CI and by any contributor.
  7. 07Packaged with per-provider optional extras, so installing it for S3 alone does not pull in the Google and Cloudflare stacks.
(04) System design

Drawn from the source

Services, data ownership, message flow and failure paths. Drag to pan, scroll to zoom, or open any diagram fullscreen.

One interface, three providers

The abstract provider contract is the whole design. Every provider implements exactly that surface, so the client never branches on which cloud it is talking to, and provider-specific behaviour stays behind the boundary.

Fig. cs-arch
Rendering diagram
compute / servicestate / storagemessagingobservabilitythird partygood outcomefailure pathkey decision
(05) Stack

Python / boto3 / AWS S3 / GCS / Cloudflare R2 / pytest

Next case study

envy

→