How We Replaced AWS S3 with Ceph RGW and Saved €30,000/Year
If you’re storing hundreds of terabytes in AWS S3, you may already feel the pain: unpredictable bills, vendor lock-in, and limited control. At a certain point, it’s not just about convenience anymore—it’s about cloud cost optimization, infrastructure control, and data ownership.
In this post, we break down when and why it makes sense to move off S3 and build your own object storage solution using Ceph and Rook—with real-world cost comparisons, operational challenges, and S3-compatible architecture insights.
Why Companies Are Rethinking AWS S3 for Object Storage
Amazon S3 is the dominant solution for cloud-based object storage. It’s stable, scalable, and deeply integrated into the AWS ecosystem. But as usage scales, so do the downsides:
Egress fees make hybrid and multi-cloud strategies difficult
Compliance and data residency limitations arise
Vendor lock-in impacts architectural and financial flexibility
Even companies like Basecamp & 37signals publicly moved off AWS S3 to regain control and reduce costs.
Why Many Companies Avoid Ceph as an AWS S3 Replacement (and How Rook Fixes That)
The most common pushback we hear:
“Ceph is too complex. And hiring engineers to run it is expensive.”
They’re not wrong.
Ceph is a powerful, distributed open-source storage platform—but it’s designed for scalability and flexibility, not simplicity. Operating Ceph clusters requires experience in storage operations and monitoring.
That’s where Rook comes in.
Rook + Ceph: The Best Open-Source Alternative to AWS S3
Rook is a Kubernetes-native operator that simplifies Ceph cluster deployment and management:
OSD and storage pool automation
S3-compatible Ceph RGW deployment
Monitoring via Prometheus & Grafana
Native Kubernetes integration and GitOps readiness
With Rook, you can run S3-compatible object storage inside Kubernetes with a declarative and cloud-native approach. But let’s be honest: no setup is frictionless.
In our first deployment, we underestimated tuning of OSDs and pool CRDs. The system ran, but not well. After two reworks and proper alerting with Prometheus + Ceph exporter, it stabilized. That kind of experience is exactly why we document our stack—and share it.
We’ve helped multiple companies replace AWS S3 with a fully managed Ceph-RGW solution—no in-house storage team needed. We’ve followed a similar philosophy with our support stack — switching from Zendesk to Zammad for cost and control. Here’s how and why.
When Should You Build Your Own S3-Compatible Storage?
Self-hosted object storage becomes cost-effective and strategic when:
One of our clients migrated their logging and archival data (240 TB) from AWS S3 to a Rook-Ceph deployment.
Results:
Annual cloud storage cost reduced from €36,400 to €3,500
S3 API compatibility preserved (no application changes)
Observability stack improved SLA tracking by 26%
We’ve replicated this success across several environments—in Germany, the UK, and Türkiye.
Example Architecture: S3-Compatible Ceph Deployment
For that 240 TB client setup, the configuration included:
6 bare-metal nodes (Hetzner EX101) with 64 GB RAM, 2x NVMe (journals), 4x 16TB SATA
Rook operator installed via Helm in a Kubernetes production cluster
Ceph RGW endpoints exposed via HAProxy with Let’s Encrypt TLS
Full observability via Prometheus, Grafana, Ceph-exporter, Loki
We also published internal automation playbooks for reproducibility.
💡 Reality Check: Is Ceph Always Cheaper?
Not always. Here’s a visual comparison of annual costs in different scenarios:
As you can see:
Ceph offers massive savings for standard S3 usage.
For hot data or managed deployments, savings are moderate.
If you’re already on AWS Glacier, Ceph might not reduce your costs at all.
Use case matters. That’s why we guide clients through these decisions carefully.
Frequently Asked Questions (FAQ)
Is Ceph a good alternative to AWS S3 for object storage?
Yes, especially for workloads exceeding 200 TB. Ceph offers S3-compatible APIs, cost efficiency, and full control over your infrastructure.
Do I need a dedicated team to manage Ceph?
Not necessarily. With Rook on Kubernetes and good observability tools, many organizations operate Ceph with their existing DevOps teams. Alternatively, Kubedo offers managed Ceph services.
How much can I save by switching from AWS S3 to Ceph?
In our client example, annual storage costs dropped from €36,400 to €3,500—a reduction of over 90%.
Can I still use my existing S3 applications with Ceph RGW?
