Cloud Database Backup as a Queryable Data Lake

Abstract

Managed cloud databases such as RDS, Aurora, Cloud SQL, and Azure SQL run inside the cloud provider. You cannot install an agent, read the disk, or access the log files. The only backup tool the provider offers is a snapshot API call. A snapshot is a black box. To read even one row from it, you must first restore the whole database. Long retention makes this worse. Aurora snapshots stay incremental for at most 35 days, so keeping backups longer, in another account, or in another region forces full copies. A 1-year policy stores 12–52 full copies of every Aurora database.