Managed Database: how to choose the right type of storage

Data is critical to any organization in many different ways as data is knowledge, and knowledge is power. From customer data to human resources to core business services, inventory tracking, growth monitoring, and more, databases are the cornerstone of any software application.

Cloud databases, also referred to as managed databases or Database-as-a-Service (DBaaS), make the provisioning, configuration, and management of databases dramatically easier and faster, turning databases into a true utility service.

Their agility and intuitive nature enable developers to turn and run services in seconds. Even without any particular expertise in database administration, you can create and automate the provisioning of highly available, backed-up, secure, and ready-to-use databases, allowing you to dedicate your time and energy to building applications and producing value.

This article will shed light on the main differentiators when it comes to choosing the storage type for your managed database.

Understanding managed database

The predicted volume of the global datasphere by 2025 is measured in zettabytes (that’s 21 zeros!) and many aspects of today’s technologies contribute to this growth. The data streams collected by connected devices and sensors (Internet of Things - IoT), the countless applications and services created everyday, and providing new ways to generate data, as well as the changes in software architecture towards microservices requiring multiple databases (while a monolithic application requires a single database) are all contributing to this exponential growth. All those different data types, volumes, and processing requirements, in addition to the growing variety of databases, including both SQL and NoSQL make hosting and supporting them a serious challenge from an IT perspective.

A managed database service enables you to effortlessly set up and maintain the service using a UI or an API, or even an Infrastructure as Code. It includes engine upgrades, automated remote backups, high availability, and security. In addition, Scaleway Managed Databases come with 24/7 assistance so you can be serene regarding your runtime and able to focus on development.

While managed database provides a common set of abstractions through which a database can be consumed, you still need to be able to configure your database instances in order to make the most of this service, in terms of performance, flexibility, security, resiliency, and more.

First, let us start with an overview of the different compartments forming a cloud database service:

Managed Database manages resources residing on physical hardware provisioned using an Infrastructure-as-a-Service abstraction through the orchestration engine. A set of best practices and corporate standards for database operations and policies are configured in an optimized way, with great attention being paid to the system’s security.

Storage is an integral part of any database, as your data is kept there, and you want to make sure it meets your needs perfectly in terms of data accessibility, scaling, and resilience. You can also choose the storage type that maximizes performance or improves cost management.

Choose the right storage

To understand the storage types compatible with a cloud database and their main characteristics and differentiators, below is a quick overview:

Local volume

When you create a new instance, a boot volume is created directly on the bare-metal hypervisor which can be seen as local storage for your instance. A local volume is tied to that instance, and so is its lifespan. This storage type is particularly adapted to high storage performance requirements

Block volume

You can store data in a block volume beyond the lifespan of your instance. You can also upgrade your block size or change your instance capacity independently from one another

Object Storage

Based on the Simple Storage Service (Amazon S3) protocol where objects are organized in buckets. This is suitable for storing documents and multimedia content that you can easily upload, download, and visualize any time, from anywhere in the world. A wise choice for database backups

Cold Storage

If the data you want to store is rarely used and you want to retain it for a longer period of time then this option is the most cost-efficient.

Scaleway’s Managed Database for PostgreSQL and MySQL supports two types of storage: local storage and Block Storage. The following pages will hopefully answer all your questions on the matter.

Local storage vs Block Storage

Key differences

Below is a summary of the key differences that you should consider when choosing a type of storage for your Managed Database.

Performance benchmark

Bench dataset size = 23GB
Region : Amsterdam
Instance type : db-gp-xs, 4 vCPU, 16GB RAM
TPS : Transaction Per Second

In short: Block Storage brings more flexibility and scalablity, while local storage offers optimal performance. Local is by definition faster due to its proximity to the compute node. Block Storage for Managed Database offers greater flexibility allowing cost optimization by reducing the compute power when not needed. In addition, the several upcoming improvements on our Block Storage-based databases make it a better option for cloud native uses. Some of the planned evolutions are:

• Snapshot exports in a different AZ
• Read replicas for both local storage and Block Storage databases
• Block Storage performance improvements

Backups vs. Snapshots

We’ve seen in the table above that local storage offers distant backup possibilities while Block Storage relies on snapshots. Let’s dig a little deeper into the main differences between these two disaster-recovery mechanisms.

Local Storage

On local storage database instances, backups are logical exports of a database.
Logical backups are actually data extractions from the engine which require at least one CPU thread to retrieve and compress data.
In case not all the data is available in the memory, it needs to be retrieved from the disk, generating IOs which will penalize the other ongoing operations.
To ensure a consistent database state, logical backups are made in a single transaction, which impacts the execution time of backup creation, table locks, and potential latencies.
However, once created, backups have no expiration date and can be stored in a distant region and offer the possibility of using a multi-AZ option (in Paris). They can also be downloaded, making them portable from one major version to another.

Snapshots

The large size of the block volumes makes logical exports expensive in terms of CPU and memory, in addition to being vastly time consuming.
Block snapshots are almost instantaneous, don’t require much computing power from the database, and do not disturb normal operations. This is called Hot Backup.
Snapshots restoration will not overwrite your initial data as they allow you to create a new instance from the existing snapshot. This process increases the security of this delicate operation, as you keep your original database until you are sure everything is working correctly.

Local or Block?

In general, different storage types are suitable for different use cases and needs.
While local storage wins in terms of performance (for now), Block Storage stands out with increased and seamless scaling possibilities.

Now that the strengths and limitations of each storage type have been broken down, we hope you feel more confident making the right choice for you.
The Database team is committed to continuously improving the service, Block Storage for Managed Database is one of many features designed to meet our users' needs and is now in General Availability. Give it a try and stay tuned for our next exciting announcements!