Migrating from AWS to Scaleway using Terraform

LockSelf is a French startup specializing in cybersecurity. We are developing solutions for access security and management, as well as sharing sensitive passwords and documents. We recently migrated out of AWS and into Scaleway - and a lot of European Startups will face the same mission to get back the sovereignty of their data. We wanted to share our experience doing so. We hope it will help some of you.

Lockself: data security as a core business

The data we manage is critical: passwords, sensitive data, and so on. The organizations we support have particularly high standards for their storage - both in terms of where it is hosted and by whom. Our first challenge, more than just data encryption, is to make sure employees’ accesses are secure, as they are the main gateway in the event of a cyberattack.

We used to host our data with AWS. However, as we grow, the demand for a sovereign cloud is growing too, which is a real business challenge for us. Beyond a qualitative product, our clients need their data hosted in France, and this can’t be done properly without a French cloud provider.

By changing providers, we can answer the growing demand from our clients’ end users or suppliers regarding the security of the solutions used to share and manage specific sensitive data.

It is a strategic issue as we always wanted to position ourselves on a secure end-to-end approach with a high level of security on the application and encryption side and a high level of trust on the data storage side. This will now translate into 100% French cloud hosting offers adapted to the needs of each client. The on-premise offer obviously remains, now more than ever, relevant for players wishing or having the obligation to manage our solutions internally.

Finally, working with a French stakeholder such as Scaleway, capable of providing us with the same level of service as AWS, is aligned with the values that we hold in terms of sovereignty and on which we must be consistent, especially as we work in cybersecurity.

Multi cloud as a way to back up data

Some of our customers are still hosted by Outscale as Outscale was historically used alongside AWS for our dedicated hosting offer. Today, this multi-cloud strategy allows us to rely on each provider to outsource backups from one to the other - backups made on Outscale are outsourced to Scaleway and vice versa. This operation allows us to have a healthy backup management 100% based on French hosting.

From AWS to Scaleway

We used AWS ec2 for our application instances, RDS for databases, Amazon S3 for storing our files and a load balancer to distribute the load on the different ec2 instances.

Being a data vault, we host both passwords and files. We store our files on Amazon S3 and our passwords on a Mariadb database.

The data migration was done via Rclone from Amazon S3 to Object Storage. We had planned our migration via Rclone ahead of time, so that most of the files could be transferred immediately. On migration day, the Rclone only had to sync the latest changes and the migration took no more than an hour.

From Amazon S3 to Object Storage: using Rclone to migrate

Rclone is a command-line tool that allows you to copy/synchronize data stored in objects such as Amazon S3 at AWS and Object Storage at Scaleway. It is a sort of Swiss army knife that allowed us to directly communicate between Amazon S3 and Object Storage without going through a transition server.

The database migration required more preparation because Scaleway’s Database service does not offer Mariadb. So we decided to take advantage of Kosmos to deploy a Mariadb Galera cluster.

This also allowed us to have a database replicated in several hosting zones (fr-par-1, fr-par-2, fr-par-3). Thanks to this system, we benefit from very high availability and resilience for the storage of our passwords.

Terraform to simplify the migration process

Our application, developed in Symfony, was hosted in Docker containers at AWS, on several servers to ensure high availability and positioned behind an Elastic Load Balancer.

To replicate this at Scaleway, we used a Load Balancer to continuously receive incoming requests which are then transmitted to the Ingress-Nginx Load Balancer of the Kubernetes cluster managed by Scaleway. The load is then distributed between the different application containers. This configuration allows us to achieve an even higher level of availability than on AWS.

Our backup system did not need to undergo any modifications because it still uses s3cmd, and only the destination of the backups was changed.

The configuration of this infrastructure, managed by a Terraform project, allows us to maintain an Infrastructure as Code, and to gain agility in maintaining it in operational conditions.

The aftermath

Following the migration, we observed shorter response times, which can be explained by the closer location of our new servers compared to the old ones, and by the improved performance of Scaleway’s servers. We did not initially need such high performance, but all of Scaleway's servers - including the least efficient - are equipped with high-performance processors.

As a result, we have chosen Scaleway’s least powerful servers, but even these remain more efficient than our previous servers at AWS, even if they are more expensive. However, for the moment, we find the price/performance ratio more interesting at Scaleway than at AWS. We, therefore, expect to receive a higher bill, but with an infrastructure that can support greater use.