I built an easy backup tool for VPS that focused on privacy [Free]

https://bqckup.com

Introduction

We started our small team with software and digital products. As we have handled more and more clients locally, even globally we become in deep understanding about their problem when using a platform or needed our team to handle their system including its maintenance.

For that reason, they ask us to offer the software maintenance service including hosting their app with us. Problem is although we kinda understand regarding the server we are fully aware that threats can happen to their application including the crash to their database or any file asset related to their application.

At that time, we try to find any project that is provided with open source that will be reliable to handle such issue that can let us sleep well in the night and forgetting about any type of accident that can happen anytime to all of our server.

There many type of accident that may happen :

1. The data corrupted caused by some incident from the server provider
2. The database abused by some hacker
3. The deletion of the database including the data that was accidentally caused by the server administrator or the programmer.
4. Hardware failure

Above is the notification example from the server provider regarding that server crash.

Data is priceless and the level of stress when handling the cry out from clients can make us become burnout as a programmer and can be a big treat to our company. To handle this we need to create a plan to make the backup system that handles the backup without the need of routine intervention.

Learn from the other mistakes from the data loss. We are aware of the importance of regular data backup.

Here is the story on how we built our automatic backup system from scratch.

Creating from scratch, using an open source product or using available SaaS service ?

When we started our business to the hosted service, it started from 2018 as small group of programmer that provide service to our customer to let us to handle the server. At that time, there were not many open source projects that provide that application that can handle the automatic backup.

Later we created a backup based on Bash (why not we are programmer) that can upload file to the Google Drive (Why ? yes, you are absolutely correct it is free so we choose that 🙂 ) and at that time we are using our academic account so we get abundant space that can be used to store the backup data to our client.

We call the application PGBackup ( it’s a old version of bqckup ), by the abbreviation Python Google Backup, because the name is simple.

Everything works as expected, it backs up the database and the site data without any problem.

Until ….

One day, the Google Drive “burnout” and reluctant to work with us again 😣

We have many sites that need to be synchronized to the drive, because of that the PGBackup reaches the maximum limit of request as we have to upload data in more than 300 GB / day!

And the total of the backup is almost 1 terabyte.

This kindly what happened with the Google Drive at that times.

This backup app work for about 2 years when we aware that the “free” product should always have limitation 😟

Fortunately, after this failure, there are many popularized S3 storage. After we learn about this technology, our team comes to the conclusion that we should change our platform and we also create a better backup system to allow us to synchronize to multiple storage systems.

As you have guessed, using Google Suite (pro version of Google Drive) is not an option because the cost is much higher from what the S3 provider has offered.

The other reason that we decided to upgrade and rewrite the automatic backup system is also because of the problem of the bash app that we have created. They never report any issue when the backup fails. The command line tool also make it hard for many of our team to add the new backup, the config based tool is prone to error without any knowing from our dev.

One question that may appear for us is:

Why not use available SaaS products that are already made available by some of the companies ?

Given the many data breaches that have occurred, it is natural for us to have doubts or fears about placing data in other locations. That is why we are hesitant to use third party backup and have decided to improve PGBackup, which was originally a script-based tool.

There is another reason why are we not using third party backup

1. We are a small team with limited resources and we have found that the costs associated with using third party backup solutions are too high for us
2. we have concerns about the security of our data when it is in the hands of a third party, especially in light of recent data breaches
3. We want to create our own automatic backup software that is easy to use and does not require us to rely on any external parties,

First Version of PGBackup

• Backup files & database
• Mail notification

The first version of Backup was a script-based tool that required manual setup for each website. This made the process somewhat complicated, as you needed to install a separate script for each website you wanted to back up. For example, if you had 100 websites, you would need to install 100 separate scripts to ensure that all of your data was being backed up. However, I recognized that this process could be time-consuming and cumbersome for me.

Improvements

We’ve made a few improvements to PGBackup and have renamed it Bqckup

So there’s few improvement:

YML Config Based

Storage and site config.

This is one of the simplicity implementations we apply. We no longer need to log in to the application and make configuration changes, we can directly make configuration changes in the form of a YML backup. As someone who tends to use the keyboard more than the mouse, I personally really enjoy this feature. It feels too lazy 😣 to have to click here and there, and it certainly takes more time than using the keyboard. But everyone is different, and some may prefer using the mouse rather than keyboard

Command Line Interface

Webview ( GUI )

Of course, there is also a webview (GUI), so we have covered both the command line and the interface

Future Plan

Our goal is to create a backup software that’s both safe and easy to use. I know and i understand that setting up backups can be a hassle because i already experienced it, especially if it’s confusing or takes a lot of time. And i want to make sure that other developer who using this tools is not experienced it and we are trying our best to keep it simple to set up and then let it run automatically in the background. That way, you can sleep comfortably 😪 knowing your backup is running smoothly.