Automated Backups with Node.js and Cron

If you have a web application dealing with time-sensitive data, you shouldn’t take the risk of creating backups manually. Because we, as humans, are prone to errors and things have a general tendency of going wrong exactly at those failure times.

Today, I am going to show you how to create backups using your Node.js server and automating this process with an external library; Cron.

Prerequisites;

• Node.js,
• PostgreSQL,
• Basic knowledge of command line

First things first, we need to understand what a database backup is, how to create and use it later.

So let’s start from there and move step by step.

Backup with pg_dump

A database backup is a composition of SQL commands that can easily be restored later.

PostgreSQL have a pg_dump tool to help us back up our databases.

To be able to run this utility, we first need to sign in to our Postgres server with necessary credentials, switch to Postgres user account and run this tool by adding some various configurations.

$ pg_dump -h localhost -U postgres -p 5432 -Fc -d sampledb > /Users/<username>/<path_to_dir>/sample.backup

In the above command, -host, -user, -port and -database flags are pretty straightforward. -Fc stands for custom file format; .backup in our case. And the path after the greater symbol is the location in which the backup file will be saved.

We can further improve this command to get rid of the extra authentication process by adding our postgres user password to the beginning.

$ PGPASSWORD=mysupersecret pg_dump -h localhost -U postgres -p 5432 -Fc -d sampledb > /Users/<username>/<path_to_dir>/sample.backup

This part is all set. If you run this command in your terminal with your own credentials, you should be able to see the sample.backup file in your selected directory.

Now, The next challenge is to initiate this event from Node.js server.

Run sh file with Node Child processes

Fortunately, Node.js comes with a feature called child processes that enable us to access Operating System functionalities by running any system command inside.

When we call this child process, it will trigger a special file type called shell and we will include our backup command above inside this shell file.

A shell file with .sh extension is a scripting language commands file that contains computer program to be run by Unix shell.

Ok, I accept it’s so much theory. So let’s write some code.

We will start by setting up an empty project directory.

Create a directory with a name of your own choice and cd into it.

mkdir automated-backups
cd automated-backups

Create a “backup.sh” file that will later be triggered by “exec” process. Then, copy-paste the following lines inside.

The first line means that it uses Bourne Again Shell, so called bash.

Now open your terminal and run “bash backup.sh”. You should be seeing the provided string in your terminal. If that is the case, we are good to go.

Now let’s switch the command inside, and run a pg_dump command to create a backup.

If you run the same “bash backup.sh” command again (with your own credentials and file path, of course), you should have your shiny backup file in your selected location.

So far so good. The last step of this chapter is making Node.js run this shell file, not us manually.

And it is pretty easy with the exec utility of child process module in Node.js. So let’s create a new repository and start a basic express server in our already existing directory.

npm init -y
npm install dotenv express

Copy-paste the following code by creating an index.js file in the root.

Don’t forget to add your environment variables first. You should provide NODE_ENV and PORT for now. Later, we will add our database credentials as they contain sensitive information.

Open your terminal and run `node index.js`

And, yes. Bingo. You should be seeing the sample.backup one more time.

As we are happy with the situation now, let’s find a way to automate this process in certain intervals.

Scheduling Cron Jobs

Before automating the backup process, I want to address an important issue.

On production environment, no one wants to push their credentials to Github, right.

So we should add ours to Node.js environment variables and pass them to sh file via child process.

Actually, it’s very easy to pass arguments but sometimes it might become tricky as it may vary according to operating systems.

Another potential issue might arise from the fact that special characters to be passed should be handled very carefully.

I use MacOS, so for my case the example might be like;

// passing db password and host to sh
exec("sh ./backup.sh mysupersecret localhost", (error, stdout, stderr) => {
// handle error and do sth here
})
// receiving password and host dynamically
PGPASSWORD=${1} pg_dump -h ${2} -U postgres -p 5432 -Fc -d sampledb > /Users/<username>/<path_to_dir>/sample.backup

As you can see, after “sh ./backup.sh” section we are passing them as strings with single space between one another and receiving back in sh file with their indices.

Be careful, the index starts from 1, not from 0.

Now that we learn this, let’s move all our credentials to .env file, create a time-focused unique backup name and then pass this whole stuff from exec utility to sh.

It might also be a good idea to wrap this whole process in a function.

And our sh file will be like;

Now, we can automate this process in a secure way.

We simply want to schedule a task that will be performed by Node.js in certain intervals. Frequency totally depends on your needs and the nature of your application.

There are a couple of npm libraries to carry out this task but I will be using Cron.

You can refer to its github repository to learn more from here.

So let’s install and use it.

npm i cron

Usage is pretty simple. If we want to create a backup everyday exactly at 23.59, the code will be;

Passing the fourth argument as true, we want to start the job instantly without waiting for any further command.

That’s it. Now the whole process is production-ready and you can safely use it.

In this article, I tried to explain how to create automated backups with Node.js server and Cron.

I used PostgreSQL database but you can use another if you want. The logic will not change.

I will be happy if you share your thoughts and suggestions in the comments.

You can reach the full codebase from here.