How to back up Nextcloud with Bareos

People make mistakes, and hardware breaks — even if your data is securely stored in a private cloud with Nextcloud, it’s a good idea to be prepared for the worst case. Regular backups, ideally on external storage, can save your data and even your business. Read this guest post by Heike Jurzik from the Bareos team to find out how to back up Nextcloud with Bareos.

Backup Concepts: Files, Images, and Applications

Some users will be perfectly happy to save single files or folders. It’s ok if the goal is to back up personal data, documents, pictures, etc. A file backup won’t save the applications or the operating system, though. Another approach is to create an image of the entire machine and therefore of the operating system as well. Image backups include files, programs and configuration files. If a file backup is not good enough and an image backup too much, then the alternative is to save an application and its data.

Before we take a closer look at Nextcloud and how to back up the application itself as well as the stored data, let’s discuss the challenges when planning application backups. It’s not just a few files you have to take care of because applications have an internal state that is stored in multiple places. Some parts of the data may be cached in the RAM and will be written to a file later. So, until that happens, the data in the file is incorrect. Sometimes applications store information about files in a database table for faster access. And, of course, the application might write data to a file at the same this file is being backed up. It’s a bit like a jigsaw, and a backup plan needs to consider all pieces that have to fit together again after the restore.

Planning your Nextcloud Backup

To back up a Nextcloud installation, you need to take care of files and folders located in the filesystem itself (conventional directory structures). Apart from that, there is a database that stores additional information like the number of files, permissions, timestamps, etc. All in all, there are four major components to deal with:

• The folder nextcloud/config
• The folder nextcloud/data
• The folder nextcloud/theme
• The database (can be MySQL/MariaDB, SQLite, or PostgreSQL)

It’s possible to manually back up folders, files, and the database. Have a look at the Nextcloud documentation to find out how to achieve that. It basically involves shutting down the application, letting it store its internal state on disk, backing up all files, and restarting Nextcloud.

There is only one problem with this quick and dirty approach: The service is being interrupted while the application shuts down and the backup job is running. If you’re offering 24/7 access to Nextcloud in a professional environment, your customers might not accept downtimes. So, in the next few sections we’re going to show how Bareos can help you with an automated solution. Now, if you’re familiar with Bareos, you can jump straight to section «Teamwork: Bareos and Nextcloud». If not, we’re going to give a quick introduction to the backup solution.

Open Source Backup Solution: Bareos

Bareos (Backup Archiving Recovery Open Sourced) has been around since 2010. The project started as a Bacula fork and is now 100% Open Source (AGPLv3). If you’re looking for professional support, the company Bareos GmbH & Co. KG and their partners provide this kind of service.

The cross-network backup solution preserves, archives and recovers data from all major operating systems. It’s a client-server setup, and several programs communicate over the network: the Bareos director (BD), one or more storage daemons (SD) and one or more file daemons (FD).

• The director is the central control unit and manages the database (catalog), connected clients, file sets (that define which data gets backed up), the configuration of optional plugins, before and after jobs, storage and media pool, schedules and the backup jobs.
• The catalog maintains a record of the backup jobs, saved files and volumes used. The preferred DB backend is PostgreSQL, although Bareos supports MySQL/MariaDB and SQLite (experimental) as well.
• The file daemon runs on every client machine. It’s responsible for the backup and the restore process. The FD executes the director’s instructions and sends the data to the storage daemon.
• At the director’s request, the storage daemon receives data from one or more file daemons and stores the backups on the configured backup medium (disks, tape drives, cloud storage via S3, etc.). During the restore, the SD sends the correct data back to the FD(s).

Let’s not forget to mention backup jobs, schedules, and directives. A backup job describes what to back up (the client’s FileSet directive), when to back up (schedule) and where to back up (backup media). The schedule also defines the kind of backup (full, incremental, or differential). Fore more information about Bareos, please have a look at the documentation.

Teamwork: Bareos and Nextcloud

So, it’s time to describe three different methods and give you some ideas on how Bareos can handle the backup and restore process for Nextcloud servers, including the database. Detailed listings of the scripts and the configuration files we’re mentioning can be found in a technical whitepaper that was published in January 2019.

Dump, Export, Backup!

In this first approach, the data gets exported before the actual backup runs. We’re performing a database dump with the respective tools (i.e. mysqldump, sqlite3, or pg_dump) to save the original database object definitions and table data. The Nextcloud service doesn’t have to be interrupted, so there is no downtime.

You can use the RunScript directive in Bareos to define commands you would like to execute before or after a backup/restore job. It’s possible to run those commands either on the Bareos file daemon (ClientRunBeforeJob and ClientRunAfterJob) or on the Bareos director (RunBeforeJob and RunAfterJob).

So, the whole setup defines one backup job on the client that lists two external shell scripts to be executed before and after the backup job. The first script puts Nextcloud in maintenance mode and runs mysqldump, sqlite3, or pg_dump (for all databases as a single transaction, so it doesn’t lock all tables). The second script dumps the database back and turns off the Nextcloud maintenance mode.

The disadvantage of this solution is that the dump may slow down the database daemon, and the backup job takes longer, since the export has to be done first. If you’re dealing with a large database, this might also take a while (depending on the hardware). Plus, the exported data takes up extra space. So, let’s try to avoid the temporary database dump and save some time and hard disk space.

Streaming Data

In order to avoid storing the database dump in a temporary file, you can use the Bareos bpipe plug-in to stream the database dump to Bareos. The plug-in can also transmit the data from Bareos to another specified program for restore later.

Instead of defining RunBeforeJob and RunAfterJob directives, the plug-in is set up in the director’s configuration file (section Include in the job’s FileSet resource). The bpipe plug-in also executes two external scripts that use mysqldump (or any of the other dump commands) to read from stdin and write to stdout. Since the plug-in runs on the Bareos client(s), it’s required to tell the file daemon to load the plug-in.

This method saves a lot of time because exporting the database and the backup job can run at the same time. In addition, it saves disk space because a temporary file for the database dump is no longer necessary.

Plug-in for your Database

If your Nextcloud installation uses MySQL or MariaDB as database backend, your third option is to use the bareos_percona plug-in. It relies on the xtrabackup tool from Percona to perform full and incremental backups of the database. Especially the incremental dumps are a big advantage when it comes to larger databases.

Like the bpipe plug-in, you need to configure the bareos_percona plug-in on the Bareos file daemon. It’s also mandatory to install xtrabackup from the Percona repository.

Keep in mind that this plug-in does not take care of the restore process. Instead, you end up with a temporary directory with all the files, and you need to use the percona commands to do a MySQL/MariaDB restore. Of course, writing a shell script that takes care of this is possible (see our whitepaper for an example).

Be prepared!

So, taking control over your data with Nextcloud is a first step towards more privacy. But even the most secure private cloud needs a good backup plan. Our advice is to stick to Open Source software, and, of course: Test everything, especially the restore process!