Hard to believe we are 129 comments into this and there are zero mentions of 'borg' in the comment threads ...
For those that don't know, borg is a backup utility[1] that has been called the "holy grail of backups"[2].
It takes your plaintext files and directories, chops them into gpg-encrypted chunks with encrypted, random filenames, and will upload (and maintain) them, with an efficient, changes-only update, to any SFTP/SSH capable server.
My understanding is that the reason people are using borg instead of duplicity is that duplicity forces you to re-upload your entire backup set every month or two or three, depending on how often you update ... and borg just lets you keep updating the remote copy forever.
The crypto in Borg is bad. Even has its own page in the docs about how bad it really is. The tl;dr is "there is one AES key per repository and we use this with AES-CTR and trust the remote server to not fiddle with the counter".
Last time I checked there was still no work being done on improving performance. Multithreading has been on the agenda for years and will probably never come, much like improved crypto.
Like many open source projects, Borg is low on developers. The two points you mention are both known and on the agenda for this year. Since you seem to have experience with crypto, why not add a PR? It's all in Python and fairly readable.
When someone says something is the "holy grail of backups" and it is "gpg-encrypted" I assume it does that well, not it does it poorly right now but might improve in the near future.
Thus his post was valuable to me.
"Why not add a PR" is not a reasonable response to something like this. I'm looking to use a backup system, not create one. If something works so poorly that I'd have to start submitting PRs before using it, I'm far more inclined to go elsewhere.
> When the above attack model is extended to include multiple clients independently updating the same repository, then Borg fails to provide confidentiality
Let's also mention Vorta, an open source GUI for Borg to make it as easy to use as commercial backup tools. We just started translating into different languages. All input and PRs are welcome.
Have you looked at Arq[1]? It deduplicates, encrypts, compresses, backs up to most major cloud providers and SFTP and Minio, is available for macOS and Windows. We've been improving it for 10 years. The data format is open and documented[2].
You've probably heard this before but there are a lot of people who'd be interested in a cross-platform (NAS/Linux/BSD/Raspberry Pi) version with a command-line or web interface.
As is it's really best for workstations rather than the NAS/servers a lot of people who care about their data use (not just basement-dwelling data hoarders but also professionals like photographers and videographers).
I currently use Borg on my servers, and Arq on my client machines.
I'm quite sure if Arq was available in a command-line version for Linux/FreeBSD, i would use that instead. It has been nothing but rock solid on my client machines.
Restore through the UI is a bit wonky, but once it's started it works well.
My biggest reason is probably that i prefer to use the same (well known / proven) tools across the line.
I don't think Arq brings something to the table that Borg/Restic/Duplicity doesn't already provide, besides a piece of software i'm familiar with, and trust. (and trust doesn't come easy when it comes to backup software!)
As i wrote, i already use Arq on my client machines instead of the mess that is "TimeMachine over wifi", but when it came to backing up my NAS and server/VPS, i had to look for something else, as Arq is not available on those platforms.
My use case would be backing up with Arq from a FreeBSD/Linux NAS/Server to a remote (networked at least) target, and in case of a full restore, i would use the command line as well.
I would then use Arq for Mac/Windows for "plucking" single file restores.
Have you checked out duplicacy[1]? It's support for incremental backups is fairly unparalleled in my experience. It does not have the same limitation of duplicity.
I know what you mean. It is not a normal open source license and for commercial use it requires a $20 a year subscription per user. With that said, I believe all the core source code is viewable and auditable if you use the CLI version, which is what I use on my FreeNas system and there is no charge for personal use. I think this may meet many people's personal backup requirements.
restic was mentioned a few times and is a very similar project with very similar goals. They both have encryption as a first-class concept and are snapshot based with content-defined chunking. I'd hope most people would be using these newer generations of backup tools.
My favorite benefit of restic is that it doesn't require the receiving end to have any special software: you can use restic to backup to S3 or BackBlaze B2 directly.
Same! I’ve been using Duplicity for most of my backups and had been considering switching to Borg, but passed when I found it couldn’t use B2 as a backend.
Now I’m using both Duplicity and Restic, for different backups.
Yup, though due to Australian internet I don't use restic's support for non-local backends (otherwise getting a snapshot list or doing a purge+clean is painfully slow) -- I have a sync job that runs every day that uses rclone to sync the restic backup store.
I've been using restic for a long time now. It's flexibility is amazing:
* Fully open-source, command line implementation. Script as needed.
* Supports ton of backends, including directories, SFTP, and many cloud services.
* Supports multiple backup operations in parallel. Create one repo for multiple machines.
* A repo is really an encrypted object store with a lists of directory/file/metadata references (snapshots). If the same file exists on multiple directories/drives/machines, it is deduplicated.
* Supports tagging, which allow you to identify independent snapshots and manage them.
* Supports purging old data with conditions, ie. keep last 7 daily, 3 weekly, and 6 monthly.
* Can verify and repair repositories.
I regularly use it to backup my home server to a backup drive and to Backblaze. I've also used to back up my home directory of my development machine to replicate it on another machine.
I'm setting up backups and looked into most available solutions, including restic, borg, and duplicity, but Bareos by far seems the best one. It's just a little harder to set up, but totally worth it.
Love borg, can't rate it enough. There's also a pretty great service that supports it, you might have heard of it? ;) https://rsync.net/products/attic.html
Not only is borg great, rsync.net has a super cheap plan for borg and attic that is vastly less expensive than other cloud storage. I have about 140GB backed up this way and couldn't be more happy with the toolchain.
I've been using Borg (and it's predecessor, Attic), for a while now and I'm very happy with it. Perhaps my only complaint would be that at one time they changed the hashing function which exploded the size of the repo for a while (since content was getting stored twice until all the old chunks were rotated out). Duplicity managed to lose my data several times, but so far I didn't have any such bad experiences with Borg.
For those that don't know, borg is a backup utility[1] that has been called the "holy grail of backups"[2].
It takes your plaintext files and directories, chops them into gpg-encrypted chunks with encrypted, random filenames, and will upload (and maintain) them, with an efficient, changes-only update, to any SFTP/SSH capable server.
My understanding is that the reason people are using borg instead of duplicity is that duplicity forces you to re-upload your entire backup set every month or two or three, depending on how often you update ... and borg just lets you keep updating the remote copy forever.
[1] http://borgbackup.readthedocs.io/en/stable/
[2] https://www.stavros.io/posts/holy-grail-backups/