Restic Backup Questions

Harvey

brent With no explanation this command assumes restic knows where to find excludes.txt. Uness this part of command (/srv/restic-repo) is the clue for restic?

This is the restic repo (backup) location. For you I believe you said that was /mnt/backups later /run/media/brent/backup I too find their examples confusing. --exclude is yet another way to exclude files matching a patern which is not what we are doing using an exclude file.

brent restic backup /home/brent --verbose -r /run/media/brent/backup exclude-from'/home/brent/.excludes.txt'
look allright?

Nope. I made a typo --exclude-from it should have been --exclude-file and you missed the -- and the = (I'll go back and correct mine).

restic backup /home/brent --exclude-file='/home/brent/.excludes.txt' -r /run/media/brent/backup

That should work. Never used verbose but man restic-backup says:

-v, --verbose[=0] be verbose (specify multiple times or a level using --verbose=n, max level/times is 3)

So if you use verbose it should be --verbose=n where n is just how verbose you want it to be up to 3.

brent

still, the first thing restic digs into is.config. .config is the first entry in in .excludes.txt.
so it still doesn't see list. I see list in /home...File is named right.
the command was restic backup /home/brent --exclude-file='/home/brent/.excludes.txt' --verbose -r /run/media/brent/backup

I got rid of verbose alltogether and it still went straight for the excluded .config. this has got to be something simple I'm overlooking. I see nothing else in restic docs. Sorry to keep rousting you. Will try again later with the full backup.

Harvey

brent
Can you post the contents of your .excludes.txt?

Make sure there is no leading or trailing spaces on each line.

brent

certainly; cut and paste, as it:
.cache
.local
.nv
.pki
Templates
.vscode
.wget-hsts
snap
Public
Music
.dvdcss
.pacman_sdl
.pan2
.thunderbird
.wget-hsts

Harvey

brent

I recreated the file and dropped .txt from its name just to rule out any weirdness there and any special characters that may be in your .excludes.txt as well as making sure it really is a plain text file.

EDIT file deleted

Click the download button and save it wherever you would like. Google drive / browser will strip the leading . from the file. Be sure to point your command to the new file. You can add --dry-run to the end of the restic command and it will run through the files without actually creating the backup.

It appears to be working for me.

brent

Harvey thanks for this. will give it a dry run later in the day.

brent

sudo restic backup /home/brent --exclude-file='/home/brent/excludes' -r /run/media/brent/backup

this is the command. would not run until I removed .txt.

enter password for repository: 
repository 87e00e75 opened successfully, password is correct
open /home/brent/excludes.txt: no such file or directory

But then immediately downloads the excluded .config file again. Your clean text was accepted by restic, but it only has eyes for .config....I put .excludes.txt in the trash to be sure.
top half in nautilus shows address/route correct:

where .config is plus new excludes:

permissions on excludes:

Question: why does restic backup an excluded folder?
Answer: it doesn't know .config's folder? some minor argument I'm overlooking in --help?

the only other troubleshooting I can do is to let it go and see if it backs up all the excluded stuff...
...that would prove it does not see excludes file.

I always always get the weird ones, so sorry Harvey.

brent

this isnt pressing and I've abandoned backups the rest of the evening. no hurries/worries.

Harvey

Sorry no idea 🤷‍♂️ works for me.

brent

Harvey yeah I think you emptied the trick bag upside on that one..will experiment again. I still have Grsync🙂

Axios

brent I am by no means a restic expert never used it but was doing some surfin reading on it
and all the examples I came across your command would look like this
sudo restic backup /home/brent --exclude-file=/home/brent/excludes.txt -r /run/media/brent/backup

Just some input you know more about than I do!! (dont let me mess something up..)

brent

Axios Harvey

restic backup /home/brent --exclude-file='/home/brent/excludes' -r /run/media/brent/backup

that's my command it works now.

Axios sudo restic backup /home/brent --exclude-file=/home/brent/excludes.txt -r /run/media/brent/backup

^^ wouldn't run with the .txt extension in command. it was a good guess by harvey, and axios I've seen a lot of sites where .txt DID work, thanks for looking into it.

Also Harvey, I did not take your ext4 advice and settled for fat32. reason: W10 OS on a seperate disk for 'in case of emergencies break glass.' Permissions was a pitfall you predicted when fat32 or win involves itself with my backup. Restic demands sudo in fat format.

Lastly I figured it out. I kept bitc*ing that the giant .config file was always being copied. The big .config was NEVER written into the exclude list by me at any point....am so tired at end of night when I try to figure this stuff out....probably thinking .cache or something...COLLOSAL BRAIN FART and I'm sorry I wasted your time @Harvey since yesterday at this time.

Harvey

I just noticed you're both running the command with sudo this is not needed, backup location is to /run/media/username/* which means you have full access and you are backing up your home directory which you obviously have access to. But I don't see how that would affect the reading of the exclude-file so probably not the solution to whatever is going on here.

The only other differences with axios example is not using a ' quote for the exclude-file path. I use them in my alias so I know it works and the file name of excludes.txt which of course must match the name and path of where that file is saved.

If you are running this command from an alias and you mixed up using single quotes and double quotes the command will break early. But I would expect that would break entirely not just break exclude-file.

To explain what I mean about mixing up quotes. In normal English (I think US is different) quotes are double quotes " quotes within quotes are singular '. In computing it tends to accept either one but you must be careful to alternate correctly. So if in the alias you had used double or singular quotes everywhere or do not use them correctly in the alias it will break, but it should break entirely based on the commands syntax not just ignore exclude-file so I don't think that is the answer either.

brent

Harvey Forget all snapshots except the last 8 (keep as many as you want). Might seem confusing but if something is forgotten it doesn't actually reclaim the space on your drive, for that you need --prune which will delete all unreferenced data (data from the forgotten snapshots).

alias resticprune="restic -r /mnt/restic2TB/Backup/Home forget --keep-last 8 --prune

This makes sure you have a valid restic backup and is recommended to be run after a prune.

alias resticcheck="restic -r /mnt/restic2TB/Backup/Home check

Hi Harvey, I'm doing my third backup since I started this thread/starting using Restic.
By "last 8 snapshots" do you mean the last 8 backups?

The new backup took about 6 minutes and produced a single snapshot, so it had to rely on the TONS of data from the last 2, I assume. In other words tonight's single snapshot and single binary produced are part of/dependent on the dozens of existing binaries from the last 2 backups? In other words, too soon to prune?

thanks for any info and sorry to revive this one.

Harvey

brent

Not entirely sure I understand the question but I'll try explain anyway.

A backup and a snapshot can be thought of as the same thing.

First your initial backup / snapshot takes much much longer than subsequent backups as they only backup files that are new or that have changed.

Scenario
If you backup every day and forget and prune every day to make sure you only keep the last 14 snapshots. You have 14 restore points at any given time.

Week 1: Mon (1) Tue (2) Wed (3) Thu (4) Fri (5) Sat (6) Sun (7)
Week 2: Mon (8) Tue (9) Wed (10) Thu (11) Fri (12) Sat (13) Sun (14)
Week 3: Mon (15 del 1) Tue (16 del 2) Wed (17 del 3) Thu (18 del 4) Fri (19 del 5) Sat (20 del 6) Sun (21 del 7)

If you accidentally delete a file on week 1 Wednesday and week 3 Monday you realise it, you can still restore the file from w1 tuesday snapshot because it has not been deleted yet.

Where many get confused is in the above scenario you think week 1 Monday was my initial backup / snapshot. Week 3 Monday forget and purge would delete that initial backup losing a LOT of data as each subsequent snapshot only contains the changes from that initial backup.

This is not the case while it will only backup new and modified files every snapshot, it keeps track of what still exists on your system. A snapshot is a point in time, not just the files it backs up that day to create the new snapshot.

While week 1 Mon was your initial backup and week 3 Sun might only back up a single text file. The snapshot may still contain files from week 1 mon despite the initial snapshot being long gone.

Forget and prune just limits how far back in time you can go, nothing more. Make sense?

brent

Harvey Forget and prune just limits how far back in time you can go, nothing more. Make sense?

Absolutely, this is what I needed and I appreciate your time.

palto42 Hi, I've started using restic quite recently as my secondary backup tool (maybe I'm a bit paranoid ;-)

My other backup tool was Rsync so your paranoia is founded....🙂

palto42

Hi, I've started using restic quite recently as my secondary backup tool (maybe I'm a bit paranoid ;-) , my other backup tool is borg which uses a similar deduplicating backup method.
For both tools I use a wrapper to make the usage easier. For restic you can use runrestic and for borg there is borgmatic. The main advantage of the wrapper tools is that you can put all important settings in a config file which is easier to manage/understand than direct CLI. runrestic also provides you a nice --shell option which just starts a a new shell with all your settings loads loaded, so that you can run the restic CLI commands easily without specifying the whole long options list.

« Previous Page