Software for Syncing files between storages (Linux)

Syncthing? git-annex?

rsync, SyncThing, or Nextcloud. I use all three, often on the same machine at the same time, for various purposes.

rsync is great for scheduled syncs (but intentionally not TOO frequent -- where blowing away one copy would almost instantly blow away the other copy could be a concern.) and outstanding for one-off syncs. With the right command line options it can even create a rough approximation of snapshots with copy-on-write rolling incremental backups. I believe this is the original documentation for that particular trick although it has spawned many genuine backup tools which operate very similarly using rsync under the hood, like rsnapshot.

SyncThing is great for large, intensive datasets that you want to keep perfectly synchronized across multiple computers at the same time. One example is my saved games, so without relying on Steam "cloud saves" or anything, I can just save my game and hop from gaming on one computer onto my laptop and pick up right where I left off.

Nextcloud is what I use for keeping separate copies of all my documents, photos from my phone, etc. It's easier than SyncThing to setup and manage in my opinion, it's simple and reliable and reasonably user-friendly in most use-cases, and seems to be more compatible with a lot of software due to its similar design to something like Dropbox. It isn't ideal for handling huge numbers of files or very large folders in my personal opinion, but it does work if you want to try it.

man rsync

I do something like that with rsync. The syntax is a bit tricky to figure out at first, but the idea is that you take a known-good copy of your data and run rsync to copy it to the removable drive. After the first sync, you can run the command to check for any changes and push the changes from your main data to the copy on the drive.

You'll want to research the command and the options, but you can theoretically run something like: rsync -avP --delete /mnt/Data_Drive/Data/ /run/media/user/Backup/Data/ where a is archive mode which helps to do things like preserve file attributes, v is verbose so you can see more details when you run the command, and P gives a progress line and keeps any partial files on the receiving end if the transfer is interrupted for some reason. --delete removes any files on the receiving end that have been deleted from your main data folder since the last time you've run rsync. The mnt folder path can be replaced with wherever your main data backup lives and the /run folder is a hypothetical destination folder on a removable drive.

An additional trick is to run the command with -n at the end of it before you run it without -n at the end. -n performs a 'dry run' where it will output what the changes are going to be when you actually run the sync operation. This can help you spot issues like mistyping the file path in your command. If you're only syncing one or two changes and the dry run indicates that it is going to basically re-transfer your entire data folder, then you probably got something wrong.

Another fun thing about rsync is that if you've got ssh server set up on your home computers, you can run rsync to sync files from one computer to another one on your network.

I have bash scripts of the form rsync -av source dest for this. Make it -avz if copying over a network.

Syncthing is the closest to what you want.

I rsync everything with ssh keys.

Really easy to do.

So I recently quitted Google Drive

Congrats, great life choice

downloaded all my data and hoping to keep it on Hard drives myself, They are external plug in types

You have multiple external drives. Are they USB? How many?

So when I plug it in

Are they not usually plugged in? Is there a requirement to keep them air gapped between syncs?

it copies/syncs the things on my computer, anyone knows somethign like that?

Like others have said, a shell script that calls rsync is maybe the way to go and with some more information on number of drives, drive interface type, and air gap requirements it may change

@Nickelalloy Maybe Back In Time is a possible solution.
Its using rsync and it is possible to be scheduled.

https://github.com/bit-team/backintime

Rsync.

You could leave the drive plugged in and schedule a daily/hourly sync, or you could have the system trigger a sync whenever you plug the drive in.

So when I plug it in it copies/syncs the things on my computer, anyone knows somethign like that?

So you want to sync files from a drive whenever you plug it in to the computer? Sounds like a udev + rsync solution. Udev mounts the drive and runs rsync.

I use freefilesync. You can tailor what is copied where and save it to an executable batch file. If you want to trigger the batch file when your external drive is plugged in you can create a udev rule. Best is you look that up yourself, since it heavily depends on your system (no chatgpt tho!).

Be aware that if you have a single machine syncthing won't work.

Use any software that can do software merging, like meld, it does the job

Kinda depends on exactly how much you want it to sync, e.g. should it truly sync all files from the synced volume to all clients, or do you want the files to only sync/download to clients when requested, keeping a "master copy" on a server?

For me, I use Syncthing for a simple setup where I want multiple machines to have a few different shared folders that sync with each other, all files updated on-disk when connected to a network.

For a more Google Drive-style approach with only fetching/syncing files onto client disk when explicitly chosen, I believe Nextcloud has what you need, but I've not used it on any of my own machines. It can be self-hosted or paid for, depending on what you prefer.