Using Amazon S3 for backups under Linux
Dear Lazyweb,
Amazon's S3 services look very interesting for doing backups, but I can't find a perfect solution to integreate it into our current setup (using either rdiffbackup or rsnapshot).
Did anyone of you ever used it in a similar way, and if so, how?
My basic idea was:
1. mount S3 (via FUSE?)
2. mount some encrypted FS on top of it (EncFS?)
3. run rdiffbackup/rsnapshot as usual on the resulting encrypted bucket
Any suggestions on tools how to do this? Are s3fs and EncFS what I'm looking for?
Comments
On Windows you can try <a href="http://cloudberrylab.com">CloudBerry S3 Explorer</a> freeware and <a href="http://cloudberrydrive.com">CloudBerry S3 Backup</a>
Well, Windows, that’s not Linux.
And I feel somehow very uncomfortable when installing Windows to do backups of a Linux system :)
have a look at duplicity (http://duplicity.nongnu.org), it already supports S3, encrypted and incremental backups.
I set up encfs on s3fs not more than 12 hours ago, and it works fine. I’m using it to hold my music collection and stream via MPD, though, not do backups. I think encfs+s3fs+rdiffbackup could work, but take a look at the discussion thread on the s3fs google page- there’s some talk about s3fs blocking IO while it uploads to S3. For simple backups, and not file storage, I would echo mru’s suggestion of Duplicity. It encrypts everything with GPG, and supports S3 directly. I’d be interested to see BoxBackup writing to an S3 bucket, but I’d expect the same blocking IO problem could show up.
Have a look at tarsnap, http://www.tarsnap.com/.
Also have a look at duplicity, which works pretty well for encrypted rsync-like incremental backup and offers an S3 mode.
Hmm, what I don’t like at duplicity, is that one needs duplicity to restore from the backup.
The nice with a mounted FS is, one can even share it on the network via NFS or Samba, so users can just open their filemanager and copy the file they need from the backup.
I’ll have a look at the discussion and test both solutions during next week.
PS the data is about 150G of stuff, not changing too much (when a new customer comes, his information, documents etc is stored, making a new folder with maybe 20-30MB)
Mhhh, anyone tried s3backer? http://code.google.com/p/s3backer/
Seems to export a loopback device on top of which one can do anything you want.
Mate,
Have you heard of Jungledisk?
http://www.jungledisk.com/
Not completely free but extremely cheap. You can do all that fancy encryption stuff on top of it. Works fine with rdiff-backup in my experience.
Cheers,
Josh