|
The old backup thread has gone to archives, but backing up your data is still important! Why do you need backups? - Data are deleted and overwritten by stupid mistakes, backups prevent those from becoming disasters. - Data are destroyed by malicious intent, be it a disgruntled employee, or your great-grandmother clicking an email-attachment with a crypto virus. - Hardware fails, taking the data with it. - Hardware is destroyed or lost by fire, earthquake, theft, etc. What is a backup? You need data stored in more than one copy, the copies need to be separated so changes to one copy do not automatically transfer to other copies, and the backup needs to be restorable so you can actually get your data back when necessary. It's generally also a good idea to encrypt the backups, if the data is important, so theft is less of a risk. And you generally want to keep geographically separated copies so a disaster at one location doesn't prevent you from restoring. RAID IS NOT A BACKUP! Using RAID for storage in a set of harddisks can be fine for higher availability in case of hardware failure, but it fails the requirement that changes to one copy do not propagate to other copies, and does not protect against physical incidents (like fire, electrical damage, etc.) either. Always keep a real backup too. CLOUD SYNC SERVICES ARE NOT BACKUP! (Dropbox, OneDrive, and the like.) They have the same problem that changes synchronize, and it's often possible for an attacker to intentionally destroy your online data. It typically also only syncs data under a specific folder, not anything you choose. YOUR BACKUP DOES NOT WORK UNTIL YOU HAVE TESTED IT CAN RESTORE! Have a plan for how often you test your restore procedure. Make sure you can get access to your backup, retrieve a file from it, and the file contains the correct data. Don't just test once. Keep testing. Your media might go bad, someone might have destroyed the data, or maybe your backup procedure is failing silently. Cloud or local? - Cloud backup, i.e. sending your data to a third-party hosting service, is easy to get in to, can usually run with very little oversight, and can be very cheap depending on amount of data. The most obvious disadvantage is that it requires a good upload rate on your internet connection to not take forever. - Local backup, to disk drives or to tape, requires up-front investment, more configuration, and manual janitoring in monitoring the media and keeping the media safe. Advantage is typically higher transfer speeds, and direct control over physical security of the backed up data. Also not handing over data to a third party, even encrypted. Do not use SSD for offline backups. Flash memory cells degrade slowly when not actively maintained. Leaving an SSD without power for months can have a significant risk of data becoming unreadable. Magnetic harddrives are the best choice. Tape can be an option if you have huge amounts of data and can afford the cost of entry. Recommended services In the old thread, Duplicati was recommended often as a backup client, capable of many different targets. Especially pairing it with Blackblaze B2 cloud storage was suggested. In this setup, the software is free, and you pay per gigabyte per month of data backed up. Other recommendations welcome! ---- My own question: I'd like to have offline backups at home, and imagine having a set of external harddrives I rotate between backing up to at home and keeping safe in my locker at work. E.g. rotating every Monday. Does anyone have experience with this kind of setup? Is it a good/bad idea and are there any pitfalls? For external harddrives used in a multiple backup-sets setup like this, would cheap USB 2.5" drives be okay, or smarter to use a dock or the like and use 3.5" internal desktop drives? nielsm fucked around with this message at 16:47 on Sep 25, 2018 |
#
?
Sep 25, 2018 16:29
|
|
|
|
| # ? Apr 25, 2024 21:05 |
|
|
Backup question: We currently use Backup Exec "Not for resale edition" (seriously, that's what it says on the installer disc) to back up 6-7 physical servers to tape. We are retiring servers and will soon have just 2 physical servers, each running 2 HyperV VMs and are looking to ditch Backup Exec. I'm not that familiar with backup of VMs vs. physical, differences if any, etc. but I'm looking to learn. What are some products and strategies I should be looking at? Thanks.
|
|
#
?
Sep 27, 2018 23:07
|
|
|
Look at Azure Backup which uses DPM to backup Hyper-V hosts. You pay per month based on the size of your instances, but you don’t have to use cloud storage if you don’t want to - it will happily back up locally. You just need to run it on a box with enough disk space.
|
|
#
?
Sep 28, 2018 03:09
|
|
|
Mr. Clark2 posted:Backup question: Veeam is shockingly useful for this exact thing. The completely free version does disk to tape and disk to disk to tape backups. I use it for backing up all my personal poo poo to tape, and once you get it set up right, it's pretty painless to deal with. Just make sure your SQL database isn't on C, because backing up millions of files makes the database bloat up amazingly fast.
|
|
#
?
Oct 2, 2018 18:45
|
|
|
I know literally nothing about tape backups, but I need a cheap way to backup ~50tb of data, is that a good option?
|
|
#
?
Oct 6, 2018 03:09
|
|
|
KOTEX GOD OF BLOOD posted:I know literally nothing about tape backups, but I need a cheap way to backup ~50tb of data, is that a good option? Run the numbers on tape (auto loader, having to manage replacing the tapes and drives and migrating to a new LTO spec when the old one is deprecated, getting those tapes off-site) vs. an AWS Storage Gateway and Glacier. This only works if your internet connection is up to it.
|
|
#
?
Oct 6, 2018 12:40
|
|
|
nielsm posted:Recommended services I've seen Arq come highly recommended as an alternative to Duplicati - seems to do mostly the same thing but is commercial ($50) with support. Seems a good bit more user friendly than Duplicati (which can involve a fair bit of computer janitoring). As far as cloud hosting for backups goes, I found Backblaze B2 to be prohibitively slow from Europe - we're talking uploads at like 150-200 kB/s here IIRC, and I'm on a symmetric gigabit connection. After crunching some numbers I went for Google Cloud Storage instead with the "coldline" storage tier. It's more expensive than Backblaze but uploading at 15 rather than 0.15 MB/s is worth it to me.
|
|
#
?
Oct 6, 2018 12:54
|
|
|
I guess I can help with this, I'm responsible for backups of a shitload of data. Pretty much exclusively Veeam. I use Macrium Reflect at home, though, it's just more suitable for home use. HalloKitty fucked around with this message at 10:27 on Oct 7, 2018 |
|
#
?
Oct 6, 2018 18:05
|
|
|
TheFluff posted:I've seen Arq come highly recommended as an alternative to Duplicati - seems to do mostly the same thing but is commercial ($50) with support. Seems a good bit more user friendly than Duplicati (which can involve a fair bit of computer janitoring). I use b2. For some reason they cap per-thread upload speed. You need to add threads until your uplink is saturated
|
|
#
?
Oct 11, 2018 02:33
|
|
|
Does anyone have experience with UrBackup for backing up one computer to another computer over a LAN (like what the free version of CrashPlan used to do)? Opinions? Any other recommendations for software that can backup computers on a LAN?
|
|
#
?
Oct 27, 2018 22:38
|
|
|
Splinter posted:Does anyone have experience with UrBackup for backing up one computer to another computer over a LAN (like what the free version of CrashPlan used to do)? Opinions? Any other recommendations for software that can backup computers on a LAN? Duplicati can back up to whatever you like but requires some setup on the receiving end - either a shared folder or an FTP server or whatever you like.
|
|
#
?
Oct 28, 2018 22:55
|
|
|
We use Veeam at work to replicate VMs from one site to the other. I don't administer Veeam and the guy who does keeps saying he'll check and never does. The question is can Veeam handle replicating a VM with Raw Device Mapped drives? E: I realize not strictly backup related, but I figured people might be familiar with Veeam here myron cope fucked around with this message at 02:14 on Oct 29, 2018 |
|
#
?
Oct 29, 2018 02:05
|
|
|
myron cope posted:We use Veeam at work to replicate VMs from one site to the other. I don't administer Veeam and the guy who does keeps saying he'll check and never does. The question is can Veeam handle replicating a VM with Raw Device Mapped drives? Load the Veeam agent on the VM itself, then do a file level backup? I'm not sure if it's able to more elegantly handle them.
|
|
#
?
Oct 30, 2018 18:41
|
|
|
Thanks Ants posted:Look at Azure Backup which uses DPM to backup Hyper-V hosts. You pay per month based on the size of your instances, but you don’t have to use cloud storage if you don’t want to - it will happily back up locally. You just need to run it on a box with enough disk space. DPM is, at best, alright. It tends to break pretty badly when it decides to poo poo itself, too and the UI sucks. Also requires a System Center license, but if you just want to back up to Azure you can use Azure Backup Server which is literally the same codebase but with the System Center poo poo taken out of it.
|
|
#
?
Oct 31, 2018 00:15
|
|
|
I've got a heap of archived data that the company won't let me delete for various reasons, but I'd like to get it off of live storage so I don't have to keep expanding "expensive" storage and don't have to keep backing it up locally. I'm thinking of dumping it all onto S3, and I'm wondering what people do for backups of this type of data. It's not critical enough that I have to worry about redundancy beyond the default 3 AZ's, the only thing I'd be protecting against is someone (me) accidentally pressing the wrong thing and deleting/overwriting something. Would S3 versioning be enough for that use case, or should I do something more?
|
|
#
?
Oct 31, 2018 17:03
|
|
|
If you hope the archive will be write-only, and never need to fetch data from it under normal circumstances, Amazon Glacier is likely more appropriate. It's intended for backup and archive usage where you'll rarely need to fetch data, and it's okay to have a delay measured in hours between request and delivery.
|
|
#
?
Oct 31, 2018 17:40
|
|
|
Does Glacier protect against someone fat fingering something and wiping a directory? The glacier lock write-once thing is overkill, I'm hoping that at some point I will actually be allowed to delete the stuff.
|
|
#
?
Oct 31, 2018 17:55
|
|
|
Is there any cloud backup provider that (a) has a linux cli app (b) does incremental backups I need to backup two linux servers and I've been looking for over a year and haven't found anything. Googling shows TONS of providers but they all seem to have significant drawbacks. I've probably evaluated 10 providers and have been unsuccessful.
|
|
#
?
Oct 31, 2018 21:54
|
|
|
revmoo posted:Is there any cloud backup provider that Duplicati runs on Linux, its GUI is a frontend for the commandline interface, and supports incremental backups. It's just the backup software though, the cloud storage part you need to sort out yourself. It supports a whole bunch of providers out of the box so just go whichever cloud service you like - Backblaze B2, Amazon S3 and Google Cloud Storage are obvious candidates. If you want a fully integrated solution there's Tarsnap, I guess, but their storage rates are very expensive. TheFluff fucked around with this message at 22:27 on Oct 31, 2018 |
|
#
?
Oct 31, 2018 22:02
|
|
|
I tried duplicati, it's not even alpha status. I do not trust it with my data. I also evaluated tarsnap but abandoned it. I can't remember why so I'll give it another look.
|
|
#
?
Oct 31, 2018 22:11
|
|
|
myron cope posted:We use Veeam at work to replicate VMs from one site to the other. I don't administer Veeam and the guy who does keeps saying he'll check and never does. The question is can Veeam handle replicating a VM with Raw Device Mapped drives? If they are virtual RDMs, the answer is "yes with some caveats." There's a bunch more info here - https://www.veeam.com/blog/rdms-explained-for-veeam-backup-replication.html Realistically, unless you have some really good reason to use RDMs then you should probably convert off of them.
|
|
#
?
Oct 31, 2018 23:59
|
|
|
So I was trying out duplicati which seems cool but I'm not sure is right for me. I'm not doing cloud backups so it might be kinda overkill. I do like the part about incremental backup with multiple versions of files kept. But this part is kinda annoying: nielsm posted:YOUR BACKUP DOES NOT WORK UNTIL YOU HAVE TESTED IT CAN RESTORE! Have a plan for how often you test your restore procedure. Make sure you can get access to your backup, retrieve a file from it, and the file contains the correct data. Don't just test once. Keep testing. Your media might go bad, someone might have destroyed the data, or maybe your backup procedure is failing silently. Does anyone have some backup software they like that does both incremental history and a plain old files that'll be easy to restore from years from now? I can janitor real good, so an obtuse UI doesn't frighten me.
|
|
#
?
Dec 8, 2018 05:23
|
|
|
I believe UrBackup does what you're looking for.
|
|
#
?
Dec 8, 2018 08:30
|
|
|
https://www.duplicati.com/articles/FactSheet/quote:Basic: Online Backup Verification
|
|
#
?
Dec 8, 2018 09:29
|
|
|
Glad to see the backup thread is back up. I'll see myself out.
|
|
#
?
Jan 23, 2019 15:43
|
|
|
Since this thread got bumped I'll post my trip report: UrBackup is a neat program, and if I had a NAS I would be using that. But while the idea of an always-on automated backup program is a great one, it just doesn't particularly fit with oldschool "plug in a USB drive and run a backup" method. When the process starts with taking the HD out of a drawer and plugging it in, the software might as well be manual as well. So instead I spent an afternoon playing with robocopy and figuring out all the interesting switches it has. I came up with a neat trick that I'm kinda proud of, using the /CREATE switch (which makes zero-length files and is very fast even on thousands of files) to produce fully incremental backups. The main backup directory contains the current mirror, and all changed or deleted files get moved into dated archives. That's more or less exactly how I've always wanted my backups to work. Recovery will be easy if I have data loss, but also I have a preservation of old stuff. If anyone is interested, here's a shortened version of what I made: code:
|
|
#
?
Jan 23, 2019 22:12
|
|
|
Hi thread, I just found this, and I have a backup problem. Please help me! It's a little complicated, at least in my head, so I'll try to dump whatever I can. We're moving from EMC Networker to Veeam, and we have a third-party DBA that does all of the MS SQL work and takes backups via SQL directly and copies them off for us to backup through other means. Before, Networker was simply set to exclude any SQL-related files, and back up everything else, which seemed to work just fine. Now, with Veeam, it's doing a whole lot more stuff that we're trying to either get it to stop doing, or explain to the DBAs that this is how it works and it's not actually breaking the SQL backups. Note that these are physical servers, not VMs. We have the agents set to not do application aware backups, so it shouldn't be messing with SQL. We're seeing Veeam stop VSS writers and pause SQL for the backups, but we're also seeing Veeam try to log into SQL server with the veeam backup user account, which fails because it's not an allowed account in SQL server, so it then tries to do it via NT AUTHORITY\SYSTEM, which succeeds and it takes a backup? I guess? of a few of the DBs (maybe just the master DB) then exits. It seems to be doing this via a WITH COPY_ONLY option which seems to not break the backup chains, but it's a real pain to have to deal with for the DBA so they don't have to verify their own backups, make sure logs weren't truncated, etc. Their responsibility is to make sure the SQL backups are good and can be restored, while our responsibility is to get the server, OS, etc back up and running. My question is, can we get Veeam to stop messing with SQL? It's fine if it needs to stop VSS to do its thing, but logging into SQL is causing all kinds of headaches for everyone. I just want to back up the C: drive and some random files on other volumes, that may or may not have SQL DBs on it as well. All the looking I've done has primarily how to deal with Veeam and SQL when you want it to back up, but I haven't been able to find much for if you don't want it to touch SQL.
|
|
#
?
Jan 25, 2019 21:34
|
|
|
Tell Veeam not to do Application Aware Processing and it'll stop doing anything more than VSS pauses. Without Application Aware Processing, Veeam will take crash-consistent image backups of the machine using VSS, which seems to be all you want. I have many times brought up Veeam image-based backups of MSSQL, Oracle, and other databases lacking application aware processing into prod or validation environments without a hitch. Worst I've had to do is rerun an archive redo log after restoration when an oracle DB lost power in unfavorable conditions. Potato Salad fucked around with this message at 06:41 on Jan 26, 2019 |
|
#
?
Jan 26, 2019 06:33
|
|
|
Potato Salad posted:Tell Veeam not to do Application Aware Processing and it'll stop doing anything more than VSS pauses. Yeah, App aware processing is already turned off, but it's still screwing around with SQL. Here's a log from SQL side: code:Panthrax fucked around with this message at 21:18 on Jan 26, 2019 |
|
#
?
Jan 26, 2019 21:13
|
|
|
Panthrax posted:Yeah, App aware processing is already turned off, but it's still screwing around with SQL. There's a registry key that's utterly undocumented that allows you to entirely exclude an SQL instance, but I'm at home, and I have it noted down at work... I have no idea if it works with the agent, though... I know it works when VBR takes a VM backup. If I actually remember about this thread on Monday or whatever, I'll post it.
|
|
#
?
Jan 26, 2019 22:18
|
|
|
Just want to note that Carbonite on MacOS does not appear to work from Hong Kong or Japan, I had to VPN in to get the backup updating 
|
|
#
?
Jan 27, 2019 10:59
|
|
|
Panthrax posted:Yeah, App aware processing is already turned off, but it's still screwing around with SQL. Right, I actually did remember to reply with something useful. Assuming we are talking about VBR backups (again, it might work with an agent, but I haven't tried), you need to create a registry key on the server to be backed up: HKEY_LOCAL_MACHINE\SOFTWARE\Veeam\Veeam Backup and Replication ExcludeSQLInstances As a Multi-String Value Then simply write the exact name of every SQL instance you want to exclude, each on a newline inside the multi-string value. Hope this helps.
|
|
#
?
Jan 28, 2019 20:31
|
|
|
Could someone check or post their SQL logs for when Veeam takes a backup of a volume with both flat files that we want backed up (.bak, other business files) and live SQL files that we don't (.mdf and .ldf files) with app aware processing turned off? Veeam support tells me that in order for VSS to stop showing up in SQL logs, we need to remove the files we want to back up and put them on a separate drive than the SQL files, then it'll stop. However unfortunately I'm not the one talking to Veeam, and I can't tell if they're getting the whole logging into the DB thing confused with just VSS, or if we all know what we're talking about, or what.
|
|
#
?
Jan 28, 2019 21:54
|
|
|
HalloKitty posted:Right, I actually did remember to reply with something useful. Thanks, I'll shoot this over to the guy handling the backups to test. For this we're specifically concerned about agent jobs on physical machines, so we'll definitely need to test it. Thanks for the info!
|
|
#
?
Jan 28, 2019 21:57
|
|
|
Posting my set-up for a quick sanity check... I have a Windows 10 desktop machine (SSD boot drive plus a 2TB internal drive for storage) that I want to back up. I only really care about family photos, documents and such so my data footprint I need to protect is pretty small right now. No large Blu-ray rips or anything else that isn't easily replaceable. On the recommendation of some co-workers I bought Acronis for back up service ($69/yr iirc). The program seems ok... It does a full backup image of the system to a 6TB NAS drive locally and a cloud back-up of <500GB of multimedia/documents only. These are done weekly, and the software allows creation of rescue media etc. Does this set up make sense? I'm coming up on a cloud data limit which will bump me to $99/year, and I am considering switching backup software to something like Duplicati/Backblaze. I like that Acronis is pretty hands off and runs in the background, but I wanted to see if there's anything better out there. egyptian rat race fucked around with this message at 14:07 on Mar 14, 2019 |
|
#
?
Mar 14, 2019 14:04
|
|
|
Frankly, for $99/year, look at getting Office 365 Family. Five people, many many devices, 1TB cloud space per person in OneDrive. It sounds like you care more about the photos/small files than the OS itself, so Acronis works but it's possibly focused on the wrong thing.
|
|
#
?
Mar 18, 2019 00:41
|
|
|
Is there a good way to deal with backups of one large volume to multiple smaller drives, with regards specifically to figuring out how to replace the contents of one of those backup drives should the need arise? Let me explain: When I first started my Plex server I had media on two HDDs, backed up to 2 more drives of the same capacity, but I got tired of dealing with that 4-drive arrangement. I put all my media on a single 6 TB drive, and used two of the original 3 TB HDDs as backups, but I did this by dumping files on the first drive until it was full, then continuing with the second drive. So the first drive was full and in storage, and I was only ever dealing with the main drive and the current open backup. I realized that if I had to replace the 6 TB drive it would be straightforward by copying from the backups, but if I had to replace one of the older 3 TB drives there would be no good way to determine what files belonged on that drive from the original library, or at least no way that I know of. So is there a way to do what I'm asking here, to use some software to look at the source drive, plus backup drive A, and determine the files not on A that need to go from the source to the new drive B? I replaced those 3 TB drives with a single 6 TB HDD for the backup so the issue isn't nagging at the moment, but I'm going to have to upgrade capacity again in the next year or so. I have tons of older, lower-capacity drives that could be conveniently be repurposed as backups if I could figure out how to make it easier to manage their contents. Any ideas? nielsm posted:My own question: Since nobody addressed this let me offer my thoughts. It sounds like what you've proposed makes sense, but I'd be slightly concerned about eventual damage to one of the rotated drives; hauling drives back and forth every week increases their exposure to shock (both physical, and electrical.) Other than that the only issue I can think of is theft; I'd suggest encrypting the contents just in case. 3.5" drives are cheaper per-capacity than 2.5" drives and are faster too, but the latter are easier to work with (they can run off 5 V and a single USB connector with no additional power supply.) For internal drives, you can get a 2.5" 2 TB for <$100 or a 5 TB 15 mm height (which won't fit in a laptop or regular enclosure) for >$150; 3.5" drive pricing is considered "good" at $20/TB or better. You can also get external drives (i.e. ones that come in their own enclosure and can usually be "shucked" but aren't intended to be) for better prices than internal ones, often: there are 4 TB portable drives from WD, Seagate, for ~$100 (these contain the 15 mm height drives); desktop (3.5", with external PSU) 6 TB drives are often $100, 8 TB as low as ~$120 with sales, and 10 TB for as low as ~$160 (typically the ~monthly sales at BB.) So, depending on your budget and the amount of data you need to backup, I might go with the aforementioned 4 TB portable USB drives for $100. If you have tens of TB to backup however, the way to go would be reasonably-priced 3.5" drives and a heavy-duty carrying case like this. Just note as above, if you're dealing with bare drives that may increase the potential for handling issues, drops (you have to deal with inserting & removing them from an enclosure repeatedly in addition to the normal transportation concerns - also, the SATA connectors aren't rated for a ton of cycles AFAIK) in addition to ESD-related damage.
|
|
#
?
Mar 20, 2019 05:13
|
|
|
I use a pool of smaller drives as a backup target, using mergerfs to make them one big volume while keeping the filesystem available from each individual drive. If you go that way and lose a drive, rsync would generate the list of missing files for you without confusion.
|
|
#
?
Mar 20, 2019 14:48
|
|
|
Anyone have suggestions for backing up a small amount of files (less than 50GB at the moment) and a larger amount of photos and videos (3+ TB at the moment). Right now I have the files stored on Dropbox, but would like to combine everything into one solution (and get my photos backed up online) Currently I store the photos on a local HDD in my computer, and copied on my Synology NAS and would prefer something that can sync through Cloyd Sync. I have been looking at Amazon Cloud as I already have a Prime account and can store unlimited photos free. Any other suggestions?
|
|
#
?
May 13, 2019 20:25
|
|
|
|
| # ? Apr 25, 2024 21:05 |
|
|
refleks posted:Anyone have suggestions for backing up a small amount of files (less than 50GB at the moment) and a larger amount of photos and videos (3+ TB at the moment). Right now I have the files stored on Dropbox, but would like to combine everything into one solution (and get my photos backed up online) Backblaze B2? I don't have any experience with them, but I heard good things about them, and I'm going to try using them as a backup for my photos.
|
|
#
?
May 13, 2019 21:36
|
|
