Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us $3,400 per month for bandwidth bills alone, and since we don't believe in shoving popup ads to our registered users, we try to make the money back through forum registrations.
«2 »
  • Post
  • Reply
nielsm
Jun 1, 2009




Fallen Rib

The old backup thread has gone to archives, but backing up your data is still important!

Why do you need backups?
- Data are deleted and overwritten by stupid mistakes, backups prevent those from becoming disasters.
- Data are destroyed by malicious intent, be it a disgruntled employee, or your great-grandmother clicking an email-attachment with a crypto virus.
- Hardware fails, taking the data with it.
- Hardware is destroyed or lost by fire, earthquake, theft, etc.

What is a backup?
You need data stored in more than one copy, the copies need to be separated so changes to one copy do not automatically transfer to other copies, and the backup needs to be restorable so you can actually get your data back when necessary.
It's generally also a good idea to encrypt the backups, if the data is important, so theft is less of a risk. And you generally want to keep geographically separated copies so a disaster at one location doesn't prevent you from restoring.

RAID IS NOT A BACKUP! Using RAID for storage in a set of harddisks can be fine for higher availability in case of hardware failure, but it fails the requirement that changes to one copy do not propagate to other copies, and does not protect against physical incidents (like fire, electrical damage, etc.) either. Always keep a real backup too.

CLOUD SYNC SERVICES ARE NOT BACKUP! (Dropbox, OneDrive, and the like.) They have the same problem that changes synchronize, and it's often possible for an attacker to intentionally destroy your online data. It typically also only syncs data under a specific folder, not anything you choose.

YOUR BACKUP DOES NOT WORK UNTIL YOU HAVE TESTED IT CAN RESTORE! Have a plan for how often you test your restore procedure. Make sure you can get access to your backup, retrieve a file from it, and the file contains the correct data. Don't just test once. Keep testing. Your media might go bad, someone might have destroyed the data, or maybe your backup procedure is failing silently.

Cloud or local?
- Cloud backup, i.e. sending your data to a third-party hosting service, is easy to get in to, can usually run with very little oversight, and can be very cheap depending on amount of data. The most obvious disadvantage is that it requires a good upload rate on your internet connection to not take forever.
- Local backup, to disk drives or to tape, requires up-front investment, more configuration, and manual janitoring in monitoring the media and keeping the media safe. Advantage is typically higher transfer speeds, and direct control over physical security of the backed up data. Also not handing over data to a third party, even encrypted.

Do not use SSD for offline backups. Flash memory cells degrade slowly when not actively maintained. Leaving an SSD without power for months can have a significant risk of data becoming unreadable. Magnetic harddrives are the best choice. Tape can be an option if you have huge amounts of data and can afford the cost of entry.

Recommended services
In the old thread, Duplicati was recommended often as a backup client, capable of many different targets. Especially pairing it with Blackblaze B2 cloud storage was suggested. In this setup, the software is free, and you pay per gigabyte per month of data backed up.
Other recommendations welcome!

----

My own question:
I'd like to have offline backups at home, and imagine having a set of external harddrives I rotate between backing up to at home and keeping safe in my locker at work. E.g. rotating every Monday. Does anyone have experience with this kind of setup? Is it a good/bad idea and are there any pitfalls?
For external harddrives used in a multiple backup-sets setup like this, would cheap USB 2.5" drives be okay, or smarter to use a dock or the like and use 3.5" internal desktop drives?

nielsm fucked around with this message at 15:47 on Sep 25, 2018

Adbot
ADBOT LOVES YOU

Mr. Clark2
Sep 17, 2003

Rocco sez: Oh man, what a bummer. Woof.


Backup question:

We currently use Backup Exec "Not for resale edition" (seriously, that's what it says on the installer disc) to back up 6-7 physical servers to tape.
We are retiring servers and will soon have just 2 physical servers, each running 2 HyperV VMs and are looking to ditch Backup Exec. I'm not that familiar with backup of VMs vs. physical, differences if any, etc. but I'm looking to learn. What are some products and strategies I should be looking at? Thanks.

Thanks Ants
May 21, 2004

Bless You Ants, Blants



Fun Shoe

Look at Azure Backup which uses DPM to backup Hyper-V hosts. You pay per month based on the size of your instances, but you donít have to use cloud storage if you donít want to - it will happily back up locally. You just need to run it on a box with enough disk space.

Methylethylaldehyde
Oct 23, 2004

BAKA BAKA


Mr. Clark2 posted:

Backup question:

We currently use Backup Exec "Not for resale edition" (seriously, that's what it says on the installer disc) to back up 6-7 physical servers to tape.
We are retiring servers and will soon have just 2 physical servers, each running 2 HyperV VMs and are looking to ditch Backup Exec. I'm not that familiar with backup of VMs vs. physical, differences if any, etc. but I'm looking to learn. What are some products and strategies I should be looking at? Thanks.

Veeam is shockingly useful for this exact thing. The completely free version does disk to tape and disk to disk to tape backups. I use it for backing up all my personal poo poo to tape, and once you get it set up right, it's pretty painless to deal with. Just make sure your SQL database isn't on C, because backing up millions of files makes the database bloat up amazingly fast.

KOTEX GOD OF BLOOD
Jul 7, 2012



Fallen Rib

I know literally nothing about tape backups, but I need a cheap way to backup ~50tb of data, is that a good option?

Thanks Ants
May 21, 2004

Bless You Ants, Blants



Fun Shoe

KOTEX GOD OF BLOOD posted:

I know literally nothing about tape backups, but I need a cheap way to backup ~50tb of data, is that a good option?

Run the numbers on tape (auto loader, having to manage replacing the tapes and drives and migrating to a new LTO spec when the old one is deprecated, getting those tapes off-site) vs. an AWS Storage Gateway and Glacier. This only works if your internet connection is up to it.

TheFluff
Dec 13, 2006

FRIENDS, LISTEN TO ME
I AM A SEAGULL
OF WEALTH AND TASTE


Slippery Tilde

nielsm posted:

Recommended services
In the old thread, Duplicati was recommended often as a backup client, capable of many different targets. Especially pairing it with Blackblaze B2 cloud storage was suggested. In this setup, the software is free, and you pay per gigabyte per month of data backed up.
Other recommendations welcome!

I've seen Arq come highly recommended as an alternative to Duplicati - seems to do mostly the same thing but is commercial ($50) with support. Seems a good bit more user friendly than Duplicati (which can involve a fair bit of computer janitoring).

As far as cloud hosting for backups goes, I found Backblaze B2 to be prohibitively slow from Europe - we're talking uploads at like 150-200 kB/s here IIRC, and I'm on a symmetric gigabit connection. After crunching some numbers I went for Google Cloud Storage instead with the "coldline" storage tier. It's more expensive than Backblaze but uploading at 15 rather than 0.15 MB/s is worth it to me.

HalloKitty
Sep 30, 2005

Adjust the bass and let the Alpine blast


I guess I can help with this, I'm responsible for backups of a shitload of data.
Pretty much exclusively Veeam. I use Macrium Reflect at home, though, it's just more suitable for home use.

HalloKitty fucked around with this message at 09:27 on Oct 7, 2018

svenkatesh
Sep 5, 2016

by FactsAreUseless


TheFluff posted:

I've seen Arq come highly recommended as an alternative to Duplicati - seems to do mostly the same thing but is commercial ($50) with support. Seems a good bit more user friendly than Duplicati (which can involve a fair bit of computer janitoring).

As far as cloud hosting for backups goes, I found Backblaze B2 to be prohibitively slow from Europe - we're talking uploads at like 150-200 kB/s here IIRC, and I'm on a symmetric gigabit connection. After crunching some numbers I went for Google Cloud Storage instead with the "coldline" storage tier. It's more expensive than Backblaze but uploading at 15 rather than 0.15 MB/s is worth it to me.

I use b2. For some reason they cap per-thread upload speed. You need to add threads until your uplink is saturated

Splinter
Jul 4, 2003
Cowabunga!

Does anyone have experience with UrBackup for backing up one computer to another computer over a LAN (like what the free version of CrashPlan used to do)? Opinions? Any other recommendations for software that can backup computers on a LAN?

TheFluff
Dec 13, 2006

FRIENDS, LISTEN TO ME
I AM A SEAGULL
OF WEALTH AND TASTE


Slippery Tilde

Splinter posted:

Does anyone have experience with UrBackup for backing up one computer to another computer over a LAN (like what the free version of CrashPlan used to do)? Opinions? Any other recommendations for software that can backup computers on a LAN?

Duplicati can back up to whatever you like but requires some setup on the receiving end - either a shared folder or an FTP server or whatever you like.

myron cope
Apr 21, 2009



We use Veeam at work to replicate VMs from one site to the other. I don't administer Veeam and the guy who does keeps saying he'll check and never does. The question is can Veeam handle replicating a VM with Raw Device Mapped drives?

E: I realize not strictly backup related, but I figured people might be familiar with Veeam here

myron cope fucked around with this message at 01:14 on Oct 29, 2018

Methylethylaldehyde
Oct 23, 2004

BAKA BAKA


myron cope posted:

We use Veeam at work to replicate VMs from one site to the other. I don't administer Veeam and the guy who does keeps saying he'll check and never does. The question is can Veeam handle replicating a VM with Raw Device Mapped drives?

E: I realize not strictly backup related, but I figured people might be familiar with Veeam here

Load the Veeam agent on the VM itself, then do a file level backup? I'm not sure if it's able to more elegantly handle them.

high six
Feb 6, 2010


Thanks Ants posted:

Look at Azure Backup which uses DPM to backup Hyper-V hosts. You pay per month based on the size of your instances, but you donít have to use cloud storage if you donít want to - it will happily back up locally. You just need to run it on a box with enough disk space.

DPM is, at best, alright. It tends to break pretty badly when it decides to poo poo itself, too and the UI sucks. Also requires a System Center license, but if you just want to back up to Azure you can use Azure Backup Server which is literally the same codebase but with the System Center poo poo taken out of it.

wyoak
Feb 14, 2005

a glass case of emotion



Fallen Rib

I've got a heap of archived data that the company won't let me delete for various reasons, but I'd like to get it off of live storage so I don't have to keep expanding "expensive" storage and don't have to keep backing it up locally. I'm thinking of dumping it all onto S3, and I'm wondering what people do for backups of this type of data. It's not critical enough that I have to worry about redundancy beyond the default 3 AZ's, the only thing I'd be protecting against is someone (me) accidentally pressing the wrong thing and deleting/overwriting something. Would S3 versioning be enough for that use case, or should I do something more?

nielsm
Jun 1, 2009




Fallen Rib

If you hope the archive will be write-only, and never need to fetch data from it under normal circumstances, Amazon Glacier is likely more appropriate. It's intended for backup and archive usage where you'll rarely need to fetch data, and it's okay to have a delay measured in hours between request and delivery.

wyoak
Feb 14, 2005

a glass case of emotion



Fallen Rib

Does Glacier protect against someone fat fingering something and wiping a directory?

The glacier lock write-once thing is overkill, I'm hoping that at some point I will actually be allowed to delete the stuff.

revmoo
May 25, 2006

#basta


Is there any cloud backup provider that

(a) has a linux cli app
(b) does incremental backups

I need to backup two linux servers and I've been looking for over a year and haven't found anything. Googling shows TONS of providers but they all seem to have significant drawbacks. I've probably evaluated 10 providers and have been unsuccessful.

TheFluff
Dec 13, 2006

FRIENDS, LISTEN TO ME
I AM A SEAGULL
OF WEALTH AND TASTE


Slippery Tilde

revmoo posted:

Is there any cloud backup provider that

(a) has a linux cli app
(b) does incremental backups

I need to backup two linux servers and I've been looking for over a year and haven't found anything. Googling shows TONS of providers but they all seem to have significant drawbacks. I've probably evaluated 10 providers and have been unsuccessful.

Duplicati runs on Linux, its GUI is a frontend for the commandline interface, and supports incremental backups. It's just the backup software though, the cloud storage part you need to sort out yourself. It supports a whole bunch of providers out of the box so just go whichever cloud service you like - Backblaze B2, Amazon S3 and Google Cloud Storage are obvious candidates.

If you want a fully integrated solution there's Tarsnap, I guess, but their storage rates are very expensive.

TheFluff fucked around with this message at 21:27 on Oct 31, 2018

revmoo
May 25, 2006

#basta


I tried duplicati, it's not even alpha status. I do not trust it with my data.

I also evaluated tarsnap but abandoned it. I can't remember why so I'll give it another look.

Internet Explorer
Jun 1, 2005





Oven Wrangler

myron cope posted:

We use Veeam at work to replicate VMs from one site to the other. I don't administer Veeam and the guy who does keeps saying he'll check and never does. The question is can Veeam handle replicating a VM with Raw Device Mapped drives?

E: I realize not strictly backup related, but I figured people might be familiar with Veeam here

If they are virtual RDMs, the answer is "yes with some caveats." There's a bunch more info here - https://www.veeam.com/blog/rdms-exp...eplication.html

Realistically, unless you have some really good reason to use RDMs then you should probably convert off of them.

Klyith
Aug 3, 2007

GBS Pledge Week


So I was trying out duplicati which seems cool but I'm not sure is right for me. I'm not doing cloud backups so it might be kinda overkill. I do like the part about incremental backup with multiple versions of files kept.

But this part is kinda annoying:

nielsm posted:

YOUR BACKUP DOES NOT WORK UNTIL YOU HAVE TESTED IT CAN RESTORE! Have a plan for how often you test your restore procedure. Make sure you can get access to your backup, retrieve a file from it, and the file contains the correct data. Don't just test once. Keep testing. Your media might go bad, someone might have destroyed the data, or maybe your backup procedure is failing silently.
With a normal "copy poo poo to a backup drive" backup it's easy to test the backups. I can open some docs and play some mp3s or videos straight from the drive to randomly spot-check that it's good. A standard filesystem is reassuring. With duplicati you need to go through the restore process to test anything. And I'm just a bit suspicious of a backup method that turns everything into a bunch of mystery meat that needs a database to restore.


Does anyone have some backup software they like that does both incremental history and a plain old files that'll be easy to restore from years from now? I can janitor real good, so an obtuse UI doesn't frighten me.

Splinter
Jul 4, 2003
Cowabunga!

I believe UrBackup does what you're looking for.

nielsm
Jun 1, 2009




Fallen Rib

https://www.duplicati.com/articles/FactSheet/

quote:

Basic: Online Backup Verification

Duplicati is built to work with simple storage systems. Many providers offer compatible storages and often at cheap prices. As a downside of this, some storage system might store corrupt data. And most people usually notice that, when they need their backup to restore files they have lost and restoring fails. To avoid that Duplicati regularly downloads a random set of backup files, restores their content and checks their integrity. That way you can detect problems with your online storage before you run into troubles.
Sounds like you might not really need to do a test restore, if you trust that this feature works as advertised.

Potato Salad
Oct 23, 2014

Nobody Cares




Tortured By Flan

Glad to see the backup thread is back up.




I'll see myself out.

Klyith
Aug 3, 2007

GBS Pledge Week


Since this thread got bumped I'll post my trip report:

UrBackup is a neat program, and if I had a NAS I would be using that. But while the idea of an always-on automated backup program is a great one, it just doesn't particularly fit with oldschool "plug in a USB drive and run a backup" method. When the process starts with taking the HD out of a drawer and plugging it in, the software might as well be manual as well.

So instead I spent an afternoon playing with robocopy and figuring out all the interesting switches it has. I came up with a neat trick that I'm kinda proud of, using the /CREATE switch (which makes zero-length files and is very fast even on thousands of files) to produce fully incremental backups. The main backup directory contains the current mirror, and all changed or deleted files get moved into dated archives. That's more or less exactly how I've always wanted my backups to work. Recovery will be easy if I have data loss, but also I have a preservation of old stuff.


If anyone is interested, here's a shortened version of what I made:
code:
@echo off
set tdate_=%date:~10,4%_%date:~4,2%_%date:~7,2%
mkdir U:\%tdate_%
rem // creates empty placeholder files from everything with A-bit, unsets A-bit
echo *** Indexing changed files *** >> U:\%tdate_%\files.log
robocopy "D:\Files" "U:\%tdate_%\Files" /S /CREATE /M /XA:S /A-:RSH /NP /LOG+:U:\%tdate_%\files.log
rem // saves changed files by overwriting empty files created in previous step from the mirror
echo *** Archiving old file versions *** >> U:\%tdate_%\files.log
robocopy "U:\Mirror\Files" "U:\%tdate_%\Files" /S /MOV /XL /DCOPY:T /NP /LOG+:U:\%tdate_%\files.log /NDL
rem // main mirror copy
echo *** Mirroring the backup *** >> U:\%tdate_%\files.log
robocopy "D:\Files" "U:\Mirror\Files" /E /XO /XA:S /DCOPY:T /R:4 /W:10 /NP /LOG+:U:\%tdate_%\files.log /V
rem // 2nd CREATE pass doesn't overwrite any files
robocopy "D:\Files" "U:\%tdate_%\Files" /S /CREATE /XO /XN /XC /XA:S /A-:RSH /NP /LOG+:U:\%tdate_%\createtree.log
rem // moves anything that exists in mirror but not in 2nd pass to the archive
echo *** Archiving deleted files *** >> U:\%tdate_%\files.log
robocopy "U:\Mirror\Files" "U:\%tdate_%\Files" /E /MOV /DCOPY:T /XO /XN /XC /NP /LOG+:U:\%tdate_%\files.log /NDL /FP
rem // clean up the empty files
delempty -f -d -s -y "U:\%tdate_%"
echo Backup complete!
pause
This is definitely not a professional grade solution, since the last two steps take some time N based on number of files. Reasonable with 25k, not great for dealing with millions of files.

Panthrax
Jul 12, 2001
I'm gonna hit you until candy comes out.

Hi thread, I just found this, and I have a backup problem. Please help me! It's a little complicated, at least in my head, so I'll try to dump whatever I can.

We're moving from EMC Networker to Veeam, and we have a third-party DBA that does all of the MS SQL work and takes backups via SQL directly and copies them off for us to backup through other means. Before, Networker was simply set to exclude any SQL-related files, and back up everything else, which seemed to work just fine. Now, with Veeam, it's doing a whole lot more stuff that we're trying to either get it to stop doing, or explain to the DBAs that this is how it works and it's not actually breaking the SQL backups. Note that these are physical servers, not VMs.

We have the agents set to not do application aware backups, so it shouldn't be messing with SQL. We're seeing Veeam stop VSS writers and pause SQL for the backups, but we're also seeing Veeam try to log into SQL server with the veeam backup user account, which fails because it's not an allowed account in SQL server, so it then tries to do it via NT AUTHORITY\SYSTEM, which succeeds and it takes a backup? I guess? of a few of the DBs (maybe just the master DB) then exits. It seems to be doing this via a WITH COPY_ONLY option which seems to not break the backup chains, but it's a real pain to have to deal with for the DBA so they don't have to verify their own backups, make sure logs weren't truncated, etc. Their responsibility is to make sure the SQL backups are good and can be restored, while our responsibility is to get the server, OS, etc back up and running.

My question is, can we get Veeam to stop messing with SQL? It's fine if it needs to stop VSS to do its thing, but logging into SQL is causing all kinds of headaches for everyone. I just want to back up the C: drive and some random files on other volumes, that may or may not have SQL DBs on it as well. All the looking I've done has primarily how to deal with Veeam and SQL when you want it to back up, but I haven't been able to find much for if you don't want it to touch SQL.

Potato Salad
Oct 23, 2014

Nobody Cares




Tortured By Flan

Tell Veeam not to do Application Aware Processing and it'll stop doing anything more than VSS pauses.

Without Application Aware Processing, Veeam will take crash-consistent image backups of the machine using VSS, which seems to be all you want.

I have many times brought up Veeam image-based backups of MSSQL, Oracle, and other databases lacking application aware processing into prod or validation environments without a hitch. Worst I've had to do is rerun an archive redo log after restoration when an oracle DB lost power in unfavorable conditions.

Potato Salad fucked around with this message at 05:41 on Jan 26, 2019

Panthrax
Jul 12, 2001
I'm gonna hit you until candy comes out.

Potato Salad posted:

Tell Veeam not to do Application Aware Processing and it'll stop doing anything more than VSS pauses.

Without Application Aware Processing, Veeam will take crash-consistent image backups of the machine using VSS, which seems to be all you want.

I have many times brought up Veeam image-based backups of MSSQL, Oracle, and other databases lacking application aware processing into prod or validation environments without a hitch. Worst I've had to do is rerun an archive redo log after restoration when an oracle DB lost power in unfavorable conditions.

Yeah, App aware processing is already turned off, but it's still screwing around with SQL.

Here's a log from SQL side:

code:
2019-01-25 01:21:32.37 spid87      I/O is frozen on database Applications. No user action is required. However, if I/O is not resumed promptly, you could cancel the backup.
2019-01-25 01:21:32.37 spid92      I/O is frozen on database ArchivedCDRRECORD. No user action is required. However, if I/O is not resumed promptly, you could cancel the backup.
2019-01-25 01:21:32.37 spid111     I/O is frozen on database aspnet-BVOSS2api. No user action is required. However, if I/O is not resumed promptly, you could cancel the backup.
2019-01-25 01:21:32.37 spid112     I/O is frozen on database BulkData. No user action is required. However, if I/O is not resumed promptly, you could cancel the backup.
2019-01-25 01:21:32.37 spid113     I/O is frozen on database CCMI_LERG. No user action is required. However, if I/O is not resumed promptly, you could cancel the backup.
2019-01-25 01:21:32.37 spid114     I/O is frozen on database Costar. No user action is required. However, if I/O is not resumed promptly, you could cancel the backup.
2019-01-25 01:21:32.37 spid115     I/O is frozen on database CovadServices. No user action is required. However, if I/O is not resumed promptly, you could cancel the backup.
<snipped for brevity>
2019-01-25 01:21:32.56 spid87      I/O was resumed on database Applications. No user action is required.
2019-01-25 01:21:32.56 spid111     I/O was resumed on database aspnet-BVOSS2api. No user action is required.
2019-01-25 01:21:32.56 spid92      I/O was resumed on database ArchivedCDRRECORD. No user action is required.
2019-01-25 01:21:32.56 spid112     I/O was resumed on database BulkData. No user action is required.
<snipped for brevity>
2019-01-25 01:21:36.48 Backup      Database backed up. Database: db_manager_CLR, creation date(time): 2016/03/19(21:08:08), pages dumped: 345, first LSN: 39:989:1, last LSN: 39:992:1, number of dump devices: 1, device information: (FILE=1, TYPE=VIRTUAL_DEVICE: {'{EBD1773B-7716-4568-959C-E47399BC0D26}10'}). This is an informational message only. No user action is required.
2019-01-25 01:21:36.48 Backup      Database backed up. Database: db_migrate, creation date(time): 2016/03/19(09:54:21), pages dumped: 229313, first LSN: 2053088:2521:1, last LSN: 2053088:2524:1, number of dump devices: 1, device information: (FILE=1, TYPE=VIRTUAL_DEVICE: {'{EBD1773B-7716-4568-959C-E47399BC0D26}11'}). This is an informational message only. No user action is required.
2019-01-25 01:21:36.48 Backup      Database backed up. Database: db01, creation date(time): 2016/03/19(23:37:54), pages dumped: 311413, first LSN: 2316773:1402:169, last LSN: 2316773:1473:1, number of dump devices: 1, device information: (FILE=1, TYPE=VIRTUAL_DEVICE: {'{EBD1773B-7716-4568-959C-E47399BC0D26}12'}). This is an informational message only. No user action is required.
2019-01-25 01:21:36.49 Backup      BACKUP DATABASE successfully processed 0 pages in 12.690 seconds (0.000 MB/sec).
2019-01-25 01:21:36.49 Backup      BACKUP DATABASE successfully processed 0 pages in 12.960 seconds (0.000 MB/sec).
2019-01-25 01:21:36.49 Backup      BACKUP DATABASE successfully processed 0 pages in 12.160 seconds (0.000 MB/sec).
<snipped for brevity>

Panthrax fucked around with this message at 20:18 on Jan 26, 2019

HalloKitty
Sep 30, 2005

Adjust the bass and let the Alpine blast


Panthrax posted:

Yeah, App aware processing is already turned off, but it's still screwing around with SQL.

Here's a log from SQL side:

code:
2019-01-25 01:21:32.37 spid87      I/O is frozen on database Applications. No user action is required. However, if I/O is not resumed promptly, you could cancel the backup.
2019-01-25 01:21:32.37 spid92      I/O is frozen on database ArchivedCDRRECORD. No user action is required. However, if I/O is not resumed promptly, you could cancel the backup.
2019-01-25 01:21:32.37 spid111     I/O is frozen on database aspnet-BVOSS2api. No user action is required. However, if I/O is not resumed promptly, you could cancel the backup.
2019-01-25 01:21:32.37 spid112     I/O is frozen on database BulkData. No user action is required. However, if I/O is not resumed promptly, you could cancel the backup.
2019-01-25 01:21:32.37 spid113     I/O is frozen on database CCMI_LERG. No user action is required. However, if I/O is not resumed promptly, you could cancel the backup.
2019-01-25 01:21:32.37 spid114     I/O is frozen on database Costar. No user action is required. However, if I/O is not resumed promptly, you could cancel the backup.
2019-01-25 01:21:32.37 spid115     I/O is frozen on database CovadServices. No user action is required. However, if I/O is not resumed promptly, you could cancel the backup.
<snipped for brevity>
2019-01-25 01:21:32.56 spid87      I/O was resumed on database Applications. No user action is required.
2019-01-25 01:21:32.56 spid111     I/O was resumed on database aspnet-BVOSS2api. No user action is required.
2019-01-25 01:21:32.56 spid92      I/O was resumed on database ArchivedCDRRECORD. No user action is required.
2019-01-25 01:21:32.56 spid112     I/O was resumed on database BulkData. No user action is required.
<snipped for brevity>
2019-01-25 01:21:36.48 Backup      Database backed up. Database: db_manager_CLR, creation date(time): 2016/03/19(21:08:08), pages dumped: 345, first LSN: 39:989:1, last LSN: 39:992:1, number of dump devices: 1, device information: (FILE=1, TYPE=VIRTUAL_DEVICE: {'{EBD1773B-7716-4568-959C-E47399BC0D26}10'}). This is an informational message only. No user action is required.
2019-01-25 01:21:36.48 Backup      Database backed up. Database: db_migrate, creation date(time): 2016/03/19(09:54:21), pages dumped: 229313, first LSN: 2053088:2521:1, last LSN: 2053088:2524:1, number of dump devices: 1, device information: (FILE=1, TYPE=VIRTUAL_DEVICE: {'{EBD1773B-7716-4568-959C-E47399BC0D26}11'}). This is an informational message only. No user action is required.
2019-01-25 01:21:36.48 Backup      Database backed up. Database: db01, creation date(time): 2016/03/19(23:37:54), pages dumped: 311413, first LSN: 2316773:1402:169, last LSN: 2316773:1473:1, number of dump devices: 1, device information: (FILE=1, TYPE=VIRTUAL_DEVICE: {'{EBD1773B-7716-4568-959C-E47399BC0D26}12'}). This is an informational message only. No user action is required.
2019-01-25 01:21:36.49 Backup      BACKUP DATABASE successfully processed 0 pages in 12.690 seconds (0.000 MB/sec).
2019-01-25 01:21:36.49 Backup      BACKUP DATABASE successfully processed 0 pages in 12.960 seconds (0.000 MB/sec).
2019-01-25 01:21:36.49 Backup      BACKUP DATABASE successfully processed 0 pages in 12.160 seconds (0.000 MB/sec).
<snipped for brevity>

There's a registry key that's utterly undocumented that allows you to entirely exclude an SQL instance, but I'm at home, and I have it noted down at work... I have no idea if it works with the agent, though... I know it works when VBR takes a VM backup. If I actually remember about this thread on Monday or whatever, I'll post it.

MrMoo
Sep 14, 2000



Just want to note that Carbonite on MacOS does not appear to work from Hong Kong or Japan, I had to VPN in to get the backup updating

HalloKitty
Sep 30, 2005

Adjust the bass and let the Alpine blast


Panthrax posted:

Yeah, App aware processing is already turned off, but it's still screwing around with SQL.

Here's a log from SQL side:

code:
2019-01-25 01:21:32.37 spid87      I/O is frozen on database Applications. No user action is required. However, if I/O is not resumed promptly, you could cancel the backup.
2019-01-25 01:21:32.37 spid92      I/O is frozen on database ArchivedCDRRECORD. No user action is required. However, if I/O is not resumed promptly, you could cancel the backup.
2019-01-25 01:21:32.37 spid111     I/O is frozen on database aspnet-BVOSS2api. No user action is required. However, if I/O is not resumed promptly, you could cancel the backup.
2019-01-25 01:21:32.37 spid112     I/O is frozen on database BulkData. No user action is required. However, if I/O is not resumed promptly, you could cancel the backup.
2019-01-25 01:21:32.37 spid113     I/O is frozen on database CCMI_LERG. No user action is required. However, if I/O is not resumed promptly, you could cancel the backup.
2019-01-25 01:21:32.37 spid114     I/O is frozen on database Costar. No user action is required. However, if I/O is not resumed promptly, you could cancel the backup.
2019-01-25 01:21:32.37 spid115     I/O is frozen on database CovadServices. No user action is required. However, if I/O is not resumed promptly, you could cancel the backup.
<snipped for brevity>
2019-01-25 01:21:32.56 spid87      I/O was resumed on database Applications. No user action is required.
2019-01-25 01:21:32.56 spid111     I/O was resumed on database aspnet-BVOSS2api. No user action is required.
2019-01-25 01:21:32.56 spid92      I/O was resumed on database ArchivedCDRRECORD. No user action is required.
2019-01-25 01:21:32.56 spid112     I/O was resumed on database BulkData. No user action is required.
<snipped for brevity>
2019-01-25 01:21:36.48 Backup      Database backed up. Database: db_manager_CLR, creation date(time): 2016/03/19(21:08:08), pages dumped: 345, first LSN: 39:989:1, last LSN: 39:992:1, number of dump devices: 1, device information: (FILE=1, TYPE=VIRTUAL_DEVICE: {'{EBD1773B-7716-4568-959C-E47399BC0D26}10'}). This is an informational message only. No user action is required.
2019-01-25 01:21:36.48 Backup      Database backed up. Database: db_migrate, creation date(time): 2016/03/19(09:54:21), pages dumped: 229313, first LSN: 2053088:2521:1, last LSN: 2053088:2524:1, number of dump devices: 1, device information: (FILE=1, TYPE=VIRTUAL_DEVICE: {'{EBD1773B-7716-4568-959C-E47399BC0D26}11'}). This is an informational message only. No user action is required.
2019-01-25 01:21:36.48 Backup      Database backed up. Database: db01, creation date(time): 2016/03/19(23:37:54), pages dumped: 311413, first LSN: 2316773:1402:169, last LSN: 2316773:1473:1, number of dump devices: 1, device information: (FILE=1, TYPE=VIRTUAL_DEVICE: {'{EBD1773B-7716-4568-959C-E47399BC0D26}12'}). This is an informational message only. No user action is required.
2019-01-25 01:21:36.49 Backup      BACKUP DATABASE successfully processed 0 pages in 12.690 seconds (0.000 MB/sec).
2019-01-25 01:21:36.49 Backup      BACKUP DATABASE successfully processed 0 pages in 12.960 seconds (0.000 MB/sec).
2019-01-25 01:21:36.49 Backup      BACKUP DATABASE successfully processed 0 pages in 12.160 seconds (0.000 MB/sec).
<snipped for brevity>

Right, I actually did remember to reply with something useful.

Assuming we are talking about VBR backups (again, it might work with an agent, but I haven't tried), you need to create a registry key on the server to be backed up:

HKEY_LOCAL_MACHINE\SOFTWARE\Veeam\Veeam Backup and Replication
ExcludeSQLInstances

As a Multi-String Value

Then simply write the exact name of every SQL instance you want to exclude, each on a newline inside the multi-string value. Hope this helps.

Panthrax
Jul 12, 2001
I'm gonna hit you until candy comes out.

Could someone check or post their SQL logs for when Veeam takes a backup of a volume with both flat files that we want backed up (.bak, other business files) and live SQL files that we don't (.mdf and .ldf files) with app aware processing turned off? Veeam support tells me that in order for VSS to stop showing up in SQL logs, we need to remove the files we want to back up and put them on a separate drive than the SQL files, then it'll stop. However unfortunately I'm not the one talking to Veeam, and I can't tell if they're getting the whole logging into the DB thing confused with just VSS, or if we all know what we're talking about, or what.

Panthrax
Jul 12, 2001
I'm gonna hit you until candy comes out.

HalloKitty posted:

Right, I actually did remember to reply with something useful.

Assuming we are talking about VBR backups (again, it might work with an agent, but I haven't tried), you need to create a registry key on the server to be backed up:

HKEY_LOCAL_MACHINE\SOFTWARE\Veeam\Veeam Backup and Replication
ExcludeSQLInstances

As a Multi-String Value

Then simply write the exact name of every SQL instance you want to exclude, each on a newline inside the multi-string value. Hope this helps.

Thanks, I'll shoot this over to the guy handling the backups to test. For this we're specifically concerned about agent jobs on physical machines, so we'll definitely need to test it. Thanks for the info!

egyptian rat race
Jul 13, 2007

Lowtax Spine Fund 2019


Ultra Carp

Posting my set-up for a quick sanity check...

I have a Windows 10 desktop machine (SSD boot drive plus a 2TB internal drive for storage) that I want to back up. I only really care about family photos, documents and such so my data footprint I need to protect is pretty small right now. No large Blu-ray rips or anything else that isn't easily replaceable. On the recommendation of some co-workers I bought Acronis for back up service ($69/yr iirc). The program seems ok... It does a full backup image of the system to a 6TB NAS drive locally and a cloud back-up of <500GB of multimedia/documents only. These are done weekly, and the software allows creation of rescue media etc.

Does this set up make sense? I'm coming up on a cloud data limit which will bump me to $99/year, and I am considering switching backup software to something like Duplicati/Backblaze. I like that Acronis is pretty hands off and runs in the background, but I wanted to see if there's anything better out there.

egyptian rat race fucked around with this message at 13:07 on Mar 14, 2019

Potato Salad
Oct 23, 2014

Nobody Cares




Tortured By Flan

Frankly, for $99/year, look at getting Office 365 Family. Five people, many many devices, 1TB cloud space per person in OneDrive.

It sounds like you care more about the photos/small files than the OS itself, so Acronis works but it's possibly focused on the wrong thing.

Atomizer
Jun 24, 2007

Bote McBoteface. so what


Is there a good way to deal with backups of one large volume to multiple smaller drives, with regards specifically to figuring out how to replace the contents of one of those backup drives should the need arise? Let me explain: When I first started my Plex server I had media on two HDDs, backed up to 2 more drives of the same capacity, but I got tired of dealing with that 4-drive arrangement. I put all my media on a single 6 TB drive, and used two of the original 3 TB HDDs as backups, but I did this by dumping files on the first drive until it was full, then continuing with the second drive. So the first drive was full and in storage, and I was only ever dealing with the main drive and the current open backup.

I realized that if I had to replace the 6 TB drive it would be straightforward by copying from the backups, but if I had to replace one of the older 3 TB drives there would be no good way to determine what files belonged on that drive from the original library, or at least no way that I know of. So is there a way to do what I'm asking here, to use some software to look at the source drive, plus backup drive A, and determine the files not on A that need to go from the source to the new drive B?

I replaced those 3 TB drives with a single 6 TB HDD for the backup so the issue isn't nagging at the moment, but I'm going to have to upgrade capacity again in the next year or so. I have tons of older, lower-capacity drives that could be conveniently be repurposed as backups if I could figure out how to make it easier to manage their contents. Any ideas?

nielsm posted:

My own question:
I'd like to have offline backups at home, and imagine having a set of external harddrives I rotate between backing up to at home and keeping safe in my locker at work. E.g. rotating every Monday. Does anyone have experience with this kind of setup? Is it a good/bad idea and are there any pitfalls?
For external harddrives used in a multiple backup-sets setup like this, would cheap USB 2.5" drives be okay, or smarter to use a dock or the like and use 3.5" internal desktop drives?

Since nobody addressed this let me offer my thoughts. It sounds like what you've proposed makes sense, but I'd be slightly concerned about eventual damage to one of the rotated drives; hauling drives back and forth every week increases their exposure to shock (both physical, and electrical.) Other than that the only issue I can think of is theft; I'd suggest encrypting the contents just in case.

3.5" drives are cheaper per-capacity than 2.5" drives and are faster too, but the latter are easier to work with (they can run off 5 V and a single USB connector with no additional power supply.) For internal drives, you can get a 2.5" 2 TB for <$100 or a 5 TB 15 mm height (which won't fit in a laptop or regular enclosure) for >$150; 3.5" drive pricing is considered "good" at $20/TB or better. You can also get external drives (i.e. ones that come in their own enclosure and can usually be "shucked" but aren't intended to be) for better prices than internal ones, often: there are 4 TB portable drives from WD, Seagate, for ~$100 (these contain the 15 mm height drives); desktop (3.5", with external PSU) 6 TB drives are often $100, 8 TB as low as ~$120 with sales, and 10 TB for as low as ~$160 (typically the ~monthly sales at BB.)

So, depending on your budget and the amount of data you need to backup, I might go with the aforementioned 4 TB portable USB drives for $100. If you have tens of TB to backup however, the way to go would be reasonably-priced 3.5" drives and a heavy-duty carrying case like this. Just note as above, if you're dealing with bare drives that may increase the potential for handling issues, drops (you have to deal with inserting & removing them from an enclosure repeatedly in addition to the normal transportation concerns - also, the SATA connectors aren't rated for a ton of cycles AFAIK) in addition to ESD-related damage.

ChiralCondensate
Nov 13, 2007

what is that man doing to his colour palette?


Grimey Drawer

I use a pool of smaller drives as a backup target, using mergerfs to make them one big volume while keeping the filesystem available from each individual drive. If you go that way and lose a drive, rsync would generate the list of missing files for you without confusion.

refleks
Nov 21, 2006




Anyone have suggestions for backing up a small amount of files (less than 50GB at the moment) and a larger amount of photos and videos (3+ TB at the moment). Right now I have the files stored on Dropbox, but would like to combine everything into one solution (and get my photos backed up online)

Currently I store the photos on a local HDD in my computer, and copied on my Synology NAS and would prefer something that can sync through Cloyd Sync. I have been looking at Amazon Cloud as I already have a Prime account and can store unlimited photos free.

Any other suggestions?

Adbot
ADBOT LOVES YOU

ringu0
Feb 24, 2013




refleks posted:

Anyone have suggestions for backing up a small amount of files (less than 50GB at the moment) and a larger amount of photos and videos (3+ TB at the moment). Right now I have the files stored on Dropbox, but would like to combine everything into one solution (and get my photos backed up online)

Currently I store the photos on a local HDD in my computer, and copied on my Synology NAS and would prefer something that can sync through Cloyd Sync. I have been looking at Amazon Cloud as I already have a Prime account and can store unlimited photos free.

Any other suggestions?

Backblaze B2?

I don't have any experience with them, but I heard good things about them, and I'm going to try using them as a backup for my photos.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply
«2 »