Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
Thanks Ants
May 21, 2004

#essereFerrari


I signed up to Backblaze a couple of years ago because CrashPlan was some piece of poo poo Java app. I have no idea if this has changed but I have had no reason to stop using Backblaze.

Adbot
ADBOT LOVES YOU

thebigcow
Jan 3, 2001

Bully!

D. Ebdrup posted:


Finally, something I've been meaning to look into the economics of, but haven't really had time to, is rsync.net which now supports native zfs send with snapshots and everything else you could want.

It costs a whole lot more than most everything else, you're paying for being able to just rsync things and have it work. It wasn't bad if you had a small office worth of office docs but forget about movie collections. They had a discount for people with freenas forum accounts, don't know if it still exists.

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

Thanks Ants posted:

I signed up to Backblaze a couple of years ago because CrashPlan was some piece of poo poo Java app. I have no idea if this has changed but I have had no reason to stop using Backblaze.

It's still a Java app, and it's not great,but I pretty much never have to open the app, so I don't care.

Skandranon
Sep 6, 2008
fucking stupid, dont listen to me

Thermopyle posted:

It's still a Java app, and it's not great,but I pretty much never have to open the app, so I don't care.

I did have an issue where it stopped uploading... turns out it needs more RAM allocated to it (in the config pane) as more stuff gets uploaded.

kri kri
Jul 18, 2007

Skandranon posted:

I ran my Unraid server with a very similar setup, and it worked great for a number of years. I recently switched to using SnapRaid, which has a similar idea, but more flexible. Either one will allow you to easily add new disks (assuming they are smaller than the largest you have), and even if you are upgrading your Parity drive(s), it can be done without destroying the array. Another nice thing about Unraid/SnapRaid is that, if your array DOES become broken, you only lose data on the drives that died. The remaining drives are still perfectly accessible drives all by themselves, and can be moved to another machine with no effort.

Can you expand on why you switched from unraid to snapraid? What OS are you running now?

Skandranon
Sep 6, 2008
fucking stupid, dont listen to me

kri kri posted:

Can you expand on why you switched from unraid to snapraid? What OS are you running now?

A few reasons. Unraid didn't fail me in any significant way, SnapRaid had a few compelling features that I wanted. Mainly that it could run on top of Windows, and that the drive filesystem could be any partition, instead of being wrapped up in a fixed Linux distro using RieserFS. I'm now running Windows Server 2012, so I can manage my file permissions to work with the rest of my Windows boxes better. SnapRaid also supports up to 6 (or maybe more, with extra config) parity drives, whereas Unraid is still stuck at 1. I also like that the parity calculation isn't live, so if I am moving files between drives in the array, it doesn't thrash the parity for every single file operation. I can move files around at max speed, then sync when I'm done.

SnapRaid is also completely free, while Unraid does cost money, though that is a more compelling feature for people who haven't already bought Unraid.

Kenshin
Jan 10, 2007
I've been using a Buffalo LS441D with 3x2TB drives in a RAID5 array for a while now, and for some reason I've only now noticed that the block size of the XFS filesystem is set to 128MB. Since the majority of the files on the NAS are <128MB (hundreds of video files >128MB, and thousands of my RAW image files from my cameras ~30MB) this wastes immense amounts of space.

That said, I can't find any settings on the NAS to change this, if indeed it can even be changed, or changed without destroying the existing filesystem. Could anybody offer any experience here?

G-Prime
Apr 30, 2003

Baby, when it's love,
if it's not rough it isn't fun.
I don't know how to change it on your NAS, but the easiest (though very time consuming) solution is to group smaller files into large archives. It wouldn't be too hard to write a script that crawls through a directory and builds out individual archives that are just under 128MB in an intelligent fashion.

thebigcow
Jan 3, 2001

Bully!

Kenshin posted:

I've been using a Buffalo LS441D with 3x2TB drives in a RAID5 array for a while now, and for some reason I've only now noticed that the block size of the XFS filesystem is set to 128MB. Since the majority of the files on the NAS are <128MB (hundreds of video files >128MB, and thousands of my RAW image files from my cameras ~30MB) this wastes immense amounts of space.

That said, I can't find any settings on the NAS to change this, if indeed it can even be changed, or changed without destroying the existing filesystem. Could anybody offer any experience here?

Are you sure you have a block size of 128MB and not 128KB?

Kenshin
Jan 10, 2007

thebigcow posted:

Are you sure you have a block size of 128MB and not 128KB?

Quite sure, unfortunately.



Every single file under 128MB on the NAS looks similar to that.

Anything >128MB <256MB takes up 256, etc. It's a very inefficient use of space for files under a few GB.

Vidaeus
Jan 27, 2007

Cats are gonna cat.
Hey all, I’m pretty new to setting up my own fileservers and especially with OSes other than Windows, although I would consider myself a fairly advanced user of that.

I have a brand new machine setup as a NAS, specs are:

CPU: Intel Pentium G3260 3.3Ghz
MoBo: Asus B85M-E
RAM: 16GB something Geil (4x 4GB)
HDDs: 2x WD reds 3TB setup as RAID mirror

I have installed Openmediavault on it as the OS on a spare hdd I had lying around. The problem I am having is that file transfers to and from my main PC with Windows 10 are very slow, only around 8-12 megabytes/s, looks like it is maxing out a 100 megabit connection instead of gigabit. I used ethtool to check the connection speed on the NAS and it reported the text below, so it looks like it is connected at gigabit speeds? My main PC is also reporting gigabit speeds in Windows. No idea what could be causing it to be so slow? Please let me know if I can post any other info or diagnostics to help diagnose this, keeping in mind I am unfamiliar with Linux.

Settings for eth0:
Supported ports: [ TP MII ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Half 1000baseT/Full
Supported pause frame use: No
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Half 1000baseT/Full
Advertised pause frame use: Symmetric Receive-only
Advertised auto-negotiation: Yes
Link partner advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Half 1000baseT/Full
Link partner advertised pause frame use: Symmetric Receive-only
Link partner advertised auto-negotiation: Yes
Speed: 1000Mb/s
Duplex: Full
Port: MII
PHYAD: 0
Transceiver: internal
Auto-negotiation: on
Supports Wake-on: pumbg
Wake-on: g
Current message level: 0x00000033 (51)
drv probe ifdown ifup
Link detected: yes

Zorak of Michigan
Jun 10, 2006

What does the network between the file server and the NAS box look like?

Vidaeus
Jan 27, 2007

Cats are gonna cat.

Zorak of Michigan posted:

What does the network between the file server and the NAS box look like?

Do you mean the cabling setup? It goes like this:

NAS>cable>router (the NAS sits like less than a foot from the router)

PC>cable>wall port+cabling>cable>router (I have Ethernet run through the walls in my home)

The router is Netgear Nighthawk R7000 so it should be plenty fast. I guess I can try a direct cable between my PC and the router to rule out any issues with the other cables or the wall cabling. But the PC is reporting gigabit when I check the network status in Windows.

smax
Nov 9, 2009

Vidaeus posted:

Do you mean the cabling setup? It goes like this:

NAS>cable>router (the NAS sits like less than a foot from the router)

PC>cable>wall port+cabling>cable>router (I have Ethernet run through the walls in my home)

The router is Netgear Nighthawk R7000 so it should be plenty fast. I guess I can try a direct cable between my PC and the router to rule out any issues with the other cables or the wall cabling. But the PC is reporting gigabit when I check the network status in Windows.

You might have a crappy cable in there somewhere. Try different patch cables, though I'd say it's more likely it's an issue with the terminations on the in-wall cable. Also be sure the cables (including the one in your walls) is cat5e+.

Edit: Derp- just saw that you removed that possibility.

It would be a good idea to run iperf between the two machines. Sometimes you can get lower speeds even if the link was negotiated at gigabit.

smax fucked around with this message at 00:06 on Feb 4, 2016

mayodreams
Jul 4, 2003


Hello darkness,
my old friend

Vidaeus posted:

Do you mean the cabling setup? It goes like this:

NAS>cable>router (the NAS sits like less than a foot from the router)

PC>cable>wall port+cabling>cable>router (I have Ethernet run through the walls in my home)

The router is Netgear Nighthawk R7000 so it should be plenty fast. I guess I can try a direct cable between my PC and the router to rule out any issues with the other cables or the wall cabling. But the PC is reporting gigabit when I check the network status in Windows.

Realtek nics can give really lovely performance in NAS configs. A hardware Intel nic can make a huge difference.

http://m.newegg.com/Product/index?i...uQ&gclsrc=aw.ds

Vidaeus
Jan 27, 2007

Cats are gonna cat.
Thanks for the tips. I'll try a direct connection to pinpoint if I have a dodgy cable/termination (likely as I made the cables and did the terminations myself hehe) and try iperf when I get home from work. Will see how that goes before I go get a new NIC

sharkytm
Oct 9, 2003

Ba

By

Sharkytm doot doo do doot do doo


Fallen Rib

Vidaeus posted:

Thanks for the tips. I'll try a direct connection to pinpoint if I have a dodgy cable/termination (likely as I made the cables and did the terminations myself hehe) and try iperf when I get home from work. Will see how that goes before I go get a new NIC

I had the same issue with my NAS. Turned out, I had a broken wire on the punchdown terminal in one of the walls. Both the NAS and the PC reported gigabit connections, but it was only seeing 100-BaseT. I used a cable checker to check the wiring, and mysteriously, one pair was borked. Strip and re-punch down the wire, gigabit speeds restored. If you've got a cable checker, I'd start by checking the in-wall wiring and patch cables.

Vidaeus
Jan 27, 2007

Cats are gonna cat.

sharkytm posted:

I had the same issue with my NAS. Turned out, I had a broken wire on the punchdown terminal in one of the walls. Both the NAS and the PC reported gigabit connections, but it was only seeing 100-BaseT. I used a cable checker to check the wiring, and mysteriously, one pair was borked. Strip and re-punch down the wire, gigabit speeds restored. If you've got a cable checker, I'd start by checking the in-wall wiring and patch cables.

Ok good to see that I'm heading down the correct path then, thanks!

sharkytm
Oct 9, 2003

Ba

By

Sharkytm doot doo do doot do doo


Fallen Rib

Vidaeus posted:

Ok good to see that I'm heading down the correct path then, thanks!

The KISS principle applies. Chase the easy, simple, cheap fixes first, then spend money if you have to.

Vidaeus
Jan 27, 2007

Cats are gonna cat.
Righto, so I found my problem. It was simple, AND stupid.

So I got my path from my main PC to the router wrong. I forgot I had a surge board with onboard network and telephone connection. I thought I might as well put my main PC through the ethernet ports on that since they were there. Turns out this is the device that is throttling my main PC to 100 megabits. I bypassed this and BAM, straight up to 100+ megabytes/s transfer!

Strange the PC was still reporting a gigabit link though.

EDIT: Made me feel better about the DIY job I did running the ethernet throughout the house hehe.

EDIT2: Also I kept getting weird internet connection problems whilst downloading files from servers. Some servers would be super slow while others would be fast. Steam downloads would keep cutting out and going into "scheduled" Especially sensitive files such as the .isos for open media vault as well as firmware upgrades etc. kept failing and the hashes were different. I had to use my connection at work to download stuff. Just tried a couple things now and it looks like it's all good now! A firmware upgrade I could never download previously as the hashes never matched downloaded the first time with the correct hash!
Stupid loving surge protector was loving up my network.

Vidaeus fucked around with this message at 09:41 on Feb 4, 2016

Curvature of Earth
Sep 9, 2011

Projected cost of
invading Canada:
$900
Are there any specific recommendations for a painless NAS that does automatic daily backups? My parents need a backup system for, eventually, two laptops. They apparently bought the worst Asus laptops on the face of the earth, and have now lost irreplaceable data multiple times. They're getting new (more reliable) laptops, but they're now very concerned about preventing future data losses. They need a way to create daily backups over wifi.

I would greatly appreciate a specific pre-built NAS model recommendation.

As for tech level, my dad dropped his computer-building/tweaking hobby around 2003ish, so he's very out-of-date but comfortable reading manuals and rooting around in hostile interfaces. That being said, (a) he's an old man with frankly better things to do than piecing together a custom hardware/software setup (b) the actual act of backing up data must be painfully easy and automatic, because my mother has the technical ability of an eggplant.

Thanks Ants
May 21, 2004

#essereFerrari


File History that has been a part of Windows since Windows 8 will happily use a network target for backups. If they are getting Macs then the standard Qnap/Synology/ReadyNAS options will all pretend to be Time Machine targets.

Gyshall
Feb 24, 2009

Had a couple of drinks.
Saw a couple of things.
Anyone use Unraid here that can give me a recommendation on if having 3 250 GB SSDs in my cache pool will provide any read/write benefit? Or would this be the same as having one or two drives in the cache pool?

Skandranon
Sep 6, 2008
fucking stupid, dont listen to me

Gyshall posted:

Anyone use Unraid here that can give me a recommendation on if having 3 250 GB SSDs in my cache pool will provide any read/write benefit? Or would this be the same as having one or two drives in the cache pool?

I haven't had an Unraid server for about a year now (well, still use the preclear for burning in drives), but I don't see how having more drives would make it faster than one, unless it is striping the drives. Even then, a 3xRaid0 SSD would be way faster than your ethernet. A single decent SSD should be faster than your ethernet, so I think it's safe to say it's not likely to help any.

suck my woke dick
Oct 10, 2012

:siren:I CANNOT EJACULATE WITHOUT SEEING NATIVE AMERICANS BRUTALISED!:siren:

Put this cum-loving slave on ignore immediately!
He might be someone with too much money and 10 Gigabit ethernet. In which case the bottleneck is SATA.

BlankSystemDaemon
Mar 13, 2009



A modern platter disk is faster than 1000BASE-T for sequential read and write.
My understanding is that unraid uses caching only for writes in RAID5+ arrays, which also means it's very useful for scratch-disks and similar.
That being said, it looks like writing isn't particularily good on unraid, unless improved by caching.

BlankSystemDaemon fucked around with this message at 16:53 on Feb 8, 2016

Skandranon
Sep 6, 2008
fucking stupid, dont listen to me

D. Ebdrup posted:

A modern platter disk is faster than 1000BASE-T for sequential read and write.
My understanding is that unraid uses caching only for writes in RAID5+ arrays, which also means it's very useful for scratch-disks and similar.
That being said, it looks like writing isn't particularily good on unraid, unless improved by caching.

Unraid does not do RAID5, it is a different sort of parity protection. And it's well known that this method causes slower writes, that's why there is a cache drive feature.

Gyshall
Feb 24, 2009

Had a couple of drinks.
Saw a couple of things.

blowfish posted:

He might be someone with too much money and 10 Gigabit ethernet. In which case the bottleneck is SATA.

I wish, just looking for a decent solution for my new NAS. Is there something better than Unraid that I can run containers for my various apps and still run Windows/Linux VMs as well?

BlankSystemDaemon
Mar 13, 2009



By containers, do you mean OS-level virtualization like jails on FreeBSD or Docker/LXC on Linux?

I only know of one solution that can handle both OS along with bare-metal hypervisor, and that's FreeBSD with zfs on root with raidz1+, jails+iocage and bhyve+iohyve, because I'm using it as we speak.

According to FreeNAS' documentation, it can do it with VirtualBox, but I suspect it'll have as much tinkering as FreeBSD will, at that point. FreeNAS is at its best when it's used as an appliance NAS for network shares, and not a whole lot more.

Skandranon
Sep 6, 2008
fucking stupid, dont listen to me

Gyshall posted:

I wish, just looking for a decent solution for my new NAS. Is there something better than Unraid that I can run containers for my various apps and still run Windows/Linux VMs as well?

The main benefit of using Unraid is a) better disk space utilization b) better recovery options in the case of a failed array (ie, you can still recover data from working disks) and c) read operations only need the individual disk that has the files, instead of spinning up the entire array. If you are running VMs from these containers, it sounds like you'll be doing a fair bit more read/write activity than what Unraid is designed for, which is closer to https://en.wikipedia.org/wiki/Nearline_storage than anything else. If you just need space for your VM containers, I'd pick what's best for your VM architecture and run some RAID-1 arrays of either SSDs or HDDs, which will be much faster and simpler.

Gyshall
Feb 24, 2009

Had a couple of drinks.
Saw a couple of things.
Thanks - that is what I think I'll end up doing, actually.

wang souffle
Apr 26, 2002

D. Ebdrup posted:

By containers, do you mean OS-level virtualization like jails on FreeBSD or Docker/LXC on Linux?

I only know of one solution that can handle both OS along with bare-metal hypervisor, and that's FreeBSD with zfs on root with raidz1+, jails+iocage and bhyve+iohyve, because I'm using it as we speak.

According to FreeNAS' documentation, it can do it with VirtualBox, but I suspect it'll have as much tinkering as FreeBSD will, at that point. FreeNAS is at its best when it's used as an appliance NAS for network shares, and not a whole lot more.

https://smartos.org

I run multiple OS zones, a couple "native" LX zones for applications which require Linux (ie, Plex), and Windows on KVM when things get desperate. All on top of a ZFS datastore.

Mr Shiny Pants
Nov 12, 2012

wang souffle posted:

https://smartos.org

I run multiple OS zones, a couple "native" LX zones for applications which require Linux (ie, Plex), and Windows on KVM when things get desperate. All on top of a ZFS datastore.

I wish I could run SmartOS as a Hyper Visor. No Infininband subnet manager means I need Linux though. I already have an Open Indiana NAS box.

Works out pretty well though.

Combat Pretzel
Jun 23, 2004

No, seriously... what kurds?!
Ars Technica's guide to building a NAS:

http://arstechnica.com/gadgets/2016/02/the-ins-and-outs-of-planning-and-building-your-own-home-nas/

Spoiler:
He builds a regular PC, sticks two hard drives in, slaps Windows on it and also plans to game with it. "NAS."

--edit:
Out of curiosity, is there something like FreeNAS, but based on Illumos?

Combat Pretzel fucked around with this message at 20:44 on Feb 10, 2016

IOwnCalculus
Apr 2, 2003





And yet it's still better than the guy whose "brother that works with servers" built him a 40-odd-drivecount box that used USB for all of it.

thebigcow
Jan 3, 2001

Bully!

Combat Pretzel posted:

Ars Technica's guide to building a NAS:

http://arstechnica.com/gadgets/2016/02/the-ins-and-outs-of-planning-and-building-your-own-home-nas/

Spoiler:
He builds a regular PC, sticks two hard drives in, slaps Windows on it and also plans to game with it. "NAS."

--edit:

Out of curiosity, is there something like FreeNAS, but based on Illumos?

Does napp-it do what you're thinking of?

Skandranon
Sep 6, 2008
fucking stupid, dont listen to me

Boy, I sure am glad he provided that handy graph showing how many FPS to expect from my NAS.

Combat Pretzel
Jun 23, 2004

No, seriously... what kurds?!

thebigcow posted:

Does napp-it do what you're thinking of?
Yeah, but I'm not too excited about its UI. I'm going to look into it.

Mr Shiny Pants
Nov 12, 2012

Combat Pretzel posted:

Yeah, but I'm not too excited about its UI. I'm going to look into it.

Open Indiana is nice, and the Solaris way doing things, while at first seem odd, are pretty nifty.

I loving hated nwamcfg at first, but yesterday I needed to re-ip some stuff and it sure is nice when things "click".

Stmfadm is nice, dladm is also pretty cool. It beats systemd on Debian, if you want restart the network, by a long shot.

Adbot
ADBOT LOVES YOU

Combat Pretzel
Jun 23, 2004

No, seriously... what kurds?!
I have no problems with Solaris. I ran it as main system on my desktop since shortly after OpenSolaris was public until Oracle hosed it up. The userland is pretty cool. I'm just not too excited about that napp-it UI. Having a decent web interface to check up on things would be nice.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply