NAS/Storage Megathread: What is this "File Deletion" You Speak of

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > NAS/Storage Megathread: What is this "File Deletion" You Speak of

Wheelchair Stunts: Dec 17, 2005

The more I dig into this the more it reminds me of car work. I've got to have more capacity, I need to push more iops, I need more data throughput.

:iiaca:

# ¿ Aug 27, 2011 12:28

Adbot: ADBOT LOVES YOU

# ¿ Apr 19, 2024 23:33

Wheelchair Stunts: Dec 17, 2005

I suspect I'm doing something retarded. I'm running Ubuntu (hahahahhaha) with native ZFS. I'm not seeing any way to configure the size of the arc cache. I've made some adjustments on boot so vmalloc complains a lot less but this is still biting me in the rear end. I'll probably end up grabbing another 4-8GB of memory, but I'd like to be able to resolve this as-is. I see nothing via zfs set/get other than setting what gets cached of on/off/metadata.

# ¿ Oct 1, 2011 03:19

Wheelchair Stunts: Dec 17, 2005

For the FreeBSD ZFS options, I was wondering if these offer NFS/SMB/CIFS in-kernel like Solaris does? I'm sure that NFS support is but not so sure on CIFS and/or if the kernel version in FreeBSD is compatible with like

code:

zfs set sharesmb=on;bullshit fartbullet

# ¿ Oct 28, 2011 05:44

Wheelchair Stunts: Dec 17, 2005

Nam Taf posted:

Help!

code:

  pool: raid
 state: DEGRADED
status: One or more devices could not be used because the label is missing or
        invalid.  Sufficient replicas exist for the pool to continue
        functioning in a degraded state.
action: Replace the device using 'zpool replace'.
   see: [url]http://www.sun.com/msg/ZFS-8000-4J[/url]
 scrub: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        raid        DEGRADED     0     0     0
          raidz1    DEGRADED     0     0     0
            ad4     ONLINE       0     0     0
            ad10    UNAVAIL      2    77     0  experienced I/O failures
            ad8     ONLINE       0     0     0
            ad6     ONLINE       0     0     0

errors: No known data errors

My gut tells me that this is a 'freak out and get a new drive' moment, but what I'm reading suggests that if it's small it might just suggest a hiccup by the PC and not actually be a problem.

What's the concensus on appropriate action to take with this sort of issue?

77 write failures seems pretty high. Is there anything telling in /var/adm/messages (or the equivalent)?

# ¿ Dec 18, 2011 11:33

Wheelchair Stunts: Dec 17, 2005

Wouldn't it be a better benchmark to use pseudo-random data rather than just zeros? I imagine compression settings and caching could impact this greatly, especially considering how closely tied ARC is to the pool.

# ¿ Dec 27, 2011 10:47

Wheelchair Stunts: Dec 17, 2005

I just realized, does the flood impact just production or research as well?

# ¿ Jan 12, 2012 08:43

Wheelchair Stunts: Dec 17, 2005

KennyG posted:

Define 'impact.'

Awesome Information Here.

Further on the long term it may be good for the industry and consumers as it will likely lead to temporarily increased profit margins leading to increased ability to conduct R&D as well as a willingness to expand and increase capacity.

This is about what I was thinking. I was also hoping that capacities would increase when production comes back online about where we'd figure it. As in having 4GB drives / whatever the new hotness would be at the time had R&D not staggered.

# ¿ Jan 15, 2012 01:35

Wheelchair Stunts: Dec 17, 2005

kill your idols posted:

OpenIndiana is just a mess with the new napp-it, so I went with Nex Core on any of my new builds. Solaris 11 isn't anything great compared to Solaris "Express" 11; which isn't for download off Oracle's site anymore.

NexentaCore3's site seems to have gone off-line for good with illumian rolling out as a 1.0 pre-release.

I'm pretty sure these ISO's can be shared since they were open source community related and what not. I'll seed them if anyone has a tracker we can use for goons.

I think this is what Alluvion is for. To serve as a torrent tracker for goons with no :filez:

or whatever.

# ¿ Mar 6, 2012 01:22

Wheelchair Stunts: Dec 17, 2005

kill your idols posted:

First time I have ever created a torrent, so hopefully I did it right.

Nexenta Core Platform 3.0.1 (nexenta-core-platform_3.0.1-b134_x86.iso):
Download torrent ⁸¹ (1 file; 549.12 MB)
View torrent stats

You are awesome.

As an aside, are there anyways to export the zpool on an Ubuntu zfs (via SPL) to import on a (Open/)Solaris stack? I swear I've been tracking the thread and did a quick browse.

# ¿ Mar 6, 2012 08:52

Wheelchair Stunts: Dec 17, 2005

Oh, man. You don't have to use fuse! Check this out. Especially since you use Ubunut, it even has its own little PPA setup. Performance was a lot better for me through this than fuse.

# ¿ May 2, 2012 03:17

Wheelchair Stunts: Dec 17, 2005

If you have access to the arc stats and whatnot, that can help a ton too. ARC will eat some motherfuckin' rams son.

# ¿ May 2, 2012 03:27

Wheelchair Stunts: Dec 17, 2005

Longinus00 posted:

I guess they better tell Oracle/Sun they're deploying their own technology incorrectly.

Considering the general consensus among almost everyone I work with / communicate with, this wouldn't be a terrible surprise.

edit: With regard to Oracle.

# ¿ May 2, 2012 05:27

Wheelchair Stunts: Dec 17, 2005

How can one OS handle flipped bits in ram better than another OS? I'm not doubting the validity, just based on my knowledge of how that kind of memory corruption happens I'm at a loss to understand how something as high-level as the OS would help.

# ¿ May 10, 2012 00:51

Wheelchair Stunts: Dec 17, 2005

FISHMANPET posted:

I suppose you could do some kind of software based ECC? I know, ZFS for memory!

What if your ARC gets corrupted? ZFS still does a non-trivial amount of semi-critical operation in memory.

# ¿ May 10, 2012 05:22

Wheelchair Stunts: Dec 17, 2005

movax posted:

So I switched out my U-Verse RG like two months ago, and I think I've forgotten to re-add ports for Crashplan to it, because I can't seem to backup to it remotely anymore. Is it actually OK to forward the control port (Telnet based I think, 4242), or is opening that to the world a horrible idea?

I wouldn't knowingly forward a port that sends any authentication/control data over plaintext. Could always tunnel it through ssh / openVPN.

# ¿ May 13, 2012 04:15

Wheelchair Stunts: Dec 17, 2005

Telex posted:

well...

troubleshooting a shitload of things DOES involve "telnet to the port and see if you get a response".

I mean honestly telnet is just raw TCP to a port isn't it? Not the cleanest way to refer to it, but it gets the job done.

Yes, telnet is very simple and the telnet client can be used to test tcp ports and the like. However, it has its own standard and has been historically used as a means of authentication that sends data over plaintext so some assumption is to be expected.

# ¿ May 14, 2012 08:38

Wheelchair Stunts: Dec 17, 2005

Why not just use version control? It handles diffs/versioning and duplication.

# ¿ May 17, 2012 03:04

Wheelchair Stunts: Dec 17, 2005

devmd01 posted:

Horrible idea. Vmware server 2 is horribly slow and lacks the features you would need to do what I think you think you want to do. If you go the esxi/vsphere route, you won't get to do any video output. Adding layers of complexity like virtualization is a really bad idea unless you know what you're doing and why. Do you really want to gently caress something up messing with esxi and delete all of your movies?

KISS always, especially for a home setup. Do you really want to have to gently caress with it when something breaks? My file server for video/music is a fully separate box from my vm server for this reason. If my file server fucks up for any reason, I don't run the risk of my network being completely down since the router is a pfsense vm.

Just use windows software raid for whatever volumes you want if you can't afford a hardware controller, they'll transfer to new hardware just fine.

Edit: don't use intel matrix raid, it's garbage (unless it's improved significantly since the ICH10R)

I'd go further to say don't trust anything you don't understand technically and/or intimately with data you give a poo poo about.

# ¿ May 20, 2012 10:02

Wheelchair Stunts: Dec 17, 2005

Jonny 290 posted:

As far as I know, ZFS is FreeBSD territory these days, or possibly OpenSolaris? It's incompatible with the GPL so there are hurdles to clear before they put out a real Linux port. I have casually looked into installing the Debian distro with freebsd kernel to utilize it, but for now just run 3x1T drives in a simple mdadm RAID5 array with LVM on top for partition management, which you honestly don't even really need. I'm just storing TV shows and such on it and the performance is just fine.

I have had luck with Ubuntu and zfs on linux. You just add the ppa and aptitude install what you need. It does it through a "portability layer" which is just them putting compatibility for Solaris APIs in the kernel. This taints the kernel from a licensing perspective, so don't expect this to be distributed ever. I seemed to be able to get decent performance especially in comparison to ZFS over Fuse. I would still probably go with FreeBSD or more likely a Nexenta/Illumos/OpenSolaris type.

# ¿ Jun 9, 2012 12:49

Wheelchair Stunts: Dec 17, 2005

kapalama posted:

So does that ZFS work by just taking advantage of all the storage space you got, and dynamically resizing and all that as you add random new drives?

I'm not so familiar with which options are provided in which version. I believe it is compatible up to version 28 zpools. I did have issues running some of the default software for monitoring ARC, but that may have been my own silliness.

# ¿ Jun 9, 2012 14:07

Wheelchair Stunts: Dec 17, 2005

DrDork posted:

File-server stuff takes very little actual CPU power by today's standards, and can be capably handled by pretty much any current CPU (well, except maybe some of those VIA super-low power chips). I'm sure they'd be fine. You don't have to worry unless you want it to do other stuff, too, like HTPC work or transcoding.

As for the transcoding part, I think AMD offers that via the GPU part of the on-die APU so they might even be squared away there.

# ¿ Jun 12, 2012 14:33

Adbot: ADBOT LOVES YOU

# ¿ Apr 19, 2024 23:33

Wheelchair Stunts: Dec 17, 2005

Do you UPSes no longer offer serial output? I mean, that's about as ubiquitous as you can get.

# ¿ Jun 13, 2012 22:30

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > NAS/Storage Megathread: What is this "File Deletion" You Speak of