Can I build a home network faster than 1 gigabit?

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > Can I build a home network faster than 1 gigabit?

«‹›3 »

KS: Jun 10, 2003; Outrageous Lumpwad

I've been selling a bunch of X520-DA2s on Ebay. Guess since there's continued interest I'll try some in SA-Mart instead.

Besides the brand of twinax cables, another thing to watch out for is length. The cables move from passive to active electronics above 5M, and at least for the Intel X520 you need the Windows driver toolset installed to use an active cable. Last I looked, there was no way to use an active cable in ESXi, for instance.

# ? Sep 17, 2014 04:41

Adbot: ADBOT LOVES YOU

# ? Apr 24, 2024 10:20

Computer viking: May 30, 2011; Now with less breakage.

I'm on the wrong continent (and customs handling is insane on anything above ~ $30), otherwise that would be quite interesting.

# ? Sep 17, 2014 09:59

Methylethylaldehyde: Oct 23, 2004; BAKA BAKA

KS posted:

I've been selling a bunch of X520-DA2s on Ebay. Guess since there's continued interest I'll try some in SA-Mart instead.

Besides the brand of twinax cables, another thing to watch out for is length. The cables move from passive to active electronics above 5M, and at least for the Intel X520 you need the Windows driver toolset installed to use an active cable. Last I looked, there was no way to use an active cable in ESXi, for instance.

How much are you charging for one?

# ? Sep 17, 2014 22:54

Lowen SoDium: Jun 5, 2003; Highen Fiber; Clapping Larry

Potato Alley posted:

Well the other problem is that the switch makers are always playing catchup with blocking the third parties, which means that a new release of software will inevitably render some previously-compatible third-party SFPs incompatible. Depending on how often you update your switches this may or may not be a problem, but if you have a large deployment of third-party SFPs and upgrade, suddenly that "free" switch upgrade (which these days is less and less free, at least with Cisco) just cost you a mint in new SFPs. Plus potential downtime and complete confusion if you're not aware of the issue, since if you have a mix of SFPs, SOME may stop working and others may be fine, and you'll have no idea why.

And of course the companies say up front "we absolutely don't support third party SFPs because they could not work". Hey listen. I get that. The third party stuff may not be held to the same quality standards. But don't tell me that the supposed quality difference is worth literally 10 times as much. When it's clearly just gouging me is when I get pissed off and start calling it a racket. (Also, I've never heard of any real quality differences in these things in terms of packet errors or speed issues).

It's the same issue as Ford or Toyota claiming you need to use their parts and only their parts, and I believe there were laws passed to prevent them from disallowing warranty service etc. Though maybe I'm misremembering. But there's certainly nothing currently to prevent switch makers from doing it, until RJ-45 becomes more prevalent in 10 GBe switches.

For Cisco:

service unsupported-transceiver
no errdisable detect cause gbic-invalid

# ? Sep 18, 2014 17:09

KillHour: Oct 28, 2007

http://www.newegg.com/Product/Product.aspx?Item=N82E16833122529

http://www.newegg.com/Product/Product.aspx?Item=N82E16833122542

Finding NICs is left as an exercise to the reader.

# ? Sep 18, 2014 17:21

Combat Pretzel: Jun 23, 2004; No, seriously... what kurds?!

gggiiimmmppp posted:

In the end I learned that while the iSCSI extent can be allowed to grow dynamically, it cannot be shrunk again so after bloating it up to 2tb shuffling temporary stuff around I was faced with having to delete the whole disk and start over to get the space back so I just said gently caress it and have put up with a measly 1gbps ever since.

To get back on this, since I just read the FreeNAS forums, apparently the current iSCSI target service is to blame. istgt can't punch holes, whereas CTL seemingly can. Latter will be default in 9.3 (which is however still in development). I've just tested it, creating an iSCSI extent, put a NTFS volume on it, copied a 600MB ISO file on it, caused the ZVOL pool usage to raise to around 600MB, deleted the ISO, ZVOL pool usage dropped down to 1.7MB. So if you want TRIM/DISCARD, 9.3+ is the way to go.

# ? Sep 19, 2014 00:44

Computer viking: May 30, 2011; Now with less breakage.

Or 10 - keep in mind that -CURRENT is 11 these days.

# ? Sep 19, 2014 14:51

Combat Pretzel: Jun 23, 2004; No, seriously... what kurds?!

If you go the straight FreeBSD route instead of FreeNAS. Latter said to expect the FreeBSD 10 based version of their stuff for next summer only.

I'd like to go with FreeBSD direct, but I can't be assed to deal with Samba 4.

# ? Sep 19, 2014 16:56

Computer viking: May 30, 2011; Now with less breakage.

Combat Pretzel posted:

If you go the straight FreeBSD route instead of FreeNAS. Latter said to expect the FreeBSD 10 based version of their stuff for next summer only.

I'd like to go with FreeBSD direct, but I can't be assed to deal with Samba 4.

I've set it up a few times at work - and yeah, assuming you need the things that aren't in 3.6, setting it up is one of those things best avoided.
(In my case, I'm using a windows 2003 R2 AD domain that I don't have domain admin access to for authenticating users. Ever so fun.)

That said, you could just copy the config files over from your current setup. (smb.conf, at least; perhaps the kerberos stuff if you're doing something unpleasant).

Computer viking fucked around with this message at 20:18 on Sep 19, 2014

# ? Sep 19, 2014 18:41

Combat Pretzel: Jun 23, 2004; No, seriously... what kurds?!

I've looked some into adapters. Infiniband ones are affordable, but the cables are not, 10GBe adapters are not affordable, but cables are (if you go with RJ45). :psyduck:

# ? Sep 19, 2014 20:49

Alereon: Feb 6, 2004; Dehumanize yourself and face to Trumpshed; College Slice

I just upgraded my LAN wiring to CAT7 so I'll be ready when 10GBASE-T adapters come down in price :v:

Sure the actual reason was that CAT7 wasn't much more expensive and STP holds up better mechanically when installed improperly, but still

# ? Sep 21, 2014 17:40

Captain Foo: May 11, 2004; we vibin'
we slidin'
we breathin'
we dyin'

Alereon posted:

I just upgraded my LAN wiring to CAT7 so I'll be ready when 10GBASE-T adapters come down in price Sure the actual reason was that CAT7 wasn't much more expensive and STP holds up better mechanically when installed improperly, but still

# ? Sep 21, 2014 20:03

Nam Taf: Jun 25, 2005; I am Fat Man, hear me roar!

Combat Pretzel posted:

I'd like to go with FreeBSD direct, but I can't be assed to deal with Samba 4.

So pkg install samba36 not samba4? Have I done something wrong by doing that??

# ? Sep 22, 2014 01:09

Combat Pretzel: Jun 23, 2004; No, seriously... what kurds?!

The configuration was a pain in the rear end the last time I've tried.

# ? Sep 22, 2014 01:30

Nam Taf: Jun 25, 2005; I am Fat Man, hear me roar!

Samba36 config is as it's always been. I haven't bothered delving into samba4 yet either but there's ample documentation out there running you through getting a basic 36 setup sorted.

# ? Sep 22, 2014 02:59

Computer viking: May 30, 2011; Now with less breakage.

IIRC, samba4 isn't too bad (or too different) to set up if you only want to do things you could also do in 3.6. The madness enters when you want to use it as an AD controller.

# ? Sep 22, 2014 07:24

Combat Pretzel: Jun 23, 2004; No, seriously... what kurds?!

Eh, I thought you would get more performance out of Samba4, because of its support of the newer SMB protocols.

# ? Sep 22, 2014 09:34

Computer viking: May 30, 2011; Now with less breakage.

Combat Pretzel posted:

Eh, I thought you would get more performance out of Samba4, because of its support of the newer SMB protocols.

That could well be true.

# ? Sep 22, 2014 17:41

Creizai: Jun 10, 2005; I love Rizien

Straker posted:

http://en.wikipedia.org/wiki/10-gigabit_Ethernet

Newegg has decent enough 10gige NICs for like $400, so guess it depends on whether it's worth it to you? That seems way more useful/less of a headache than bonding and poo poo just for double the bandwidth.

Do you think those 10gige NICs setups would allow for better steam box streaming?

# ? Oct 6, 2014 02:55

Lowen SoDium: Jun 5, 2003; Highen Fiber; Clapping Larry

Creizai posted:

Do you think those 10gige NICs setups would allow for better steam box streaming?

Better than 1Gb NICs?

No.

# ? Oct 6, 2014 03:04

Cpl Clegg: May 18, 2008

I'm thinking about used Infiniband switches and HCAs myself as well. Not that I really need it at home (or anyone really) but it'll be an interesting thing to tinker with.

# ? Oct 7, 2014 21:31

KillHour: Oct 28, 2007

Creizai posted:

Do you think those 10gige NICs setups would allow for better steam box streaming?

It's all about latency, bro.

First, you need one of these:

http://www.digilentinc.com/Products/Detail.cfm?NavPath=2,400,1228&Prod=NETFPGA-1G-CML

Then you're gonna want to write your own network stack. Why? Latency. You don't need no TCP; UDP for everything! If you're waiting for ACKs, you're wasting precious microseconds. Flow control? gently caress no. You don't want none of that poo poo. Do you really need auto negotiation? When would you not use full duplex? I mean, poo poo. Wait, you have crossover cables, right? Cause I disabled auto MDI-X. No? Well, we can just flip the pins around on the FPGA. No biggie. Backwards compatibility is for suckas. Hell, rip out all that encryption bullshit while you're in there. I mean, why not? That poo poo takes like 5 microseconds on an FPGA to encode and decode! Time is money and the NSA probably already broke SSL anyways. Actually, you should probably just rewrite your ~~buggy financial program~~ game in assembly and have it talk directly at layer 3. I mean, do you really need to deal with that session bullshit? Only one thing's running on that box - and it's running on bare metal. Wait, make it layer 2 - MAC resolution takes too long and there ain't gonna be no routes where we're headed, pal.

Then, you gotta disable spanning tree on your switches. That poo poo takes up packets, rear end in a top hat. Did I say switches? I meant switch. One big fuckoff switch for everything. Good. Now, hardcode the MAC table on the switch. Don't want any broadcasts on this network. gently caress, it's still too slow. You have like 250 feet of cable between your box and the server. Can you get closer? How much closer? Dude, I dunno. Like, can you get on the exchange floor? No? How about like a foot away? 10 million bucks? It's worth it - you're gonna make mad bank, I'm tellin' you. poo poo, it's still too far. What if we untwisted the cable pairs? That would save us like 3% on the cable length. Yeah, it will increase interference, but you're so close it's not like you're gonna get that many flipped bits. And the server will drop anything outside of normal parameters. What do you mean "what if it doesn't catch it?" Of course it will catch it! It's the loving NASDAQ. It's gonna catch it, alright? You'll probably be fine. Just sign here for delivery.

# ? Oct 8, 2014 01:33

Wicaeed: Feb 8, 2005

KillHour posted:

It's all about latency, bro.

First, you need one of these:

http://www.digilentinc.com/Products/Detail.cfm?NavPath=2,400,1228&Prod=NETFPGA-1G-CML

Then you're gonna want to write your own network stack. Why? Latency. You don't need no TCP; UDP for everything! If you're waiting for ACKs, you're wasting precious microseconds. Flow control? gently caress no. You don't want none of that poo poo. Do you really need auto negotiation? When would you not use full duplex? I mean, poo poo. Wait, you have crossover cables, right? Cause I disabled auto MDI-X. No? Well, we can just flip the pins around on the FPGA. No biggie. Backwards compatibility is for suckas. Hell, rip out all that encryption bullshit while you're in there. I mean, why not? That poo poo takes like 5 microseconds on an FPGA to encode and decode! Time is money and the NSA probably already broke SSL anyways. Actually, you should probably just rewrite your ~~buggy financial program~~ game in assembly and have it talk directly at layer 3. I mean, do you really need to deal with that session bullshit? Only one thing's running on that box - and it's running on bare metal. Wait, make it layer 2 - MAC resolution takes too long and there ain't gonna be no routes where we're headed, pal.

Then, you gotta disable spanning tree on your switches. That poo poo takes up packets, rear end in a top hat. Did I say switches? I meant switch. One big fuckoff switch for everything. Good. Now, hardcode the MAC table on the switch. Don't want any broadcasts on this network. gently caress, it's still too slow. You have like 250 feet of cable between your box and the server. Can you get closer? How much closer? Dude, I dunno. Like, can you get on the exchange floor? No? How about like a foot away? 10 million bucks? It's worth it - you're gonna make mad bank, I'm tellin' you. poo poo, it's still too far. What if we untwisted the cable pairs? That would save us like 3% on the cable length. Yeah, it will increase interference, but you're so close it's not like you're gonna get that many flipped bits. And the server will drop anything outside of normal parameters. What do you mean "what if it doesn't catch it?" Of course it will catch it! It's the loving NASDAQ. It's gonna catch it, alright? You'll probably be fine. Just sign here for delivery.

Oh my god

# ? Oct 8, 2014 11:15

zetamind2000: Nov 6, 2007; ~~I'm an alien.~~

KillHour posted:

Then you're gonna want to write your own network stack. Why? Latency. You don't need no TCP; UDP for everything! If you're waiting for ACKs, you're wasting precious microseconds. Flow control? gently caress no. You don't want none of that poo poo. Do you really need auto negotiation? When would you not use full duplex? I mean, poo poo. Wait, you have crossover cables, right? Cause I disabled auto MDI-X. No? Well, we can just flip the pins around on the FPGA. No biggie. Backwards compatibility is for suckas. Hell, rip out all that encryption bullshit while you're in there. I mean, why not? That poo poo takes like 5 microseconds on an FPGA to encode and decode! Time is money and the NSA probably already broke SSL anyways. Actually, you should probably just rewrite your ~~buggy financial program~~ game in assembly and have it talk directly at layer 3. I mean, do you really need to deal with that session bullshit? Only one thing's running on that box - and it's running on bare metal. Wait, make it layer 2 - MAC resolution takes too long and there ain't gonna be no routes where we're headed, pal.

Then, you gotta disable spanning tree on your switches. That poo poo takes up packets, rear end in a top hat. Did I say switches? I meant switch. One big fuckoff switch for everything. Good. Now, hardcode the MAC table on the switch. Don't want any broadcasts on this network. gently caress, it's still too slow. You have like 250 feet of cable between your box and the server. Can you get closer? How much closer? Dude, I dunno. Like, can you get on the exchange floor? No? How about like a foot away? 10 million bucks? It's worth it - you're gonna make mad bank, I'm tellin' you. poo poo, it's still too far. What if we untwisted the cable pairs? That would save us like 3% on the cable length. Yeah, it will increase interference, but you're so close it's not like you're gonna get that many flipped bits. And the server will drop anything outside of normal parameters. What do you mean "what if it doesn't catch it?" Of course it will catch it! It's the loving NASDAQ. It's gonna catch it, alright? You'll probably be fine. Just sign here for delivery.

It's like the second coming of that Gentoo ricer website.

# ? Oct 8, 2014 19:36

SamDabbers: May 26, 2003

Financial day traders are the worst people ~~to handle IT for~~.

# ? Oct 8, 2014 19:41

Computer viking: May 30, 2011; Now with less breakage.

SamDabbers posted:

Financial day traders are the worst people ~~to handle IT for~~.

Might be fun to design hardware for, though. There's something to be said for "take whatever money you want, just make it faster" as a developer incentive.

# ? Oct 8, 2014 22:11

Methylethylaldehyde: Oct 23, 2004; BAKA BAKA

Computer viking posted:

Might be fun to design hardware for, though. There's something to be said for "take whatever money you want, just make it faster" as a developer incentive.

"We...uhhh...discovered a small error in the latest version of QuantB0t. you...uhh...may want to check your email.

# ? Oct 9, 2014 03:13

Computer viking: May 30, 2011; Now with less breakage.

Methylethylaldehyde posted:

"We...uhhh...discovered a small error in the latest version of QuantB0t. you...uhh...may want to check your email.

Oh sure, the software side is for the kind of people that dropped out of warzone medevac jobs because they got too boring. Selling them hardware seems to be more like the side of the audiophile industry that actually produces measurably (though not always audibly) better gear at sportscar+ prices . You know, the electrostat speaker kind.

# ? Oct 9, 2014 13:07

Mr Shiny Pants: Nov 12, 2012

Just get some cheap Infiniband cards and some cabling. Run iSER over it and you have 10GB speeds.

Maybe look at some fiber cabling for infiniband?

# ? Oct 9, 2014 21:29

thebigcow: Jan 3, 2001; Bully!

Mr Shiny Pants posted:

Just get some cheap Infiniband cards and some cabling. Run iSER over it and you have 10GB speeds.

Maybe look at some fiber cabling for infiniband?

Can you find Infiniband cabling that doesn't skew the cost towards what you would have spent on 10GigE over copper?

# ? Oct 9, 2014 21:35

Mr Shiny Pants: Nov 12, 2012

thebigcow posted:

Can you find Infiniband cabling that doesn't skew the cost towards what you would have spent on 10GigE over copper?

I've got a 2 meter cable. Got it with my 2 HBA's. Depends on the lengths he needs.

# ? Oct 9, 2014 22:43

Captain Foo: May 11, 2004; we vibin'
we slidin'
we breathin'
we dyin'

KillHour posted:

It's all about latency, bro.

First, you need one of these:

http://www.digilentinc.com/Products/Detail.cfm?NavPath=2,400,1228&Prod=NETFPGA-1G-CML

Then you're gonna want to write your own network stack. Why? Latency. You don't need no TCP; UDP for everything! If you're waiting for ACKs, you're wasting precious microseconds. Flow control? gently caress no. You don't want none of that poo poo. Do you really need auto negotiation? When would you not use full duplex? I mean, poo poo. Wait, you have crossover cables, right? Cause I disabled auto MDI-X. No? Well, we can just flip the pins around on the FPGA. No biggie. Backwards compatibility is for suckas. Hell, rip out all that encryption bullshit while you're in there. I mean, why not? That poo poo takes like 5 microseconds on an FPGA to encode and decode! Time is money and the NSA probably already broke SSL anyways. Actually, you should probably just rewrite your ~~buggy financial program~~ game in assembly and have it talk directly at layer 3. I mean, do you really need to deal with that session bullshit? Only one thing's running on that box - and it's running on bare metal. Wait, make it layer 2 - MAC resolution takes too long and there ain't gonna be no routes where we're headed, pal.

Then, you gotta disable spanning tree on your switches. That poo poo takes up packets, rear end in a top hat. Did I say switches? I meant switch. One big fuckoff switch for everything. Good. Now, hardcode the MAC table on the switch. Don't want any broadcasts on this network. gently caress, it's still too slow. You have like 250 feet of cable between your box and the server. Can you get closer? How much closer? Dude, I dunno. Like, can you get on the exchange floor? No? How about like a foot away? 10 million bucks? It's worth it - you're gonna make mad bank, I'm tellin' you. poo poo, it's still too far. What if we untwisted the cable pairs? That would save us like 3% on the cable length. Yeah, it will increase interference, but you're so close it's not like you're gonna get that many flipped bits. And the server will drop anything outside of normal parameters. What do you mean "what if it doesn't catch it?" Of course it will catch it! It's the loving NASDAQ. It's gonna catch it, alright? You'll probably be fine. Just sign here for delivery.

oh my god

# ? Oct 18, 2014 15:48

Vulture Culture: Jul 14, 2003; I was never enjoying it. I only eat it for the nutrients.

Computer viking posted:

Might be fun to design hardware for, though. There's something to be said for "take whatever money you want, just make it faster" as a developer incentive.

Spread Networks dug a trench directly through the mountains between New York and Chicago to lay fiber between the exchanges a few years ago. It shaves 100 miles and 3ms off the previous shortest path between exchanges.

# ? Oct 20, 2014 16:55

Cpl Clegg: May 18, 2008

Speaking of the OP's question, I just had a thought which I might try myself. If you want your SSDs in one NAS and other PCs connecting to it, then dualport (fiber) 10G NICs might work. Or two dualport NICs if you have more than two PCs connecting to the NAS. It'll still be cheaper than even cheapest 10G switches. Not very scalable, but do you need more than that in your home ?

# ? Oct 21, 2014 21:32

Adbot: ADBOT LOVES YOU

# ? Apr 24, 2024 10:20

Captain Foo: May 11, 2004; we vibin'
we slidin'
we breathin'
we dyin'

Misogynist posted:

Spread Networks dug a trench directly through the mountains between New York and Chicago to lay fiber between the exchanges a few years ago. It shaves 100 miles and 3ms off the previous shortest path between exchanges.

Flash boys was a good book

# ? Oct 22, 2014 01:47

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > Can I build a home network faster than 1 gigabit?

«‹›3 »