Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us $3,400 per month for bandwidth bills alone, and since we don't believe in shoving popup ads to our registered users, we try to make the money back through forum registrations.
«3 »
  • Locked thread
KS
Jun 10, 2003


Outrageous Lumpwad

I've been selling a bunch of X520-DA2s on Ebay. Guess since there's continued interest I'll try some in SA-Mart instead.

Besides the brand of twinax cables, another thing to watch out for is length. The cables move from passive to active electronics above 5M, and at least for the Intel X520 you need the Windows driver toolset installed to use an active cable. Last I looked, there was no way to use an active cable in ESXi, for instance.

Adbot
ADBOT LOVES YOU

Computer viking
May 30, 2011
Now with less breakage.

I'm on the wrong continent (and customs handling is insane on anything above ~ $30), otherwise that would be quite interesting.

Methylethylaldehyde
Oct 23, 2004

BAKA BAKA


KS posted:

I've been selling a bunch of X520-DA2s on Ebay. Guess since there's continued interest I'll try some in SA-Mart instead.

Besides the brand of twinax cables, another thing to watch out for is length. The cables move from passive to active electronics above 5M, and at least for the Intel X520 you need the Windows driver toolset installed to use an active cable. Last I looked, there was no way to use an active cable in ESXi, for instance.

How much are you charging for one?

Lowen SoDium
Jun 5, 2003

Highen Fiber


Clapping Larry

Potato Alley posted:

Well the other problem is that the switch makers are always playing catchup with blocking the third parties, which means that a new release of software will inevitably render some previously-compatible third-party SFPs incompatible. Depending on how often you update your switches this may or may not be a problem, but if you have a large deployment of third-party SFPs and upgrade, suddenly that "free" switch upgrade (which these days is less and less free, at least with Cisco) just cost you a mint in new SFPs. Plus potential downtime and complete confusion if you're not aware of the issue, since if you have a mix of SFPs, SOME may stop working and others may be fine, and you'll have no idea why.

And of course the companies say up front "we absolutely don't support third party SFPs because they could not work". Hey listen. I get that. The third party stuff may not be held to the same quality standards. But don't tell me that the supposed quality difference is worth literally 10 times as much. When it's clearly just gouging me is when I get pissed off and start calling it a racket. (Also, I've never heard of any real quality differences in these things in terms of packet errors or speed issues).

It's the same issue as Ford or Toyota claiming you need to use their parts and only their parts, and I believe there were laws passed to prevent them from disallowing warranty service etc. Though maybe I'm misremembering. But there's certainly nothing currently to prevent switch makers from doing it, until RJ-45 becomes more prevalent in 10 GBe switches.


For Cisco:


service unsupported-transceiver
no errdisable detect cause gbic-invalid

KillHour
Oct 28, 2007

Wake up and
smell the murder.


http://www.newegg.com/Product/Produ...N82E16833122529

http://www.newegg.com/Product/Produ...N82E16833122542

Finding NICs is left as an exercise to the reader.

Combat Pretzel
Jun 23, 2004

No, seriously... what kurds?!

gggiiimmmppp posted:

In the end I learned that while the iSCSI extent can be allowed to grow dynamically, it cannot be shrunk again so after bloating it up to 2tb shuffling temporary stuff around I was faced with having to delete the whole disk and start over to get the space back so I just said gently caress it and have put up with a measly 1gbps ever since.
To get back on this, since I just read the FreeNAS forums, apparently the current iSCSI target service is to blame. istgt can't punch holes, whereas CTL seemingly can. Latter will be default in 9.3 (which is however still in development). I've just tested it, creating an iSCSI extent, put a NTFS volume on it, copied a 600MB ISO file on it, caused the ZVOL pool usage to raise to around 600MB, deleted the ISO, ZVOL pool usage dropped down to 1.7MB. So if you want TRIM/DISCARD, 9.3+ is the way to go.

Computer viking
May 30, 2011
Now with less breakage.

Or 10 - keep in mind that -CURRENT is 11 these days.

Combat Pretzel
Jun 23, 2004

No, seriously... what kurds?!

If you go the straight FreeBSD route instead of FreeNAS. Latter said to expect the FreeBSD 10 based version of their stuff for next summer only.

I'd like to go with FreeBSD direct, but I can't be assed to deal with Samba 4.

Computer viking
May 30, 2011
Now with less breakage.

Combat Pretzel posted:

If you go the straight FreeBSD route instead of FreeNAS. Latter said to expect the FreeBSD 10 based version of their stuff for next summer only.

I'd like to go with FreeBSD direct, but I can't be assed to deal with Samba 4.

I've set it up a few times at work - and yeah, assuming you need the things that aren't in 3.6, setting it up is one of those things best avoided.
(In my case, I'm using a windows 2003 R2 AD domain that I don't have domain admin access to for authenticating users. Ever so fun.)

That said, you could just copy the config files over from your current setup. (smb.conf, at least; perhaps the kerberos stuff if you're doing something unpleasant).

Computer viking fucked around with this message at Sep 19, 2014 around 19:18

Combat Pretzel
Jun 23, 2004

No, seriously... what kurds?!

I've looked some into adapters. Infiniband ones are affordable, but the cables are not, 10GBe adapters are not affordable, but cables are (if you go with RJ45).

Alereon
Feb 6, 2004

Dehumanize yourself and face to Trumpshed

College Slice

I just upgraded my LAN wiring to CAT7 so I'll be ready when 10GBASE-T adapters come down in price Sure the actual reason was that CAT7 wasn't much more expensive and STP holds up better mechanically when installed improperly, but still

Captain Foo
May 11, 2004

the cyberpunk dystopia is now
you are not the runner


Clever Betty

Alereon posted:

I just upgraded my LAN wiring to CAT7 so I'll be ready when 10GBASE-T adapters come down in price Sure the actual reason was that CAT7 wasn't much more expensive and STP holds up better mechanically when installed improperly, but still

Nam Taf
Jun 25, 2005

I am Fat Man, hear me roar!



Combat Pretzel posted:

I'd like to go with FreeBSD direct, but I can't be assed to deal with Samba 4.

So pkg install samba36 not samba4? Have I done something wrong by doing that??

Combat Pretzel
Jun 23, 2004

No, seriously... what kurds?!

The configuration was a pain in the rear end the last time I've tried.

Nam Taf
Jun 25, 2005

I am Fat Man, hear me roar!



Samba36 config is as it's always been. I haven't bothered delving into samba4 yet either but there's ample documentation out there running you through getting a basic 36 setup sorted.

Computer viking
May 30, 2011
Now with less breakage.

IIRC, samba4 isn't too bad (or too different) to set up if you only want to do things you could also do in 3.6. The madness enters when you want to use it as an AD controller.

Combat Pretzel
Jun 23, 2004

No, seriously... what kurds?!

Eh, I thought you would get more performance out of Samba4, because of its support of the newer SMB protocols.

Computer viking
May 30, 2011
Now with less breakage.

Combat Pretzel posted:

Eh, I thought you would get more performance out of Samba4, because of its support of the newer SMB protocols.

That could well be true.

Creizai
Jun 10, 2005
I love Rizien

Straker posted:

http://en.wikipedia.org/wiki/10-gigabit_Ethernet

Newegg has decent enough 10gige NICs for like $400, so guess it depends on whether it's worth it to you? That seems way more useful/less of a headache than bonding and poo poo just for double the bandwidth.

Do you think those 10gige NICs setups would allow for better steam box streaming?

Lowen SoDium
Jun 5, 2003

Highen Fiber


Clapping Larry

Creizai posted:

Do you think those 10gige NICs setups would allow for better steam box streaming?

Better than 1Gb NICs?

No.

Cpl Clegg
May 18, 2008



I'm thinking about used Infiniband switches and HCAs myself as well. Not that I really need it at home (or anyone really) but it'll be an interesting thing to tinker with.

KillHour
Oct 28, 2007

Wake up and
smell the murder.


Creizai posted:

Do you think those 10gige NICs setups would allow for better steam box streaming?

It's all about latency, bro.

First, you need one of these:

http://www.digilentinc.com/Products...=NETFPGA-1G-CML

Then you're gonna want to write your own network stack. Why? Latency. You don't need no TCP; UDP for everything! If you're waiting for ACKs, you're wasting precious microseconds. Flow control? gently caress no. You don't want none of that poo poo. Do you really need auto negotiation? When would you not use full duplex? I mean, poo poo. Wait, you have crossover cables, right? Cause I disabled auto MDI-X. No? Well, we can just flip the pins around on the FPGA. No biggie. Backwards compatibility is for suckas. Hell, rip out all that encryption bullshit while you're in there. I mean, why not? That poo poo takes like 5 microseconds on an FPGA to encode and decode! Time is money and the NSA probably already broke SSL anyways. Actually, you should probably just rewrite your buggy financial program game in assembly and have it talk directly at layer 3. I mean, do you really need to deal with that session bullshit? Only one thing's running on that box - and it's running on bare metal. Wait, make it layer 2 - MAC resolution takes too long and there ain't gonna be no routes where we're headed, pal.

Then, you gotta disable spanning tree on your switches. That poo poo takes up packets, rear end in a top hat. Did I say switches? I meant switch. One big fuckoff switch for everything. Good. Now, hardcode the MAC table on the switch. Don't want any broadcasts on this network. gently caress, it's still too slow. You have like 250 feet of cable between your box and the server. Can you get closer? How much closer? Dude, I dunno. Like, can you get on the exchange floor? No? How about like a foot away? 10 million bucks? It's worth it - you're gonna make mad bank, I'm tellin' you. poo poo, it's still too far. What if we untwisted the cable pairs? That would save us like 3% on the cable length. Yeah, it will increase interference, but you're so close it's not like you're gonna get that many flipped bits. And the server will drop anything outside of normal parameters. What do you mean "what if it doesn't catch it?" Of course it will catch it! It's the loving NASDAQ. It's gonna catch it, alright? You'll probably be fine. Just sign here for delivery.

Wicaeed
Feb 8, 2005


KillHour posted:

It's all about latency, bro.

First, you need one of these:

http://www.digilentinc.com/Products...=NETFPGA-1G-CML

Then you're gonna want to write your own network stack. Why? Latency. You don't need no TCP; UDP for everything! If you're waiting for ACKs, you're wasting precious microseconds. Flow control? gently caress no. You don't want none of that poo poo. Do you really need auto negotiation? When would you not use full duplex? I mean, poo poo. Wait, you have crossover cables, right? Cause I disabled auto MDI-X. No? Well, we can just flip the pins around on the FPGA. No biggie. Backwards compatibility is for suckas. Hell, rip out all that encryption bullshit while you're in there. I mean, why not? That poo poo takes like 5 microseconds on an FPGA to encode and decode! Time is money and the NSA probably already broke SSL anyways. Actually, you should probably just rewrite your buggy financial program game in assembly and have it talk directly at layer 3. I mean, do you really need to deal with that session bullshit? Only one thing's running on that box - and it's running on bare metal. Wait, make it layer 2 - MAC resolution takes too long and there ain't gonna be no routes where we're headed, pal.

Then, you gotta disable spanning tree on your switches. That poo poo takes up packets, rear end in a top hat. Did I say switches? I meant switch. One big fuckoff switch for everything. Good. Now, hardcode the MAC table on the switch. Don't want any broadcasts on this network. gently caress, it's still too slow. You have like 250 feet of cable between your box and the server. Can you get closer? How much closer? Dude, I dunno. Like, can you get on the exchange floor? No? How about like a foot away? 10 million bucks? It's worth it - you're gonna make mad bank, I'm tellin' you. poo poo, it's still too far. What if we untwisted the cable pairs? That would save us like 3% on the cable length. Yeah, it will increase interference, but you're so close it's not like you're gonna get that many flipped bits. And the server will drop anything outside of normal parameters. What do you mean "what if it doesn't catch it?" Of course it will catch it! It's the loving NASDAQ. It's gonna catch it, alright? You'll probably be fine. Just sign here for delivery.

Oh my god

overmind2000
Nov 6, 2007

I'm an alien.


KillHour posted:

Then you're gonna want to write your own network stack. Why? Latency. You don't need no TCP; UDP for everything! If you're waiting for ACKs, you're wasting precious microseconds. Flow control? gently caress no. You don't want none of that poo poo. Do you really need auto negotiation? When would you not use full duplex? I mean, poo poo. Wait, you have crossover cables, right? Cause I disabled auto MDI-X. No? Well, we can just flip the pins around on the FPGA. No biggie. Backwards compatibility is for suckas. Hell, rip out all that encryption bullshit while you're in there. I mean, why not? That poo poo takes like 5 microseconds on an FPGA to encode and decode! Time is money and the NSA probably already broke SSL anyways. Actually, you should probably just rewrite your buggy financial program game in assembly and have it talk directly at layer 3. I mean, do you really need to deal with that session bullshit? Only one thing's running on that box - and it's running on bare metal. Wait, make it layer 2 - MAC resolution takes too long and there ain't gonna be no routes where we're headed, pal.

Then, you gotta disable spanning tree on your switches. That poo poo takes up packets, rear end in a top hat. Did I say switches? I meant switch. One big fuckoff switch for everything. Good. Now, hardcode the MAC table on the switch. Don't want any broadcasts on this network. gently caress, it's still too slow. You have like 250 feet of cable between your box and the server. Can you get closer? How much closer? Dude, I dunno. Like, can you get on the exchange floor? No? How about like a foot away? 10 million bucks? It's worth it - you're gonna make mad bank, I'm tellin' you. poo poo, it's still too far. What if we untwisted the cable pairs? That would save us like 3% on the cable length. Yeah, it will increase interference, but you're so close it's not like you're gonna get that many flipped bits. And the server will drop anything outside of normal parameters. What do you mean "what if it doesn't catch it?" Of course it will catch it! It's the loving NASDAQ. It's gonna catch it, alright? You'll probably be fine. Just sign here for delivery.

It's like the second coming of that Gentoo ricer website.

SamDabbers
May 26, 2003

QUITE.


Fallen Rib

Financial day traders are the worst people to handle IT for.

Computer viking
May 30, 2011
Now with less breakage.

SamDabbers posted:

Financial day traders are the worst people to handle IT for.

Might be fun to design hardware for, though. There's something to be said for "take whatever money you want, just make it faster" as a developer incentive.

Methylethylaldehyde
Oct 23, 2004

BAKA BAKA


Computer viking posted:

Might be fun to design hardware for, though. There's something to be said for "take whatever money you want, just make it faster" as a developer incentive.

"We...uhhh...discovered a small error in the latest version of QuantB0t. you...uhh...may want to check your email.

Computer viking
May 30, 2011
Now with less breakage.

Methylethylaldehyde posted:

"We...uhhh...discovered a small error in the latest version of QuantB0t. you...uhh...may want to check your email.

Oh sure, the software side is for the kind of people that dropped out of warzone medevac jobs because they got too boring. Selling them hardware seems to be more like the side of the audiophile industry that actually produces measurably (though not always audibly) better gear at sportscar+ prices . You know, the electrostat speaker kind.

Mr Shiny Pants
Nov 12, 2012


Just get some cheap Infiniband cards and some cabling. Run iSER over it and you have 10GB speeds.

Maybe look at some fiber cabling for infiniband?

thebigcow
Jan 3, 2001

Bully!

Mr Shiny Pants posted:

Just get some cheap Infiniband cards and some cabling. Run iSER over it and you have 10GB speeds.

Maybe look at some fiber cabling for infiniband?

Can you find Infiniband cabling that doesn't skew the cost towards what you would have spent on 10GigE over copper?

Mr Shiny Pants
Nov 12, 2012


thebigcow posted:

Can you find Infiniband cabling that doesn't skew the cost towards what you would have spent on 10GigE over copper?

I've got a 2 meter cable. Got it with my 2 HBA's. Depends on the lengths he needs.

Captain Foo
May 11, 2004

the cyberpunk dystopia is now
you are not the runner


Clever Betty

KillHour posted:

It's all about latency, bro.

First, you need one of these:

http://www.digilentinc.com/Products...=NETFPGA-1G-CML

Then you're gonna want to write your own network stack. Why? Latency. You don't need no TCP; UDP for everything! If you're waiting for ACKs, you're wasting precious microseconds. Flow control? gently caress no. You don't want none of that poo poo. Do you really need auto negotiation? When would you not use full duplex? I mean, poo poo. Wait, you have crossover cables, right? Cause I disabled auto MDI-X. No? Well, we can just flip the pins around on the FPGA. No biggie. Backwards compatibility is for suckas. Hell, rip out all that encryption bullshit while you're in there. I mean, why not? That poo poo takes like 5 microseconds on an FPGA to encode and decode! Time is money and the NSA probably already broke SSL anyways. Actually, you should probably just rewrite your buggy financial program game in assembly and have it talk directly at layer 3. I mean, do you really need to deal with that session bullshit? Only one thing's running on that box - and it's running on bare metal. Wait, make it layer 2 - MAC resolution takes too long and there ain't gonna be no routes where we're headed, pal.

Then, you gotta disable spanning tree on your switches. That poo poo takes up packets, rear end in a top hat. Did I say switches? I meant switch. One big fuckoff switch for everything. Good. Now, hardcode the MAC table on the switch. Don't want any broadcasts on this network. gently caress, it's still too slow. You have like 250 feet of cable between your box and the server. Can you get closer? How much closer? Dude, I dunno. Like, can you get on the exchange floor? No? How about like a foot away? 10 million bucks? It's worth it - you're gonna make mad bank, I'm tellin' you. poo poo, it's still too far. What if we untwisted the cable pairs? That would save us like 3% on the cable length. Yeah, it will increase interference, but you're so close it's not like you're gonna get that many flipped bits. And the server will drop anything outside of normal parameters. What do you mean "what if it doesn't catch it?" Of course it will catch it! It's the loving NASDAQ. It's gonna catch it, alright? You'll probably be fine. Just sign here for delivery.

oh my god

Vulture Culture
Jul 14, 2003

I was never enjoying it. I only eat it for the nutrients.


Computer viking posted:

Might be fun to design hardware for, though. There's something to be said for "take whatever money you want, just make it faster" as a developer incentive.
Spread Networks dug a trench directly through the mountains between New York and Chicago to lay fiber between the exchanges a few years ago. It shaves 100 miles and 3ms off the previous shortest path between exchanges.

Cpl Clegg
May 18, 2008



Speaking of the OP's question, I just had a thought which I might try myself. If you want your SSDs in one NAS and other PCs connecting to it, then dualport (fiber) 10G NICs might work. Or two dualport NICs if you have more than two PCs connecting to the NAS. It'll still be cheaper than even cheapest 10G switches. Not very scalable, but do you need more than that in your home ?

Adbot
ADBOT LOVES YOU

Captain Foo
May 11, 2004

the cyberpunk dystopia is now
you are not the runner


Clever Betty

Misogynist posted:

Spread Networks dug a trench directly through the mountains between New York and Chicago to lay fiber between the exchanges a few years ago. It shaves 100 miles and 3ms off the previous shortest path between exchanges.

Flash boys was a good book

  • Locked thread
«3 »