Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams
Absolute worst case I can shut everything off on the ESX3 server and import it into the new server. I was hoping to do it magically and have no downtime, but welp.

Adbot
ADBOT LOVES YOU

Dilbert As FUCK
Sep 8, 2007

by Cowcaster
Pillbug
http://www.supermicro.com/products/nfo/FatTwin.cfm

Just purely asking, does anyone use or know people who use super micro? I can't help stop thinking how awesome that FatTwin looks, then I found out the price and can't stop thinking about it

Nukelear v.2
Jun 25, 2004
My optional title text

Corvettefisher posted:

http://www.supermicro.com/products/nfo/FatTwin.cfm

Just purely asking, does anyone use or know people who use super micro? I can't help stop thinking how awesome that FatTwin looks, then I found out the price and can't stop thinking about it

Off the top of my head Etsy runs almost entirely on Supermicro. We've used them and I have no complaints. Edit: Not for virtualization though.

http://codeascraft.etsy.com/2012/08/31/what-hardware-powers-etsy-com/

Nukelear v.2 fucked around with this message at 18:02 on Sep 12, 2012

Dilbert As FUCK
Sep 8, 2007

by Cowcaster
Pillbug
Thanks either way! I hope some other goons have experiences or know some good reviews that would be awesome. Not sure about production but maybe a fat twin 2u with 4 servers for a demo/dev lab environment, or PoC for some clients.

Docjowles
Apr 9, 2009

Corvettefisher posted:

Thanks either way! I hope some other goons have experiences or know some good reviews that would be awesome. Not sure about production but maybe a fat twin 2u with 4 servers for a demo/dev lab environment, or PoC for some clients.

Seems like they'd be really sweet for any application where you don't need a "you fuckers better have a replacement part in my hands within 4 hours" support contract, or tons of direct-attached storage. A big farm of totally interchangeable stateless web servers like Etsy runs is pretty much an ideal use case for commodity parts.

It would be pretty funny to plunk down a 2U fat twin on a client's desk and say "this vSphere cluster will replace your 2 racks full of ancient underutilized machines" v:v:v

bull3964
Nov 18, 2000

DO YOU HEAR THAT? THAT'S THE SOUND OF ME PATTING MYSELF ON THE BACK.


Corvettefisher posted:

http://www.supermicro.com/products/nfo/FatTwin.cfm

Just purely asking, does anyone use or know people who use super micro? I can't help stop thinking how awesome that FatTwin looks, then I found out the price and can't stop thinking about it

We do (or, I should say did), through a Supermicro OEM, Silicon Mechanics.

Since I've taken over the purchasing decisions, they aren't going to be used for production systems anymore. They have way too many random quirks and support is scattered across the board.

If I need to update a driver on a dell system, I punch in the service code into the website and I get the right drivers. To update it on one of the supermicro systems, I need to find out what hardware flavor they were putting into the machine at that point, track down the manufacturer's website, and download the driver from there (knowing all the while that the driver was never fully validated with the rest of the hardware that was shipped with the machine.)

As lab machines, fine. If you are going to be buying 10 racks worth of the same machine and throwing the same OS image on all of them, you should be fine too. If you have a large support staff, you may be ok as well.

However, if you are a smaller shop with limited management resources, run the gently caress away. I've been bitten no less than 5 times this year alone with critical hardware failures that were nebulous as hell to track down and resolve. Two of our machines of the same type like to dump their network connection randomly and won't come back until after you do a hard boot (as in, complete power off, not soft reset.) I can't even tell you how many MegaRAID cards we've had to RMA. We've also gone through at least 3 backplanes on various servers (one of them leading to massive drive corruption when it decided to go.) Every single piece of hardware had to be ordered and shipped with a lead time of 4-7 days.

Also, the built in IPMI is comical compared to the iDrac7 Enterprise that you get on the 12th gen poweredge servers.

Dilbert As FUCK
Sep 8, 2007

by Cowcaster
Pillbug

Docjowles posted:

Seems like they'd be really sweet for any application where you don't need a "you fuckers better have a replacement part in my hands within 4 hours" support contract, or tons of direct-attached storage. A big farm of totally interchangeable stateless web servers like Etsy runs is pretty much an ideal use case for commodity parts.

It would be pretty funny to plunk down a 2U fat twin on a client's desk and say "this vSphere cluster will replace your 2 racks full of ancient underutilized machines" v:v:v

From what I hear they only do next day however the post I saw somewhere else was from 2009, I believe they would do that or at least overnight which isn't so bad with vmware HA capabilities.

The ability to show how much consolidation can be done is also great, and being able to say "What? You mean X company didn't bring in a data center for you to try out?" Could be a HUGE selling point.

bull3964 posted:

We do (or, I should say did), through a Supermicro OEM, Silicon Mechanics.

Since I've taken over the purchasing decisions, they aren't going to be used for production systems anymore. They have way too many random quirks and support is scattered across the board.

If I need to update a driver on a dell system, I punch in the service code into the website and I get the right drivers. To update it on one of the supermicro systems, I need to find out what hardware flavor they were putting into the machine at that point, track down the manufacturer's website, and download the driver from there (knowing all the while that the driver was never fully validated with the rest of the hardware that was shipped with the machine.)

As lab machines, fine. If you are going to be buying 10 racks worth of the same machine and throwing the same OS image on all of them, you should be fine too. If you have a large support staff, you may be ok as well.

However, if you are a smaller shop with limited management resources, run the gently caress away. I've been bitten no less than 5 times this year alone with critical hardware failures that were nebulous as hell to track down and resolve. Two of our machines of the same type like to dump their network connection randomly and won't come back until after you do a hard boot (as in, complete power off, not soft reset.) I can't even tell you how many MegaRAID cards we've had to RMA. We've also gone through at least 3 backplanes on various servers (one of them leading to massive drive corruption when it decided to go.) Every single piece of hardware had to be ordered and shipped with a lead time of 4-7 days.

Also, the built in IPMI is comical compared to the iDrac7 Enterprise that you get on the 12th gen poweredge servers.

<3 Dell servers. I feel your pain with the MegaRAID cards, UCS servers have them as stock I believe. Oh you wanna stop responding for 5 minutes controller? Any reason why? No? You just felt like it! how nice of you...

This would most likely only be for labs and demos, and just the ability to say "Oh I have a datacenter you guys can play with for X days" I am sure I could amaze some clients by going "Here is a VDI, Domain Controllers, SQL, Sharepoint and Exchange. Now watch what happens when I cut a server off, Neat huh?"

Dilbert As FUCK fucked around with this message at 19:34 on Sep 12, 2012

Kachunkachunk
Jun 6, 2011
Supermicro is unfortunately going to bring you some trouble if your virtualize on it (well, VMware anyway, from experience). Very random issues/quirks.

luminalflux
May 27, 2005



Corvettefisher posted:

http://www.supermicro.com/products/nfo/FatTwin.cfm

Just purely asking, does anyone use or know people who use super micro? I can't help stop thinking how awesome that FatTwin looks, then I found out the price and can't stop thinking about it

We used to like 5 years ago, and ran the gently caress away after waaaay too many random issues.

Mierdaan
Sep 14, 2004

Pillbug
Anyone updated to vCenter or ESXi 5.1 yet? Any trip reports?

Dilbert As FUCK
Sep 8, 2007

by Cowcaster
Pillbug

Mierdaan posted:

Anyone updated to vCenter or ESXi 5.1 yet? Any trip reports?

Lab yes, otherwise no I haven't gotten keys for it yet. I do like the new 'simple install'

Even if it did crash the first time

three
Aug 9, 2007

i fantasize about ndamukong suh licking my doodoo hole

Mierdaan posted:

Anyone updated to vCenter or ESXi 5.1 yet? Any trip reports?

I've been using the beta in my home lab for a while with no issues. There's a bug with the EMC VMAX when creating VMFS-5 datastores if not running a certain code level, and it's also not supported with any versions of View. Those have been the only two things I've ran in to.

Moey
Oct 22, 2010

I LIKE TO MOVE IT
My standalone lab server doesn't want to seem to upgrade.

- Downloaded VMware-ESXi-5.1.0-799733-depot.zip from VMware.
- Placed it on the local datastore.
- Tried to install
code:
esxcli software vib install --depot=/vmfs/volumes/datastorename/VMware-ESXi-5.1.0-799733-depot.zip
-Results in saying it cannot find index.xml in the zip file even though I verified it is there

I can only assume I am hamfisting or screwing up something.

Dilbert As FUCK
Sep 8, 2007

by Cowcaster
Pillbug
Not to sure about that, any reason not to use vum or are you just trying to do it cli for the experience?

Actually try
esxcli software vib update --depot /vmfs/volumes/<dataStoreName>/<folderName>/VMware-ESXi-5.1.0-799733-depot.zip

minus the =

Dilbert As FUCK fucked around with this message at 21:38 on Sep 13, 2012

Moey
Oct 22, 2010

I LIKE TO MOVE IT

Corvettefisher posted:

Not to sure about that, any reason not to use vum or are you just trying to do it cli for the experience?

Actually try
esxcli software vib update --depot /vmfs/volumes/<dataStoreName>/<folderName>/VMware-ESXi-5.1.0-799733-depot.zip

minus the =

The server is a standalone server not attached to vCenter.

Tried it without the = as well (google showed use of both = and a space).

Edit: The checksum didn't match up. Just redownloaded and it worked fine. I think that's the first time I have had a file get borked from a download.

Moey fucked around with this message at 21:48 on Sep 13, 2012

Mierdaan
Sep 14, 2004

Pillbug
How long should I spend troubleshooting terrible datastore latency when using Dell's R610 integrated broadcom NICs before I just replace them? Do these things work OK for anyone?

Moey
Oct 22, 2010

I LIKE TO MOVE IT

Mierdaan posted:

How long should I spend troubleshooting terrible datastore latency when using Dell's R610 integrated broadcom NICs before I just replace them? Do these things work OK for anyone?

I have never had any problem with ours. For almost two years they have been used in multiple roles without issue (management, VMNetwork, iSCSI and vMotion).

Is it only with storage you are having issues, or is there bad latency when used for other traffic?

Dilbert As FUCK
Sep 8, 2007

by Cowcaster
Pillbug

Mierdaan posted:

How long should I spend troubleshooting terrible datastore latency when using Dell's R610 integrated broadcom NICs before I just replace them? Do these things work OK for anyone?

Are they broadcom by chance? I have never had good perfromance when compared to intel or the cisco VICs. Furthermore I really don't use onboard for storage, maybe as a failback, but not primary for storage.

What kind of performance issues are you having?

Erwin
Feb 17, 2006

Mierdaan posted:

How long should I spend troubleshooting terrible datastore latency when using Dell's R610 integrated broadcom NICs before I just replace them? Do these things work OK for anyone?
No issues here.

Corvettefisher posted:

Are they broadcom by chance?
Really?

Dilbert As FUCK
Sep 8, 2007

by Cowcaster
Pillbug
????

I have had a history of issues with broadcom drivers and esxi not working right. Is that not the case for you?

Erwin
Feb 17, 2006

No, he asked if anyone has issues specifically with Broadcom NICs and you asked if they were Broadcom NICs. You seem like you're always in a hurry to post post post regardless of whether it'll prove helpful. You live up to your title, for sure.

Dilbert As FUCK
Sep 8, 2007

by Cowcaster
Pillbug
Wow go me :/

Oh well class is starting time try and help some students

Dilbert As FUCK fucked around with this message at 23:07 on Sep 13, 2012

Moey
Oct 22, 2010

I LIKE TO MOVE IT

Moey posted:

The server is a standalone server not attached to vCenter.

Tried it without the = as well (google showed use of both = and a space).

Edit: The checksum didn't match up. Just redownloaded and it worked fine. I think that's the first time I have had a file get borked from a download.

Welp. Updating from the latest 5.0 to 5.1 seemed screw this thing up.

Update ran fine, needed to reboot after. Reboot server. It comes back online (pingable) but cannot get in with the vSphere client (server could not interpret the communication from the client error). Try pulling up the servers IP in a browser, I just get "503 Service Unavailable".

Good thing this is just a test/dev box (although I do need to get it back online for other testing).

Edit: Booting into recovery and rolling back the version fixed it right up.

I should probably double check to see if this aging hardware is still supported in 5.1.

Double Edit: PowerEdge 2950s are still supported. Will investigate later.

Triple Edit: Wowza, the bios on that server is extremely old, probably should update that.

Moey fucked around with this message at 23:43 on Sep 13, 2012

Mierdaan
Sep 14, 2004

Pillbug

Moey posted:

I have never had any problem with ours. For almost two years they have been used in multiple roles without issue (management, VMNetwork, iSCSI and vMotion).

Is it only with storage you are having issues, or is there bad latency when used for other traffic?

My issue is seeing the normal 5-15ms latency on my Compellent disks, but crap like this at the datastore level.



This is with 2x onboard Broadcom interfaces configured as I was describing earlier in the thread (look at post history for screenshots).

code:
# ethtool -i vmnic2
driver: bnx2
version: 2.0.15g.v50.11-5vmw
firmware-version: 5.2.7 bc 5.2.2 NCSI 2.0.8
bus-info: 0000:02:00.0
# ethtool -i vmnic3
driver: bnx2
version: 2.0.15g.v50.11-5vmw
firmware-version: 5.2.7 bc 5.2.2 NCSI 2.0.8
bus-info: 0000:02:00.1
code:
Switch Ports Model              SW Version            SW Image
------ ----- -----              ----------            ----------
*    1 30    WS-C3560X-24       12.2(53)SE2           C3560E-UNIVERSALK9-M

...

!
interface GigabitEthernet0/15
 description Uplink to VMware host
 flowcontrol receive desired
 spanning-tree portfast
!

peepsalot
Apr 24, 2007

        PEEP THIS...
           BITCH!

Stop yelling at your hard drives
https://www.youtube.com/watch?v=tDacjrSCeq4

Dilbert As FUCK
Sep 8, 2007

by Cowcaster
Pillbug

Mierdaan posted:

usefull stuff

That driver version is mighty old, from what I can tell from some google fu'ing it is from about a year ago.(going from this thread http://communities.vmware.com/thread/327921?start=15&tstart=0 )

http://www.dell.com/support/drivers/us/en/04/Product/poweredge-r610
Should get you the latest broadcom drivers, if you haven't already done so.

You also might want to check out what esxtop, and powercli in general, is reporting not just the vsphere client.

Does this problem occur on other 2 hosts with the same adapters/setup?

Mierdaan
Sep 14, 2004

Pillbug
Yeah updating the drivers was already on my agenda for tomorrow.

esxtop shows DAVG/rd and DAVG/wr spiking to >100 pretty regularly, so that matches the performance graphs. And yeah it's all 3 cluster hosts.

Dilbert As FUCK
Sep 8, 2007

by Cowcaster
Pillbug
Blew off a date this weekend so I can sit my rear end down, study more, and rewriting the lovely OP I did.

Also ordering some vitamins to help with neurological and critical thinking so I can be less dumb, and reformatting to Centos, and selling my 6970 so instead of playing the occasional video game on the weekends I will just study.

I am leaving the first bullet the same, and revising 2 down.


Open to criticism(I know there is a lot), suggestions, and talks about the weather. Mostly gearing the OP to VMware since I really don't see people talking about Xen, or Hyper-V much.

Syano
Jul 13, 2005

Corvettefisher posted:

Blew off a date this weekend so I can sit my rear end down, study more, and rewriting the lovely OP I did.

Also ordering some vitamins to help with neurological and critical thinking so I can be less dumb, and reformatting to Centos, and selling my 6970 so instead of playing the occasional video game on the weekends I will just study.

I am leaving the first bullet the same, and revising 2 down.


Open to criticism(I know there is a lot), suggestions, and talks about the weather. Mostly gearing the OP to VMware since I really don't see people talking about Xen, or Hyper-V much.
dude, we told you on the first page. Hyperv is not a host based hypervisor

Dilbert As FUCK
Sep 8, 2007

by Cowcaster
Pillbug

Syano posted:

dude, we told you on the first page. Hyperv is not a host based hypervisor

Fixing it, I did a straight copy and paste of the first paragraph guess I just never fixed it

Dilbert As FUCK fucked around with this message at 04:14 on Sep 14, 2012

luminalflux
May 27, 2005



Mierdaan posted:

How long should I spend troubleshooting terrible datastore latency when using Dell's R610 integrated broadcom NICs before I just replace them? Do these things work OK for anyone?

I used to build network hardware at my previous gig. We bought Broadcom PHYs and implemented our own MAC in VHDL. No, of course they didn't work for us and we got little documentation and support despite paying boatloads for the PHYs. gently caress Broadcom.

Corvettefisher posted:

Fixing it, I did a straight copy and paste of the first paragraph guess I just never fixed it

Should at least mention VirtualBox and Vagrant (get someone else to write about this). Maybe some stuff about Xen, KVM, Solaris/SmartOS Zones perhaps?
edit: And then there's stuff like OpenVZ as well.

luminalflux fucked around with this message at 12:47 on Sep 14, 2012

JHVH-1
Jun 28, 2002

Corvettefisher posted:

http://www.supermicro.com/products/nfo/FatTwin.cfm

Just purely asking, does anyone use or know people who use super micro? I can't help stop thinking how awesome that FatTwin looks, then I found out the price and can't stop thinking about it

We used them all the time at my last job. They are good if you need bargain systems and have a decent vendor that builds and tests them properly. Sometimes when a new machine would come out we would have a few kinks to work out for things like BIOS settings or backplane switches cause the vendor just slapped things together (probably cause my boss always wanted things like ASAP and not having any testing done first)

Bob Morales
Aug 18, 2006


Just wear the fucking mask, Bob

I don't care how many people I probably infected with COVID-19 while refusing to wear a mask, my comfort is far more important than the health and safety of everyone around me!

CentOS hogs 100% of the CPU in VirtualBox unless you select ICH9 as the motherboard type instead of PIIX - even though it will bitch at you for being an sub-optimal setting. No wonder it took a minute just to boot up.

Also, install the dkms package from RPMForge or you'll have to re-install the VBOXADDITIONS every time yum upgrades the kernel.

Dilbert As FUCK
Sep 8, 2007

by Cowcaster
Pillbug
Of course black mesa source comes out this weekend...

Oh well testing out the guest OS reclaimation, hopefully it doesn't destroy partitions and vmdk's like in the first release of it.

Mierdaan
Sep 14, 2004

Pillbug
How do you normally update the firmware on Broadcom NICs in an ESXi host, given that everyone seems to package them for Windows and Linux only? CentOS live CD?

A VMware support rep literally just suggested to me, twice, that I create a Windows guest and run the firmware update utility in there.

Syano
Jul 13, 2005
Dell server? Should have a boot cd you can use to update your firmware. Stick your service tag in the support site and you can download the latest copy if you do not have it

BTW. This is one the myriad reasons you dont use supermicro unless you really know what you are getting in to

Mierdaan
Sep 14, 2004

Pillbug

Syano posted:

Dell server? Should have a boot cd you can use to update your firmware. Stick your service tag in the support site and you can download the latest copy if you do not have it

BTW. This is one the myriad reasons you dont use supermicro unless you really know what you are getting in to

Yeah they're Dell. Thanks for reminding me that Dell puts those out.

late-edit: actually the Dell Repo Manager tool doesn't even get the latest version of the Broadcom firmware. They actually recommend the liveCD method, specifically using the OMSA LiveCD and using the RedHat .bin driver packages off the Dell site.

Mierdaan fucked around with this message at 21:26 on Sep 14, 2012

Vulture Culture
Jul 14, 2003

I was never enjoying it. I only eat it for the nutrients.

Mierdaan posted:

late-edit: actually the Dell Repo Manager tool doesn't even get the latest version of the Broadcom firmware.
This is the case for literally every vendor that gives a poo poo about their product not being unstable to the point of being a 2-amp brick (talk to me about Emulex!)

Mierdaan
Sep 14, 2004

Pillbug
I'm sorry, did I stress enough that a loving VMware tech recommended I update my NIC firmware by running the firmware update tool in a Windows guest VM?

Adbot
ADBOT LOVES YOU

1000101
May 14, 2003

BIRTHDAY BIRTHDAY BIRTHDAY BIRTHDAY BIRTHDAY BIRTHDAY FRUITCAKE!

Mierdaan posted:

I'm sorry, did I stress enough that a loving VMware tech recommended I update my NIC firmware by running the firmware update tool in a Windows guest VM?

Just use VMDirectPath!!

Not serious and I'm not surprised. VMware support has been very spotty lately.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply