Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
theperminator
Sep 16, 2009

by Smythe
Fun Shoe
I'm running ESXI on a HP Microserver with another Microserver as a NAS/ISCSI host,
NFS Datastores work great, I can max out the link easily. but iSCSI doesn't seem to work too well and I was hoping someone could tell me why?
Even with MPIO set to round robin I get 70MB/s read off my raid array which does 300MB/s locally on the nas.
Regardless of whether I'm using 2 paths or 1 to the iSCSI host the performance is terrible.

If I connect the luns using the windows iscsi initiator the performance is great, it will max out the link easily and there are no issues.

Is there something I need to be doing to get better performance out of iscsi targets on ESXI 5?

Adbot
ADBOT LOVES YOU

theperminator
Sep 16, 2009

by Smythe
Fun Shoe

Misogynist posted:

Are your hosts on the same subnet, or is your iSCSI traffic being routed?

Both machines are on the same subnet, with a switch for each path
Path 1 is on 192.168.0.0/24 and Path 2 is on 10.1.1.0/24

KS posted:

70 MB/sec isn't bad for a single gig link, but you should get something out of a second.

With a software iscsi initiator in ESXi you need to do some special setup to make it actually use multiple links to get to one datastore. You should have a vmk interface for each of your physical NICs and then you have to bind them to the vmhba using "esxcli swiscsi nic add -n vmkX -d vmhba33" Do you remember setting this up when you were trying two paths?

Not 100% sure on that as I used the vSphere client to set it up, but it is showing that they're both bound to iSCSI.

theperminator
Sep 16, 2009

by Smythe
Fun Shoe

adorai posted:

If you did it in the gui it's not setup correctly.

Ah, that would be why then
I'll give the commands a go when I have some free time, thanks for the help.

Corvettefisher posted:

Might want to try downloading the esxi verison of the server you have
http://h18004.www1.hp.com/products/servers/vmware/supportmatrix/hpvmware.html

Are those the towers with AMD boards in it(n40l), and if so are you using the onboard nic? Also are the disks thick eager 0 or thin, and what NAS OS are you using for those storage devices? Openfiler, Freenas, or what?

Mines an N36l Microserver, which isn't on the list unfortunately.
My NAS OS is Debian with the SCST daemon backed by a 4x2TB raid array using mdadm.

If the multipathing commands don't help, I'll give openfiler and freenas a shot.
Thanks for your help

theperminator
Sep 16, 2009

by Smythe
Fun Shoe

Sylink posted:

Is there a vCenter iso/installer package for a trial version somewhere on the vmware site? I cannot dig the thing up for the life of me and I want to install it under Win2k8 Server.

You can start a free trial of the vSphere server here: http://www.vmware.com/products/vsphere/mid-size-and-enterprise-business/overview.html

theperminator
Sep 16, 2009

by Smythe
Fun Shoe
Still waiting on an update on the log spew issue with windows vms as mentioned here: http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2036350

I've got a tonne of windows VMs still running tools from 5.0

theperminator
Sep 16, 2009

by Smythe
Fun Shoe
How can I add the ability to my vSphere cluster for switch port ACL's on the networks?

It looks like VShield App has been discontinued, Cisco's 1000v looks like it should do it but needs to be managed by people other than our network engineers.

:edit: it looks like vSphere 5.5 adds this functionality, no idea when that is coming out though.

theperminator fucked around with this message at 06:56 on Sep 4, 2013

theperminator
Sep 16, 2009

by Smythe
Fun Shoe
I don't think you can run OSX under ESXI on a mac anyway, it requires the hypervisor to virtualise access to the hardware TPM in a mac, so you might need to use fakesmc.kext to get it to work

theperminator
Sep 16, 2009

by Smythe
Fun Shoe

NippleFloss posted:

I believe he told someone they should run dhcp on their network gear and not use windows DHCP server because some Network guy told him that was a good idea.

*reboots DHCP server every week for critical patches, hands out 8 day leases*

99% of Windows admins are incompetent, and each DHCP lease technically requires a CAL.

That said, whether you should go for windows/router/switch whatever DHCP always depends on the environment and the requirements.

theperminator fucked around with this message at 05:11 on Apr 11, 2015

theperminator
Sep 16, 2009

by Smythe
Fun Shoe
If the task has to be compromised to the point where it's a ticking timebomb due to money/staff skill then the responsible thing to do is tell management you can't do it and why.

theperminator
Sep 16, 2009

by Smythe
Fun Shoe

Martytoof posted:

Is there any way to tell VCSA to not force a URL redirect and just use the URL I came in on?

Late reply I know but I think you can resolve this by setting the VCSA hostname to localhost.

I had the same issue with a VCSA upgraded from 5.5 that had a hostname set, but a newly created one with a default hostname of localhost does not.

theperminator
Sep 16, 2009

by Smythe
Fun Shoe
I know that NPAPI deprecation kills the vmrc plugin, but why does vsphere 6 default to telling you to download the VMRC Application rather than just use the HTML5 console?

theperminator
Sep 16, 2009

by Smythe
Fun Shoe

adorai posted:

the web client was just so poo poo to begin with, that they will probably never get that bad taste out of peoples mouths.

Probably, and the current line being "Don't use the native client, use the webapp even though it's a piece of poo poo because someday it will totes get better I promise" is poo poo.

Users need it to work well so they can get poo poo done and make sure their infrastructure keeps working, it isn't some toy. that's why people keep using the windows app.

theperminator
Sep 16, 2009

by Smythe
Fun Shoe
One thing that always irrationally irritates me is that in the Web client, if you want to do guest customization you have to save the spec whereas In the native client you can do guest customizations without having to save them first.

theperminator
Sep 16, 2009

by Smythe
Fun Shoe

Martytoof posted:

I don't know if this is a question for here or the enterprise hardware thread but:

I want to deploy an R620 standalone ESXi host for testing. It's got the integrated mirrored SD slots in the back. Each came with a 2GB card. Is that going to be fine for a hypervisor out of the box, or should I look to add larger cards? No vCenter, no extra logging, just straight host for VMs. I might install that standalone WebUI but that's about it.

edit: That is to say, it's got like 2TB datastore, I just want to throw the hypervisor alone on the SD cards.

Should be fine, I was deploying M620's at my last job with ESXI and 1GB Mirrored SDCards with no issues at all. ESXI doesn't make use of the SDCard for anything after booting

theperminator
Sep 16, 2009

by Smythe
Fun Shoe

Number19 posted:

:stare: Yeah the lack of VUM isn't why people aren't using the web client. Also VUM still requires Windows. How is that still possible after all this time?

Because they have no clue how anyone uses their product or how they feel about the garbage they are constantly spewing out and forcing on us.

theperminator
Sep 16, 2009

by Smythe
Fun Shoe
Shouldn't the HP card be ok? I'm pretty sure it's an Intel 82571EB or something close to that which will be well supported by ESXI, I bought some HP NC360T Dual-ports for my stuff.

theperminator
Sep 16, 2009

by Smythe
Fun Shoe

Internet Explorer posted:

Am I being overly paranoid? I know fixing the VMFS volume after Windows rights its signature isn't impossible... I'm just not sure the pros outweigh the cons.

You're not paranoid, there's no way in hell I would do that.

theperminator
Sep 16, 2009

by Smythe
Fun Shoe

Martytoof posted:

Does VMware "remote console" functionality just not work with Safari on OSX? I updated to Safari 9 with ElCap but this was the case with 10.10 as well.

If you click on the little console window preview thingy it should open a HTML5 console iirc

theperminator
Sep 16, 2009

by Smythe
Fun Shoe

mewse posted:

To be fair Microsoft invented Hyper-V and vmware is a pale imitation

Is this a joke?

theperminator
Sep 16, 2009

by Smythe
Fun Shoe
Has anyone had any issues with vMotioning a Database?

In all the years I've been using vSphere we've never had an issue with vMotion causing any data issues, at my current workplace our DRS threshold is set quite conservatively and we've recently had a MySQL Consultant tell us we should not vMotion our Databases at all. But obviously we do vMotion the database hosts whenever we patch our esxi hosts or perform maintenance and have never had any issues before.
I said that was bullshit, at my last place we had plenty of them with DRS in Full-Auto aggressively migrating databases & other workloads around without any issues.

Recently we did a test, we simply vmotioned our database slave to every host in our cluster, while this was happening we got relay-log corruption and slaving has stopped.
I'm scratching my head trying to figure out what happened, at the same time we were doing this it moved another DB Slave around to maintain affinity rules, the other VM didn't have any issues at all.

I Find it hard to believe that vMotion would be a risky thing, it's one of the selling points of the software and it's something we've done plenty of before but is only suddenly now causing issues?

theperminator
Sep 16, 2009

by Smythe
Fun Shoe
Still scratching my head too, there was no failover actions taken, the migration completed and the VM did not pause during the migration.

Looking at the binlog, there is 2.5MB of zero bytes starting at the query it has failed on, the binlogs continue after that as normal (the slave was lagging by a few hours when this happened, the IO thread is ahead of the SQL Thread)
FSCK reports no errors on the filesystem either.

It's like a file buffer was zero'ed somehow during the process of migration, I would expect the vmotion process to do some kind of checksum but maybe not?

theperminator
Sep 16, 2009

by Smythe
Fun Shoe

jre posted:

Do you have sync_binlog =0 set on the affected dbs my.conf ?

Just checked, we are running an old version where this is the default, confirmed with a "show global variables"
It might be worthwhile for us to modify this, I notice that on 5.7.7 and up the new default is 1

Still weird though, the vmotion shouldn't impact the guest to my knowledge but I guess there's a difference between theory and practice.

theperminator
Sep 16, 2009

by Smythe
Fun Shoe

Martytoof posted:

In TYOOL 2016 why doesn't vSphere have a way of reconnecting to an NFS datastore without me having to esxcli storage nfs delete and re-add it? :|

I mean this is just my homelab so it's not critical but c'moooon

Which version of ESXi are you running? I used to have this problem a while back with 5.5 but I think one of the updates has fixed it

edit:
https://www.vmguru.com/2015/12/best-practices-running-vmware-with-nfs

quote:

Make sure your VMware environment is updated and runs on patch 5 or newer if you have ESXi 5.5 and on Update 1a if you run with ESXi 6.0
Which version of VMware are you running?
Running ESXi 5.5 below Patch 5? Than you are running on a build number lower than 2718055 and it is time to upgrade now!
We have seen several virtual environments that ran below patch 5 and had strange and unpredictable behavior from time to time. Like disconnects (offline, greyed out) which could lead to an all paths down (APD) event (for 45 minutes) or even permanent device loss (PDL) or management interfaces which freeze for several minutes or getting stuck majorly. After updating those infrastructures to newer builds you could restart an ESXi server and datastores where almost instantly back online and visible. Also managing storage in vCenter server becomes more fluid than before. And a rescan completes about 3 to 4 times faster.

theperminator fucked around with this message at 07:11 on Oct 23, 2016

theperminator
Sep 16, 2009

by Smythe
Fun Shoe
Just did a hardware refresh in prod just in time for 6.5 upgrade, we're going to test in labs & test environments before upgrading prod.
Kinda annoying that integrated containers isn't GA yet, & NSX doesn't support 6.5 yet either :(

theperminator
Sep 16, 2009

by Smythe
Fun Shoe

BangersInMyKnickers posted:

I have two active/active clusters but one of them isn't technically active with a workload yet so I might yolo that one up to 6.5 to see if it fixes their lovely Goddamned LACP

What issues have you seen with LACP? We recently deployed a new cluster and went with LACP for the first time so I'm hoping it will be stable...
We did have issues already but they turned out to be a driver issue for the Intel i40e NIC Driver, an update cured it.

theperminator
Sep 16, 2009

by Smythe
Fun Shoe

BangersInMyKnickers posted:

Short/long timeout mismatches because vdSwitches default to long timeouts and you can't configure it without ssh-ing in to each host after startup, LACP drops negotiation on at least one link at least daily and VMware has absolutely no idea why it is doing it. It's been a trip. This is with the updated Intel driver, I did that with the default driver flooded the logs with nonsense.

gently caress me, I wish we'd gone with just active/standby like our old cluster.

mayodreams posted:

We also weren't updating firmware on the hosts so that was magical too.

Dell firmware upgrades have caused so much pain for me that I usually err on the side of being conservative with firmware updates unless it's a security update or the release notes show something relevant to an issue we are having.

Equallogic 6.x firmwares were worse every time, from rebooting after 365 days of uptime to getting stuck in a reboot loop after upgrading.

theperminator fucked around with this message at 12:27 on Nov 19, 2016

theperminator
Sep 16, 2009

by Smythe
Fun Shoe

devmd01 posted:

Boo. I use the web client for day to day but for initial host setup the fat client is way faster. I'd script them out with powerCLI but I don't have homogenous hosts.

I found that the HTML5 interface for host configuration is actually quite fast and responsive, not sure if it has 100% of the fat client feature set though


xtal posted:

Maybe this is the wrong thread but does anyone else feel like docker is the javascript of systems? So tired of my coworkers begging to use it for no reason other than hype. I see literally 0 benefit because 0 of its ideas are original

You're not the only one, I've noticed that there's a lot of "lets do X because it's cool!" with no mention of how it's helpful to the business, it seems like people think work is like adult daycare now.

I think what they focus on when they think Docker though is the API etc rather than the actual virtualization aspect

There are some benefits in that it simplifies deployment & scaling of your application, and allows you to push the container through a pipeline i.e dev > test > prod knowing that it should work the same way in all of the stages.
If you have a microservices architecture or running web services that you want to be able to easily scale horizontally it can be quite useful I think.
But I believe that the biggest problems we found were that the security isn't good enough for what we do, the networking aspect is garbage & our applications aren't really conducive to horizontal scaling yet.

This is why I'm kinda excited about vSphere Integrated containers, everything runs as it's own VM and the networking is the same as any other VM rather than a bridge, which means we could use NSX to control access between each container.

theperminator fucked around with this message at 22:37 on Nov 21, 2016

theperminator
Sep 16, 2009

by Smythe
Fun Shoe

Methanar posted:

Seconding sadness with x710 and VMware.

Thirded, but luckily for us we only recently bought the servers so the 1.4.28 driver was available to us straight away.

We're still on 5.5 though, disabled TSO/LRO & updated the driver but will have to keep the above driver link for when we upgrade to 6.5

theperminator fucked around with this message at 03:10 on Dec 6, 2016

theperminator
Sep 16, 2009

by Smythe
Fun Shoe
Anyone have much experience with VSAN in production environments?

I'm wondering what kind of issues I can expect to run into with it, any gotchas etc?

theperminator
Sep 16, 2009

by Smythe
Fun Shoe
Cool, I've checked that our R630s with PERC H730P's are supported, we will have 10G ports dedicated to VSAN traffic & our servers have tonnes of RAM & CPU so we should be all good from that point of view.

GobiasIndustries posted:

How do I get rid of orphaned vms? In my home lab (esxi/vcenter 6.5) I tried deploying an ova (sexilog) and it failed. The vm shows up as orphaned in my inventory but I can't find any options for deletion.

What I usually do in this case is create a new folder, move the VM to that folder then delete the folder.

theperminator
Sep 16, 2009

by Smythe
Fun Shoe

BangersInMyKnickers posted:

Is anyone else seeing VMs misreporting virtual memory utilization after the 6.5 upgrade? I have a couple that are wildly incorrect beginning the second they migrated to a 6.5 host.



That particular VM only has 2gb out of 5 even allocated inside the OS memory manager, there is no way the active pages are that high. I have at least a half dozen doing this and they all show that steady increase since I did the upgrade on the 10th.

From my understanding the vmware memory graphs won't generally match what the OS reports because the hypervisor is unaware of freed memory by the guest.
We've found for instance that our Java Applications only use 2GB out of the available 8, but after running for long enough with enough Garbage Collection happening, all of the memory gets touched and the memory usage of the guest is reported as 100% by vmware
What does the "Active" metric look like for your VMs?

theperminator
Sep 16, 2009

by Smythe
Fun Shoe

Martytoof posted:

Oh yeah, good point. I guess at this point I'm really only leaning VMware's offering because I'm praying for some vSphere integration down the road. My ideal setup here would be for VMware to integrate Photon into the UI such that I can power on a Photon cluster, then drill down and start/stop individual containers. A man can dream :unsmith:

Maybe someone will have come up with a fling for what I'm describing.

VIC might fit the bill, runs docker containers each inside their own photon instance on ESXI, though you still need to create/start the containers via the docker API

https://vmware.github.io/vic/

theperminator
Sep 16, 2009

by Smythe
Fun Shoe
You're not blocking tcp DNS queries somehow are you? Large DNS responses sometimes require TCP.

theperminator
Sep 16, 2009

by Smythe
Fun Shoe

Thanks Ants posted:

Everything's gone a bit quiet on the vSphere on AWS front - I assume there's a billion NDAs in place and it will launch when it's done, as opposed to the idea being canned.

I've heard on the grapevine that VMware don't actually have anything to show yet, they apparently announced super early.

theperminator fucked around with this message at 05:52 on Mar 3, 2017

theperminator
Sep 16, 2009

by Smythe
Fun Shoe
It's basically a way to have a hybrid cloud solution without having to screw around too much. Your on-premise and cloud compute will be manageable from the same interface with the same tools & workflows, you can live migrate vms to/from the cloud, you can easily set things up so that you can fail over to the cloud.
NSX allows a lot of cool things like sharing the layer 2 domain across sites easily which means a VM failover to the cloud won't require IP reconfiguration, network ACLs that apply depending on the application traffic detected, or tags you've assigned to the VM rather than static policies you have to update.

AWS products seem really geared towards web applications that you scale horizontally and load balance automatically etc, but there's a whole lot of business that doesn't work that way. Plenty of financial orgs with lovely monolithic java apps etc

There's probably a lot more but from the POV where I work the above is what's interesting to us.

theperminator
Sep 16, 2009

by Smythe
Fun Shoe

evol262 posted:

I mean, this is basically what every other hybrid cloud solution does (minus the migration from the cloud). I'm not seeing the value except "it's still VMware". But I'm not convinced that's a benefit when you're paying twice

Because if you have a multi-cloud setup across AWS/Azure/On-Prem you can easily integrate them all and migrate workloads around, deploy workloads with the same tooling.

quote:

AFAIK, AWS Direct Connect already does a lot of this. Not all. And it's not that NSX isn't an interesting technology, it's that I legitimately don't understand why orgs looking at this wouldn't just also learn AWS tooling
Direct connect is just a simple cross-connect as far as I'm aware, which means changes need to be made on your physical devices whenever you want to add a new VLAN etc, with SDN you can make network changes with API calls.
When you do micro segmentation and microservices at scale you spend an awful amount of time making firewall changes, networking changes to account for new services every week. Being able to deploy a new application and have the deployment process configure all of the networking/firewalling across different cloud providers/premises automatically is awesome.

quote:

Well, that's the cloud in general. You can spin up monster instances which will run lovely monolithic Java apps or c++ abominations which have been ported in the most half-assed way across 3 different UNIX variants then Linux (I also came out of finance)
And with features like VMware's FT you can have a synchronously running copy of any VM running somewhere else, ready to take over if poo poo hits the fan
You can also hot-plug RAM & CPU so you can vertically scale without having to shut down your instance
You can migrate the workload instead of blowing it away & starting again when amazon decide to retire the host your VM is running on.

quote:

Horizontal scaling is mostly there for relatively stateless applications or sites which simply can't be vertically scaled in any kind of efficient way once you make the front-page of Reddit (for example)

Plus, horizontal=resilient in case an AZ or site goes down. Since AWS doesn't provide a way to say "my server broke; an admin needs to connect to the ilo and unfuck it". Reprovision the VM from an image, and attach your persistent storage volume to it so it can find the database again

Still, horizontal scaling only works if your workload supports it. some of us are unlucky enough to deal with horrible messes that don't scale like that.
As a matter of preference I generally would prefer console access because if something breaks I want to know why. but there's not much business sense in that unless it's something that keeps happening.

theperminator
Sep 16, 2009

by Smythe
Fun Shoe

evol262 posted:

I'm not sure if you're secretly a VMware sales rep or if you just don't think I know what VMware offers. I do. I work on a competing solution with a similar feature set.

Nope, not affiliated with vmware in any way. not all of us are pushing vendor agendas.

quote:

This whole post really still missed the gist of my basic question. Which is not "what is VMware and what are its capabilities?" or "why do people use private clouds?", but "why should anyone give a poo poo about vSphere on AWS" or "why not learn cloud paradigms"? If you want to have an off-site hosted VMware environment, there are a lot of options. Just use Rackspace, for example.

What AWS/GCE/Azure/etc really offer isn't "virtualization hosting", but a bunch of tooling for managing/deploying/autoscaling anonymous VMs across geographically-separated environments, with solutions for object/block storage, tenant networking. It sounds like VMware is throwing all of that by the wayside or hiding it all, and the "killer feature" of vSphere on AWS is that it has "AWS" in the name.

I haven't seen any real technical details on what VMware is going to do (and I'm not going to watch some video), but nothing you've offered here really tells me anything I don't already know.

Well sorry about that, I have no idea why businesses want to use VMware on AWS but they do, probably just so they can pull their dicks over using AWS, or the fact that they have so many availability zones or other products that the clients want to use.

theperminator
Sep 16, 2009

by Smythe
Fun Shoe
Maybe they changed it to consumed rather than active? Consumed never drops unless ballooning kicks in.

Like you said, balloon driver has nothing to do with active, it's just some algorithm that deems that "x pages have been touched recently" and will work even without tools iirc.

theperminator fucked around with this message at 07:37 on Mar 24, 2017

theperminator
Sep 16, 2009

by Smythe
Fun Shoe

BangersInMyKnickers posted:

My guess is at some point something breaks and any page that has ever been written to gets marked as permanently active and it persists until the VM restarts, then promptly comes back. Having that bar represent consumed memory would be absolutely idiotic considering the opportunistic caching the OS and application code is almost certainly doing.

Guest memory management is irrelevant to the hypervisor, once a page has been touched by the guest the hypervisor considers it used by the guest until the VM is power cycled or ballooning kicks in because it has no idea what blocks have been freed by the guest. this is why if you look at the cluster overview you'll notice that memory usage doesn't go down.
I've had a lot of issues with this on account of memory overprovisioning combined with busy java apps in the guest that do a lot of garbage collection and then subsequently use up every single byte of memory assigned to them over time, while the memory usage from Linux' point of view was never over 50%

Still, Active shouldn't ever be that high and it's clearly a bug.

theperminator fucked around with this message at 00:07 on Mar 25, 2017

Adbot
ADBOT LOVES YOU

theperminator
Sep 16, 2009

by Smythe
Fun Shoe
Someone created a 10TB VM on a datastore just big enough to fit it, took a Snapshot then started making copies of 4TB of files. Only on our testing cluster but still :suicide:

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply