Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
cliffy
Apr 12, 2002

Moving this discussion here from the Intel thread re: VGA passthrough to a Windows VM under qemu.

Combat Pretzel posted:

Eh, I had a Pastebin link in my history.

http://pastebin.com/Rb5d3nKm

cliffy posted:

Thanks a lot for posting this! I just got qemu setup to run a Windows 10 VM for games with GPU pass-through. It works pretty well thus far, but passing the GPU back and forth between the host and VM would be pretty amazing.

Given that I've already setup everything for Vt-d and GPU pass-through, is there any other preparation needed to swap the GPU back and forth between the host and VM besides your script?

Would you mind elaborating on your script some? For instance, does this mean you are able to run the pass-through GPU on your host's X session, take it down dynamically, and then reattach it to the X session when you're done? I notice in the script it says not to run it under X.

You use Q35 as the emulated chipset, but most guides I see leave it on the 440fx chipset. It looks like you've got a lot of hyper-v stuff enabled, is that something you can re-enable after you get the nvidia driver installed? Or are you using AMD GPUs?

Please tell me more :allears:

Combat Pretzel posted:

The X shutdown stuff was done in another script using systemd-run, so it'd survive a change in systemd isolate. It'd change from graphical to multi-user to kill the X session, wait for xinit to go away and then run the script I linked with Pastebin. After QEMU stops blocking the execution of the scripts, it'd change back from multi-user to graphical, which would restart X. I'd upload it, but I forgot to copy it when I reinstalled Windows.

Be aware that the X session gets killed dead, and as such all apps running in it. And I was swapping the secondary controller. The primary kept running the framebuffer console. Some dude on reddit I pm'd with tried to get the primary working, seemed to have some intermittent success in releasing all drivers from it, but then I never heard again from him.

Regarding 440fx, I couldn't get a decent answer for this, because the documentation is pretty non-specific and seems out of date in places (calling Q35 relatively new and unstable, despite being there for years now, plus it's being actively maintained in the git, unlike 440fx). Q35 defines the newer ICH9 chipset, which has a proper PCIe bus hierarchy, all chipset devices support MSI interrupts (less overhead) and the OVMF UEFI stuff works better with it (for me, anyway).

As far as Hyper-V goes, the NVidia drivers look for the Hyper-V CPUID and refuse to start when they see it. QEMU 2.5 allows to specify a custom one for obfuscation via hv-vendor-id on the -cpu line (that's why it's there), which lets you use the NVidia driver without issues. For now, anyway, because months ago, the NVidia driver apparently didn't care about the Hyper-V CPUID to begin with. Chances are a newer version will look for other signs to identify Hyper-V eventually and then stop working again. Either way, enabling Hyper-V improves framerates a lot. If your CPU is older than a Haswell or Ivy Bridge-E, you should also specify hvapic for the Hyper-V virtual APIC. Do not specify it with these or newer CPUs, because they support hardware virtualization of the APIC, which has less hypervisor overhead.

Also, check whether MSI interrupts got enabled for graphics, the HD audio device and the USB controllers, to eke out some more performance. MSI interrupts have less hypervisor overhead, too. Specifics here: http://vfio.blogspot.be/2014/09/vfio-interrupts-and-how-to-coax-windows.html

Personally, I'm holding off a while of this VM stuff, hoping that NVidia will release their Wayland support finally. Pissed me off that I couldn't get a 60fps compositing desktop working with neither mutter or compton (pissed me off seeing the smooth compositing every time I booted into the VM). Load Chrome and your framerate's hosed. Worked fine with Gnome Wayland, but that required the nouveau drivers, and gently caress those.

--edit: I suppose this should be moved to the Virtualization thread, anyway.

Thanks for the tips. I have enabled MSI interrupts for all devices and things appear to be working smoothly. I am afraid NVIDIA will eventually become wise and start doing more aggressive checks, but I'll cross that bridge when I get there. Right now performance is near-native as far as I can tell, but I haven't done true win10 host versus win10 guest benchmarking.

Windows can definitely tell its running virtualized. Do you know if there is a way to completely hide that from Windows, or will that break things?

Adbot
ADBOT LOVES YOU

Zero VGS
Aug 16, 2002
ASK ME ABOUT HOW HUMAN LIVES THAT MADE VIDEO GAME CONTROLLERS ARE WORTH MORE
Lipstick Apathy
Question about AWS Elastic Load Balancing:

I've had an Asterisk PBX up on Amazon EC2 for a month now, prepping it to replace our production phone system. I spun up a separate VPN VM alongside it so we can access the internal IP (not using the Amazon public IPs for security reasons) and it been rock-solid making calls with a dozen phones for weeks so far. I have two SIP trunks, two dual-wan routers and two ISPs in, etc, but what I haven't decided on yet is the best way to give the Asterisk VM itself some better availability on Amazon.

Do I just take a snapshot of the current PBX instance and set it up in a different availability zone? I see that the Amazon ELB will give it the same IP address, but will it also get the same MAC address? I ask because the PBX server uses the MAC to tell if it is the same machine, and requires reregistration of paid modules if it differs.

Does that seem like the best way to go about improving availability? Anyone else tried something like this?

Combat Pretzel
Jun 23, 2004

No, seriously... what kurds?!

cliffy posted:

Windows can definitely tell its running virtualized. Do you know if there is a way to completely hide that from Windows, or will that break things?
Not if you want Hyper-V enabled. Windows knows about virtualization due to the related Hyper-V MSRs. What's why it can use the enlightments even with a customized CPUID. Haven't tried what happens if you disable the Hyper-V stuff.

evol262
Nov 30, 2010
#!/usr/bin/perl
It basically still runs near-native

cliffy
Apr 12, 2002

evol262 posted:

It basically still runs near-native

So you're running a Windows 10 VM which thinks its bare-metal? I take it you didn't see any performance degradation? It seems enlightened Windows guests will limit the number of logical CPUs it utilizes. I wonder if getting around that is useful at all.

cliffy fucked around with this message at 00:39 on Mar 16, 2016

evol262
Nov 30, 2010
#!/usr/bin/perl
I'm running a win10 guest with GPU passthrough, KVM hidden, and Hyper-v extensions disabled. I haven't had a physical Windows environment in 6 years, but it's fast enough to run plenty of AAA games at ultra, and SQL server benchmarks well

cliffy
Apr 12, 2002

evol262 posted:

I'm running a win10 guest with GPU passthrough, KVM hidden, and Hyper-v extensions disabled. I haven't had a physical Windows environment in 6 years, but it's fast enough to run plenty of AAA games at ultra, and SQL server benchmarks well

Any performance tips?

cliffy
Apr 12, 2002

I must have screwed something up, or maybe I never had it configured well but I'm seeing very low GPU utilization and low frames in the Windows guest. MSI Afterburner in the guest reports at most 10% gpu usage, and CPU usage is not very high. Host is not doing anything intensive either.

evol262
Nov 30, 2010
#!/usr/bin/perl
I honestly didn't do anything other than UEFI native boot+secureboot. The VM gets 3 vcpus and a passed-through GPU. Do you have CPU power states on? What does top show versus task manager in the guest (if there's a difference)?

cliffy
Apr 12, 2002

I must've been feeding it too many logical CPUs, set it to 8 at one point (4 cores, 2 threads each), now it seems to be doing a lot better with 4 logical CPUS (2 cores, 2 threads). 5x frame rate increase. Maybe it was doing too much context switching? Is there some way to pin each logical core in the VM to a specific CPU on the host?

evol262
Nov 30, 2010
#!/usr/bin/perl
In general, VMs do best with the bare minimum number of vcpus necessary due to constraints on scheduling. VMware has a good whitepaper on coscheduling, which is obviously specific to vmkernel, but the principles are the same. Assign as few vcpus as necessary. Add more if you're cpu-bound. Never, ever create a VM which has more logical cpus than you have physical (and physical-1 is almost always the max in practice).

Of course you can pin cpus. Google "libvirt cpuset". vcpupin also works

Vulture Culture
Jul 14, 2003

I was never enjoying it. I only eat it for the nutrients.

evol262 posted:

In general, VMs do best with the bare minimum number of vcpus necessary due to constraints on scheduling. VMware has a good whitepaper on coscheduling, which is obviously specific to vmkernel, but the principles are the same. Assign as few vcpus as necessary. Add more if you're cpu-bound. Never, ever create a VM which has more logical cpus than you have physical (and physical-1 is almost always the max in practice).

Of course you can pin cpus. Google "libvirt cpuset". vcpupin also works
vSMP co-scheduling is probably one of the top three most important things to understand about how virtual machines work.

cliffy, to summarize what that whitepaper is going to tell you, the idea is that the CPUs of a computer, physical or virtual, basically need to be kept in sync. VMware relaxes this slightly and keeps the CPUs mostly in sync. Each vCPU can get a few cycles ahead of the others, sort of like the runners in a three-legged race -- you can get one leg way out in front, but there's a limit to how far that one leg can move in front of the others. When one CPU runs too far ahead of any other CPU, the hypervisor issues a VM co-stop, which stops the CPUs further in the future until the ones lagging behind it have a chance to catch up. So what you'll find is that if you have any underutilized CPUs, they'll lag behind the ones that are actually running and doing work, and the VM will constantly stop-start-stop-start on the scheduler, like the racers falling down and having to get back up.

Vulture Culture fucked around with this message at 05:51 on Mar 16, 2016

BangersInMyKnickers
Nov 3, 2004

I have a thing for courageous dongles

Hyper-threading on a VM host is only going to optimize the workload of multiple VMs with their vCPUs sharing the same physical core by helping to fill in gaps in the pipeline, but by the time this can have much of an effective impact you're already so overloaded on the host that your VMs are dying for CPU overcommit. By creating a VM with twice as many vCPUs at logical cores on the host you advertised to the VM and its software that you had effectively twice as many CPU cycles as you really did in reality. Doing that and running a multi-threaded, cpu-bound program immediately committed you on CPU by 100% with a full load which isn't good, as you discovered.

Vulture Culture
Jul 14, 2003

I was never enjoying it. I only eat it for the nutrients.

BangersInMyKnickers posted:

Hyper-threading on a VM host is only going to optimize the workload of multiple VMs with their vCPUs sharing the same physical core by helping to fill in gaps in the pipeline, but by the time this can have much of an effective impact you're already so overloaded on the host that your VMs are dying for CPU overcommit. By creating a VM with twice as many vCPUs at logical cores on the host you advertised to the VM and its software that you had effectively twice as many CPU cycles as you really did in reality. Doing that and running a multi-threaded, cpu-bound program immediately committed you on CPU by 100% with a full load which isn't good, as you discovered.
this is a good post

Thanks Ants
May 21, 2004

#essereFerrari


Is there a difference at all between giving a VM 2x single core CPUs vs 2x cores, other than guest application licensing?

evol262
Nov 30, 2010
#!/usr/bin/perl
You can play with/optimize for different NUMA topologies, but in practice, that's very niche. qemu-kvm doesn't really treat it differently other than that, and I'd be surprised to hear if vmkernel or Hyper-v did.

There may be a very marginal effect on cache coherency if you're targeting a non-host CPU type, but none that I've ever observed

Gucci Loafers
May 20, 2006

Ask yourself, do you really want to talk to pair of really nice gaudy shoes?


For a virtualization lab does it matter (or will I notice?) if I get an i3 (i3-6100U) vs. an i5 (i5-6260U)? I'm looking at the specifications on Wikipedia and the only difference I'm recognizing is a slightly different overall clock rate, cache and Iris-graphics.

KillHour
Oct 28, 2007


Why are you buying a laptop for a virtualization lab?

Cactus Jack
Nov 16, 2005

If you even try to throw to my side of the field in a dream, you better wake up and apologize.

KillHour posted:

Why are you buying a laptop for a virtualization lab?

Could be a nuc. :shrug:

Gucci Loafers
May 20, 2006

Ask yourself, do you really want to talk to pair of really nice gaudy shoes?


It's a nuc.

BangersInMyKnickers
Nov 3, 2004

I have a thing for courageous dongles

Good news on vCenter 6.0u2, it looks like it fixed the problem on the HTML5 consoles where cursor input wouldn't register on the bottom 1/3 of the window so it was impossible to invoke the start menu without blindly tabbing at the desktop until you hit it.

DevNull
Apr 4, 2007

And sometimes is seen a strange spot in the sky
A human being that was given to fly

Other good news in the client/console area... http://pubs.vmware.com/Release_Notes/en/vsphere/60/vmware-host-client-10-release-notes.html

The host client is no longer just a Fling, it is now officially supported.

Pile Of Garbage
May 28, 2007



DevNull posted:

Other good news in the client/console area... http://pubs.vmware.com/Release_Notes/en/vsphere/60/vmware-host-client-10-release-notes.html

The host client is no longer just a Fling, it is now officially supported.

My colleague told me about this earlier in the week and I'm super psyched about it. The host client is extremely awesome but as a Fling it's hard to convince enterprises to push it out due to potential lack of support. But now that the feature's been rolled into ESXi it's all good!

Vulture Culture
Jul 14, 2003

I was never enjoying it. I only eat it for the nutrients.

Tab8715 posted:

For a virtualization lab does it matter (or will I notice?) if I get an i3 (i3-6100U) vs. an i5 (i5-6260U)? I'm looking at the specifications on Wikipedia and the only difference I'm recognizing is a slightly different overall clock rate, cache and Iris-graphics.
It depends on your workload. I've never, ever been CPU-bound in my lab unless I've done so intentionally (performance testing on crap hardware).

BangersInMyKnickers
Nov 3, 2004

I have a thing for courageous dongles

If anyone has the problem where vmware-dataservice-sca status keeps cycling from yellow to green and throwing alarm emails, I think I may have found a kludge on Windows. In this thread https://communities.vmware.com/thread/520429?start=15&tstart=0 someone got around in on the vCenter appliance by creating a script called dummy_path that did nothing but run 'exit 0' and put it in the directory where sca.log was complaining that the file was missing from. That seemed to shut it up in the case of the appliance. On the Windows side, its demanding a proper Win32 application to execute so I went here http://www.paulsadowski.com/wsh/cmdprogs.htm and grabbed a copy of echo.exe, then dropped it in \ProgramData\VMware\vCenterServer\cfg\sca\scripts and renamed it from echo.exe to dummy_path and that also seemed to stop the alarms. Sca.log still complains a bit that the echo application is returning unexpected data I don't think its hurting anything. I don't have visual studio at work otherwise I would have made a quick and dirty application that did nothing except exit returning errorlevel 0 and threw it in there instead.

pixaal
Jan 8, 2004

All ice cream is now for all beings, no matter how many legs.


I'm looking to implement Hyper-V using local storage (SAN isn't in the budget). Should the host have it's own drives or should I just make a large RAID10 and partition it out? I'm leaning towards 8 Drive RAID10, but thought I'd get a second opinion. Full environment is going to be 2 hosts with Veeam doing backups to a NAS. Restore using the backup is considered acceptable(both potential data loss and downtime). If there is a better way to configure this with the same hardware though I can make the adjustments since nothing is purchased yet.

Moey
Oct 22, 2010

I LIKE TO MOVE IT

This is awesome.

mayodreams
Jul 4, 2003


Hello darkness,
my old friend

BangersInMyKnickers posted:

Good news on vCenter 6.0u2, it looks like it fixed the problem on the HTML5 consoles where cursor input wouldn't register on the bottom 1/3 of the window so it was impossible to invoke the start menu without blindly tabbing at the desktop until you hit it.

loving finally. I use a Mac at work and this has been making my life painful for a while.

It also supposedly fixes the remote console SSL issues with workstation/fusion and the browser plugins.

HPL
Aug 28, 2002

Worst case scenario.
Just updated to 6.0u2 with this guide: https://tinkertry.com/easy-upgrade-to-esxi-60u2

Looks good so far. Though for some reason I'm not authorized to download the non-HTML 6.0u2 client.

Potato Salad
Oct 23, 2014

nobody cares


DevNull posted:

Other good news in the client/console area... http://pubs.vmware.com/Release_Notes/en/vsphere/60/vmware-host-client-10-release-notes.html

The host client is no longer just a Fling, it is now officially supported.

You guys are doing God's work.

Kivi
Aug 1, 2006
I care

evol262 posted:

You can check the PCI ID in:
find /sys/kernel/iommu_groups/ -type 1

quote:

### Group 1 ###
00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port (rev 09)
01:00.0 VGA compatible controller: NVIDIA Corporation GM206 [GeForce GTX 960] (rev a1)
01:00.1 Audio device: NVIDIA Corporation Device 0fba (rev a1)
### Group 13 ###
03:00.0 USB controller: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller
This is what I'm working with, the GPU passthru is flawless.

lspci -nn

quote:

03:00.0 USB controller [0c03]: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller [1b21:1042]

quote:

If it's on it's own group, pass it through. I assume you've done this. Does the USB controller appear in the VM and just USB devices plugged into it appear on the host?
Yes, it appears in the VM normally, but anything attached to it goes to the host.

quote:

virt-manager can do this very easily, or I can give you XML if you want to edit the domain by hand.

virt-manager can also handle passing through USB devices
That'd be super helpful, thanks!

Potato Salad
Oct 23, 2014

nobody cares


Crosspost from Pissing Me Off: Myself

We have some seminar / training room VDI pools in View and some temp employee / home access VDI pools.

To handle the fact that we want user desktop / document folder redirection to our storage filer on the employee pools and not the training pools, we deployed the AD OU for our VDI computer accounts with policy loopback enabled. There is a very obvious edge case I didn't account for that has certain temp employees not on redirected folders. How to much more elegantly lay out the OU for our VDI machines is obvious now, but drat it I need to apply this laaaaaaaaaaate after hours.

DevNull
Apr 4, 2007

And sometimes is seen a strange spot in the sky
A human being that was given to fly

Potato Salad posted:

You guys are doing God's work.

It was definitely a holy war getting it to this point. There was a lot of political resistance along the way.

The work for it started 3 years ago with a little hackathon the company did. One of the groups did a javascript client that allowed you do do basic functions on an ESX box such as browse the datastores and manipulate VMs for basic things. Another group of 3 engineers ported the webMKS to ESX. That gave you a console to the VM. The two groups merged their projects and tried to get the company to launch it as a real project. It didn't get picked up though because the webUI group was still pushing the Adobe Flex stuff. Their solution to connecting with vSphere was install flash on a windows host, and installing some package that you could connect to ESX. It was bad, I had trouble getting it setup. In the mean time, one of the vmkernel engineers saw the hacked together javascript/html project. He convinced the vmkernel director to staff it up directly. They hired another engineer with UI experience, and together with the vmkernel engineer, the produced the Fling. Obviously the Fling did well, so it was changed to a supported product. VMware sure as hell better give the vmkernel engineer a big promotion and pay raise. He absolutely nailed it with this project.

Mr Shiny Pants
Nov 12, 2012

DevNull posted:

It was definitely a holy war getting it to this point. There was a lot of political resistance along the way.

The work for it started 3 years ago with a little hackathon the company did. One of the groups did a javascript client that allowed you do do basic functions on an ESX box such as browse the datastores and manipulate VMs for basic things. Another group of 3 engineers ported the webMKS to ESX. That gave you a console to the VM. The two groups merged their projects and tried to get the company to launch it as a real project. It didn't get picked up though because the webUI group was still pushing the Adobe Flex stuff. Their solution to connecting with vSphere was install flash on a windows host, and installing some package that you could connect to ESX. It was bad, I had trouble getting it setup. In the mean time, one of the vmkernel engineers saw the hacked together javascript/html project. He convinced the vmkernel director to staff it up directly. They hired another engineer with UI experience, and together with the vmkernel engineer, the produced the Fling. Obviously the Fling did well, so it was changed to a supported product. VMware sure as hell better give the vmkernel engineer a big promotion and pay raise. He absolutely nailed it with this project.

I bet they fired him, kicking up a storm like that. I hope not, but I would not be surprised.

DevNull
Apr 4, 2007

And sometimes is seen a strange spot in the sky
A human being that was given to fly

Mr Shiny Pants posted:

I bet they fired him, kicking up a storm like that. I hope not, but I would not be surprised.

No, he is good. The director that was backing him is now actually a VP, so he has even better protection.

Mr Shiny Pants
Nov 12, 2012

DevNull posted:

No, he is good. The director that was backing him is now actually a VP, so he has even better protection.

Cool :)

HPL
Aug 28, 2002

Worst case scenario.
The problem I have with the new HTML client is that if I bring up a console in either a new browser window or tab, I can't get to the Action menu to ctrl-alt-del or whatever. It works fine if I open a window within the client window. I had this problem back when it was a Fling as well.

DevNull
Apr 4, 2007

And sometimes is seen a strange spot in the sky
A human being that was given to fly

HPL posted:

The problem I have with the new HTML client is that if I bring up a console in either a new browser window or tab, I can't get to the Action menu to ctrl-alt-del or whatever. It works fine if I open a window within the client window. I had this problem back when it was a Fling as well.

I just sent this to the engineer in charge of the client. I'm not sure if he is still working on it, or handing it off to someone else. It brings up an interesting problem though. You could give that feedback pretty directly when it was a Fling because the engineers could comment directly on the Fling page. Now that it is a supported product, that kind of goes away and is replace with a ton of bureaucracy.

edit: Got a response from him. He is still on it. I filed a bug for the feature to be added. He agreed that the new console window was lacking and needed some attention.

DevNull fucked around with this message at 18:12 on Mar 18, 2016

some kinda jackal
Feb 25, 2003

 
 
DevNull you are the wind beneath our wings

Adbot
ADBOT LOVES YOU

DevNull
Apr 4, 2007

And sometimes is seen a strange spot in the sky
A human being that was given to fly

Martytoof posted:

DevNull you are the wind beneath our wings

No, Etienne is. He is the engineer from the Fling: https://labs.vmware.com/flings/esxi-embedded-host-client#engineers

He just updated the bug with a screen shot of the action menu in a new console window. 20 minutes after I filed the bug.

Don't worry, it will take VMware 6 months to release it :v:

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply