Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
underlig
Sep 13, 2007
We're running hyper-v server core on a host with two active guests, both running server 2008r2 standard.

One of them, the file/dns/dhcp server, locks up once a month or so. No clients can connect to it, rdp doesn't work but the crappy thing is that it also does not log anything in the eventlogs neither on guest or host (which is weird).
We end up killing it through the hyper-v manager.

(the second guest is running but not used by anyone)

What/Where should i start to look?

a systeminfo > out.txt for the host is here
http://pastebin.com/kxcuEUxt
and for the guest
http://pastebin.com/EyH7KE89

What free alternatives are there for monitoring this thing next time it locks up?

** edit
I'm an idiot, i give troubleshooting information like i'm one of my users.
The server is an IBM x3650 m3 with a lsi-made IBM ServeRAID M5015 raid controller, two 136GB 6gb/s SAS drives in raid 1 for the hosts's system, two in raid 1 for the guests .vhds and two regular sata 1tb drives which the files the fileserver-guest is serving are placed.

The megaraid tool says that everything's running fine.

Apparently the fileserver responded to ping this morning when it locked up, but nothing more than that.

All integration-services are active.

** another edit
Looking at scheduled tasks on the client, there's one that ran at 01:30 this morning and the one at 05:00 didn't.
At 02:00 Symantec BackupExec is scheduled to do a straight file-backup, INSIDE the guest with no host/hyper-v knowledge at all. Can that be it? There's a start at 0200 and a end at 0200,so something in it was aware of something at that time.
It's run fine for the last month..

underlig fucked around with this message at 13:35 on Jun 12, 2012

Adbot
ADBOT LOVES YOU

underlig
Sep 13, 2007

Mierdaan posted:

It does, but the licensing terms basically require that the hypervisor instance of Server 2008 is used only to run the virtual instance; you can't set up that bare-metal instance of Server 2008 to do anything other than run the Hyper-V role.

You can just use the same license key, afaik.

edit: source
Wouldn't it be better in that case to run hyper-v core server? (since hyper-v server core is "free" there is no difference in what you pay).

What you loose is the ability to start/restart/setup guests directly on the host and most other settings, you need hyper-v manager installed on another machine, but at the same time i figure there would be "less" of things that needs updates when you do not have the regular windows gui. Am i completely wrong?

I'll actually expand that question, are there less updates and as a result less restarts due to installation of those updates on server core machines?

underlig
Sep 13, 2007
I'm YOTJ:ing in less than a month, for a position where i said "I've heard the name Xen but that's about it".
They were willing to give me training but i fear i'm to stupid to learn anything if i cannot try my way through it first.
What can i do to get more familiar with it? Words that were used in the interview was "xen" "xenapp" and "citrix", versions unknown.

I figure i could atleast install XenServer on some kind of hardware i have at home, the bad thing is that most of my stuff is old crap, a couple of laptops with centrino or core2duo cpus, the only really good machine is my htpc that has an i3. Will i even be able to install xenserver on the i3? I could sacrifice movie-night for a week or so if it means i can fiddle around with the technology i'll be working with.
Shared storage? I have an old consumer nas (dlink dns 323) is it at all possible to rig that up for the simplest/cheapest possible storage?

What i HAVE used is hyper-v on single servers, no san anywhere, esxi5 on a single server just to see how it works, virtualbox for non-critical stuff and a vmware workstation at home running freebsd for the last five years.

underlig
Sep 13, 2007
Can anyone guess why my host-attached storage is so slow in esxi5?

I'm running on an old IBM x3650 (m0) with two 146GB sas-drives in raid1 (i think). 24GB ram, only one Xeon E5345@2.33GHz-cpu. The storage adapter is an ServeRAID 8k/8k-l8. Esxi runs from usb-stick.
This server is NOT on the hcl, not for esxi5 anyway. It also does not support passthrough.

I tried running this at my HOST
# date && dd if=/dev/zero bs=1024 count=1000000 of=dd.file && date
Mon Aug 13 07:28:02 UTC 2012
1000000+0 records in
1000000+0 records out
Mon Aug 13 07:30:18 UTC 2012

(I'm no *nix guru) but it should mean that writing 1GB of data started at 07:28:02 and finished at 07:30:18 which means it took two minutes 16 seconds?

Running the same at home, in a GUEST bsd where the host is an old laptop with 5400 or 7200rpm sata-drive running server 2003 x86 and virtualization provided by vmware workstation version old,
Mån 13 Aug 2012 09:29:06 CEST
1000000+0 records in
1000000+0 records out
1024000000 bytes transferred in 32.284491 secs (31718016 bytes/sec)
Mån 13 Aug 2012 09:29:39 CEST

The same command on the same laptophost but in another guest in virtualbox instead takes 62 seconds.

Even if it's not on the hcl, why would a server-class machine, with sas-drives, run so insanely much slower than what a laptop does?

There is 0 room to purchase "SAN", the only "NAS" we have is a consumer-grade zyxel pos where copying an iso to/from the esxi-server took hours.

Nothing makes any sense to me, the only thing i know is that something isn't right.

(This is my last week at this place so i probably won't care to "fix" this, it runs fine for the crap it's supposed to do, i'm just curious if i've unchecked something that should be checked, or if i've done something that is the biggest NO-NO or what..)

underlig
Sep 13, 2007

cheese-cube posted:

What firmware is the ServeRAID adapter running? I've encountered multiple issues with ServeRAID 8k adapters in the past which were resolved by upgrading the firmware. Also there is a known issue with the SAS-expander backplane on the x3650s although that is only supposed to affect SATA HDDs.

Just make a bootable update disk with IBM BoMC and use that to update the firmware of all components.

Edit: I've just noticed that you might need to use v3.00 of BoMC as they seem to have discontinued support for all M0 System x servers in the newer versions.
Cool, thanks, building the iso now. Also just realized i cannot determine the old version without actually rebooting the host, serveraid manager needs something to connect to and vpshere client does not tell me anything usefull more than the name of the controller.
Will try to update anyway and see what happens.


What could possibly go wrong.. :)


Kachunkachunk posted:

dd is not a good indicator of performance due to imposed limits for console resources. You'll want to test performance using a VM with appropriate benchmarking software.

If you really want to use the console, you can do some tests like creating eager zeroed thick VMDK files and time that.
E.g.: `time vmkfstools -c 1G test.vmdk -d eagerzeroedthick`

Testing reads might still be an issue for the console. Really, just use a VM!

Edit: Wrong reply button... plus removed some derpiness.
Yeah i downloaded vmmark etc but i figured there would be some kind of use to try dd from the console anyway.
But these imposed limits, they're still there if i say copy a vmdk from vsphere client right? I mean it's the same slow deal there? I remember when i had problems resizing a vmdk and did a copy of it before testing, it took hours.

underlig
Sep 13, 2007

Corvettefisher posted:

Whom is interested in some VDI/View/Server QA on Thursday 18th live over the phone/voip.
As we're talking about testing VDI at newjob i would be interested in atleast listening to an offline version of a Q/A later on,
I don't really do live stuff, and will most likely be too tired / busy to attend it.

But good initiative man.

underlig
Sep 13, 2007
Me and my colleague was hired at a small it-company at the same time as all the old techs quit. The documentation of old jobs are often non-existing and tickets 70-75% have the resolution "Fixed it" and nothing more.

So one of our customers is a large printing company and they have merged with another company. The other company previously had a corporate it-department.
When they merged a new server was setup to handle prepress jobs, 50gb+ files with rendered data that are used and sometimes re-used (therefore storage is vital).

The server was setup as a hyper-v host running 2012 with a 1tb drive for the guest.

The guest was setup with a 600gb disk for printjob storage (e:).

Now one of the old technicians apparently created a snapshot right before me and my colleague was hired, this has created a differencing disk that when full will grow out of the physical space on the host.


I do not recognize "differencing disk" effect, snapshots i've done have not resulted in one of these that i know of.

The biggest problem is that i right now cannot see how much free space i have, the host has been showing 41gb for a few months, i've been on sick-leave and none of my colleagues have bothered looking at this during that time.
Drives as the guest see them:


* Can i somehow merge these two vhds into one 900gb+ file then shrink the space for the guest?
* Is it possible to do this live?
* Is it possible to guestimate how long it will take to merge?

I talked to the supplier of the prepress-software and he told me they recommend 1-2 tb of disk for the client, 600gb is too small, and they didn't understand why it was setup this way.
I told the customer that the easiest way would be to just upgrade to larger drives, expand the array and that would solve it, but they do not want to spend money (unless it's for macbooks and iphones).

* I have a possible plan B and that's an older HP StorageWorx unknown version-server but that will require them to purchase two fiberchannel cards to get the transferspeed they need between the host and the storageworx, i found a couple of 4gb fc cards, do you expect that to be fast enough or do i need to buy 10gb?

underlig
Sep 13, 2007

H2SO4 posted:

What's the timestamp on the last full backup of that VM?

The last full what? (Would a full backup of it remove the differencing disk -part? I could do a backup to the storageworx-server, probably.).

This host is located at the client and there are no backup at all configured for anything there. This is something that was agreed before i started, i have no idea why anyone would run something as "business critical" and not do backups.

To add to the insanity they repurposed the storageworx server with 2tb-drives totalling something like ~17tb of usable space. This has neither backups nor monitoring configured.


you know what, it's saturday evening and i realize more and more that i am slowly going insane here. YOTJ is still a thing right?..

underlig
Sep 13, 2007

Moey posted:

I hate P2V conversions.
What is your least hated p2v method? Is disk2vhd still what's used?

underlig
Sep 13, 2007

Moey posted:

So what's the word on that performance hit?

Edit: good read linked in the storage thread.

https://lonesysadmin.net/2018/01/02/intel-cpu-design-flaw-performance-degradation-security-updates/

When talking about lowered performance, how is this measured? Are there any benchmark tools that i can run before and after the patch, to have some numbers to show boss?
Boss has had the week off so this is something i'm going to have to answer on monday, i patched one of my hyper-v hosts today but i can't tell what the impact is because we dont have any monitoring at all. :bravo:

Adbot
ADBOT LOVES YOU

underlig
Sep 13, 2007
I didn't know 2012r2 with Hyper-V was that common anymore, but apparently KB5009624 breaks Hyper-V
"could not be started because the hypervisor is not running"

We're running Esxi instead, but that's also making the beginning of tyol 2022 hard,

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply