|
Vulture Culture posted:We had the problem you're talking about in Kilo and it went away when we identified and went around the MySQL issue where you can't actually do multi-master in Nova with Galera. We front our MySQL cluster with HAProxy with the MySQL backend in backup mode so that only one instance is interacted with for exactly that reason.
|
# ? Aug 28, 2015 02:07 |
|
|
# ? Mar 28, 2024 16:14 |
|
I just want to whatever genius decided that the message queue should just be used to tell services a new record was written and to check mysql instead of pub/subbing the actual content and writing it to the database as a failsafe. It's so hosed that a replacement was needed, but trove is just as bad in different ways
|
# ? Aug 28, 2015 05:09 |
|
evol262 posted:I just want to whatever genius decided that the message queue should just be used to tell services a new record was written and to check mysql instead of pub/subbing the actual content and writing it to the database as a failsafe.
|
# ? Aug 28, 2015 05:37 |
|
Vulture Culture posted:We had the problem you're talking about in Kilo and it went away when we identified and went around the MySQL issue where you can't actually do multi-master in Nova with Galera. I just got told to edit nova.conf, and add this to all nodes: code:
I just checked the defaults... 1 hour, man this is so janky. code:
ILikeVoltron fucked around with this message at 07:08 on Aug 28, 2015 |
# ? Aug 28, 2015 07:00 |
|
ILikeVoltron posted:I just got told to edit nova.conf, and add this to all nodes:
|
# ? Aug 28, 2015 07:01 |
|
Vulture Culture posted:Back up -- what does your database architecture and configuration look like? It's literally a vanilla install from redhat's OSP6. They deploy a wsep based trifecta for the database, using ha proxy as the front end (with server pinning) and replication. It's running on 3 x 56 core boxes with 128 gigs of memory or round about (all bare metal) and a pair of disks in raid 1. We're talking about 10 VMs total on the system (on say, 28ish nova hosts).
|
# ? Aug 28, 2015 07:11 |
|
ILikeVoltron posted:It's literally a vanilla install from redhat's OSP6. They deploy a wsep based trifecta for the database, using ha proxy as the front end (with server pinning) and replication. It's running on 3 x 56 core boxes with 128 gigs of memory or round about (all bare metal) and a pair of disks in raid 1. We're talking about 10 VMs total on the system (on say, 28ish nova hosts). I'm not really an OSP person (upstream only there, both ends on RHEV), but I have an old (Juno) RDO (upstream of RHOS at the time, now RHEL-OSP) running on two dell 9020s running more VMs than that. Do you have a case #? Can you PM it to me? I can at least go look at the sosreports and bits they've made you attach.
|
# ? Aug 28, 2015 15:23 |
|
evol262 posted:I'm not really an OSP person (upstream only there, both ends on RHEV), but I have an old (Juno) RDO (upstream of RHOS at the time, now RHEL-OSP) running on two dell 9020s running more VMs than that. Sent! Thanks for taking a look at this.
|
# ? Aug 28, 2015 18:32 |
|
Are there any good books/guides/online thingies for virtualization? I have a high overview concept of VMs, I finally have a bare metal VM machine with ESXi on it and a couple of basic guest OSes (Windows Vista for loving with those "I work for MS and your computer is infected", and Linux). I'm looking to start working with virtual switches and routers, and some VLANs. I'm wanting to start setting things up, but I'd like to get a better handle on virtualization in general.
|
# ? Aug 29, 2015 14:56 |
|
If you are the type that learns by immersion, go make a Xen box and read about OVS and Openflow. Even getting a basic network with vlans set up with ovs and open flow is fairly involved and will get you exposed to write a bit quickly.
|
# ? Aug 30, 2015 15:31 |
|
Have any of you OpenStackers successfully migrated from Nova networking to Neutron? We are still using Nova networking at present but want to begin laying the groundwork for a switch to Neutron, because our tenants are increasingly asking for features that are only in Neutron, we expect to eventually scale our clusters to the point where we will need something like Calico to maintain a sane network topology, and Nova networking just doesn't get any love any more (and why would it?). I'm pretty new to the OpenStack world, but I've already heard all sorts of horror stories about Neutron. A lot of them are pretty old and I imagine things have improved since then, but I'd like to hear from any ops that are currently using it, particularly if you had a prior nova-network-based setup.
|
# ? Aug 30, 2015 17:25 |
|
I've got two probably dumb questions related to Windows Sever 2012R2 within ESXi (home lab): 1. I want to sync time between ESXi and my two R2 VMs (and other future VMs); I've set ESXi to sync with time.windows.com, and checked the 'Synchronize guest time with host' for both VMs (both have VMware Tools installed). This should take care of it, correct? 2. I realized this morning that both of my hosts had shut down unexpectedly and I don't know when it happened (the time settings were all messed up for both VMs too); how can I set up notifications so I'm informed when one or both hosts shuts down or the entire system reboots for whatever reason?
|
# ? Aug 30, 2015 17:26 |
|
GobiasIndustries posted:I've got two probably dumb questions related to Windows Sever 2012R2 within ESXi (home lab):
|
# ? Aug 30, 2015 17:49 |
|
adorai posted:This may be a holdover from the binary translation days, but it used to be best practice to actually run NTP on each guest. Linux: http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1006427 Windows: http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1318
|
# ? Aug 30, 2015 20:32 |
|
Potato Salad posted:If you are the type that learns by immersion, go make a Xen box and read about OVS and Openflow. I'm not really a Linux person. Of what I'm reading, it's very Linux heavy. If that's the best solution I'll learn Linux on top of it as well, unless there's something else.
|
# ? Aug 31, 2015 00:46 |
|
The reference implementation is on Linux, but any switch vendor's openflow/ovs stuff is just as arcane. SDN on Windows is young. Expect to have to learn another OS/paradigm (or more than one) to do anything serious with SDN.
|
# ? Aug 31, 2015 02:20 |
|
You aren't likely to need any SDN experience for an enterprise job. Just buy a cheap managed switch and install esxi, you can learn a lot with those two things.
|
# ? Aug 31, 2015 02:27 |
|
May as well repeat the common: Fanless procurve Powerconnects if you can deal with fans (or swap them) GNS3 for playing with complex setups If you really want virtual routers in a "not emulating Cisco/juniper kit" way, you'll need/want SDN, even if it's just basic vyatta
|
# ? Aug 31, 2015 03:45 |
|
evol262 posted:May as well repeat the common:
|
# ? Aug 31, 2015 04:13 |
|
chutwig posted:We front our MySQL cluster with HAProxy with the MySQL backend in backup mode so that only one instance is interacted with for exactly that reason. Do you happen to know how this is configured? Like, how did you tell the other members of the MySQL cluster to be backup mode?
|
# ? Sep 3, 2015 00:29 |
|
I don't know if this is a question for here or the enterprise hardware thread but: I want to deploy an R620 standalone ESXi host for testing. It's got the integrated mirrored SD slots in the back. Each came with a 2GB card. Is that going to be fine for a hypervisor out of the box, or should I look to add larger cards? No vCenter, no extra logging, just straight host for VMs. I might install that standalone WebUI but that's about it. edit: That is to say, it's got like 2TB datastore, I just want to throw the hypervisor alone on the SD cards. some kinda jackal fucked around with this message at 03:40 on Sep 3, 2015 |
# ? Sep 3, 2015 02:21 |
|
ILikeVoltron posted:Do you happen to know how this is configured? Like, how did you tell the other members of the MySQL cluster to be backup mode? It's configuration on HAProxy only, MySQL has no awareness of it. "Backup mode" in the context of HAProxy means that HAProxy will only send traffic to a single backend instead of spreading it around the different backends: https://github.com/bloomberg/chef-bcpc/blob/master/cookbooks/bcpc/templates/default/haproxy-head.cfg.erb#L34-L44 Our setup leans heavily on the notion of a master virtual IP that floats around the cluster, i.e., there's one control plane node (head node) that is more equal than all the rest, and that's whichever one is holding the VIP. That node serves MySQL and HAProxy exclusively and also runs cronjobs that should only run on one system, everything else that goes through the VIP is distributed by HAProxy to all the control plane nodes. You'll need something like that for our jankity MySQL+HAProxy workaround to work for you.
|
# ? Sep 3, 2015 02:32 |
|
Martytoof posted:I don't know if this is a question for here or the enterprise hardware thread but: Should be fine, I was deploying M620's at my last job with ESXI and 1GB Mirrored SDCards with no issues at all. ESXI doesn't make use of the SDCard for anything after booting
|
# ? Sep 3, 2015 08:12 |
|
chutwig posted:It's configuration on HAProxy only, MySQL has no awareness of it. "Backup mode" in the context of HAProxy means that HAProxy will only send traffic to a single backend instead of spreading it around the different backends: https://github.com/bloomberg/chef-bcpc/blob/master/cookbooks/bcpc/templates/default/haproxy-head.cfg.erb#L34-L44
|
# ? Sep 3, 2015 10:29 |
|
Martytoof posted:I don't know if this is a question for here or the enterprise hardware thread but: Esxi will recognize that the memory is SD and avoid putting /scratch on it. It'll ask you where you would like to place that on install. VMware writes that 1GB is possible, but you may not have enough space for a coredump. Four gigs is recommended. Either works fine enough. After a ton of calls with Dell about this, it turns out a rep cannot add the SD riser to the server without giving you the 2GB sd cards. They can, however, sell you the SD risers in a new order group, downgrade to their cheap as dirt 1GB cards, and you buy 4GB wear-leveled Kingston or SanDisk SD cards yourself on the cheap.
|
# ? Sep 3, 2015 14:46 |
|
I'm on my phone right now, but vmware has a good page on install partition hardware considerations Google finds easily.
|
# ? Sep 3, 2015 14:49 |
|
chutwig posted:It's configuration on HAProxy only, MySQL has no awareness of it. "Backup mode" in the context of HAProxy means that HAProxy will only send traffic to a single backend instead of spreading it around the different backends: https://github.com/bloomberg/chef-bcpc/blob/master/cookbooks/bcpc/templates/default/haproxy-head.cfg.erb#L34-L44 Ok, so the haproxy config out of redhat's installer is "stick on dst" and "timeout server 90m" like so: code:
code:
|
# ? Sep 3, 2015 18:11 |
|
ILikeVoltron posted:Ok, so the haproxy config out of redhat's installer is "stick on dst" and "timeout server 90m" like so:
|
# ? Sep 3, 2015 21:06 |
|
Any major gotchas on vCenter 6 at this point? I'm bringing the DR site up from active/standby to active/active and the documentation makes it sound like linked-mode configuration is way easier on 6 than 5.5, but after the clusterfuck of the 5.0 release I've been a bit wary of jumping to a new version until its had time to settle.
|
# ? Sep 3, 2015 21:37 |
|
ILikeVoltron posted:Ok, so the haproxy config out of redhat's installer is "stick on dst" and "timeout server 90m" like so: We don't run Ceilometer any more because it kicked the grap out of our control plane nodes. Vulture Culture's suggestion of sar is good because it'll let you corroborate the stalls to load average or iowait issues going on on the node at that time. How do you mean how we're doing replication between the MySQL hosts? They all talk to each other directly, it's normal whatever Galera does.
|
# ? Sep 3, 2015 23:24 |
|
chutwig posted:We don't run Ceilometer any more because it kicked the grap out of our control plane nodes.
|
# ? Sep 4, 2015 01:07 |
|
Thanks for the SD card info everyone. Embedded hypervisor is my new favourite thing ever
|
# ? Sep 4, 2015 01:09 |
|
I migrated a dev DB from my old host to the new 620. The old host had a 120GB SSD that I was using for VFRC and the 620 has nothing. When I try to power on the DB I get an error that the 0 bytes of VFRC available on the host are insufficient to fill the request for 60gb which -- doy -- I have no VFRC. When I go to edit the VM in the webUI I can't see any option to remove the VFRC reservation, however! I don't want to migrate back to the old host just to remove VFRC and then migrate back. If I edit the VMX would that be enough? I would have thought vMotion would have warned me about this. Did I catch an edge case here? Wait nevermind I'm an idiot; I forgot VFRC is defined under each individual disk. 11pm is probably not the best time to muck around with vSphere. sorry everyone. some kinda jackal fucked around with this message at 04:12 on Sep 4, 2015 |
# ? Sep 4, 2015 04:08 |
|
I have a folder in a datastore running on ESXI5 that I just cant access. I try to browse it through the datastore browser and it just sits there searching forever. I tried to ssh in and enter it and it does the same. Its like just this one VM's folder is corrupt. Any ideas?
|
# ? Sep 10, 2015 17:41 |
|
Anyone else go to LA VMWorld?
|
# ? Sep 10, 2015 22:49 |
|
Stealthgerbil posted:I have a folder in a datastore running on ESXI5 that I just cant access. I try to browse it through the datastore browser and it just sits there searching forever. I tried to ssh in and enter it and it does the same. Its like just this one VM's folder is corrupt. Any ideas? Local datastore or on a storage device?
|
# ? Sep 10, 2015 22:50 |
|
Also is it 5.5? You can run VOMA to investigate further. And boot into ESXi 6.0 (for free) to do a repair using VOMA (don't have to install it). http://kb.vmware.com/kb/2036767 Fake Edit: The Live CD style method I mentioned is potentially not public... but the above KB does link to some documentation on what switches to run. If you're in a bind, I'll find the steps for you tomorrow.
|
# ? Sep 10, 2015 23:25 |
|
Ugh is there any way to programatically rename all the files associated with a VM when you rename the VM itself? You're supposed to be able to do a storage vmotion to rename the associated files but for that to work you have to move it to a DIFFERENT datastore. If you "vmotion" it to the same datastore it doesn't do any renaming. I have like 30 VMs I want to change the name scheme of and i know it doesn't actually MATTER what the files are named behind the scenes but I'm anal like that so it looks like I'll be spending the next day removing, renaming, and importing by hand.
|
# ? Sep 11, 2015 01:23 |
|
Martytoof posted:Ugh is there any way to programatically rename all the files associated with a VM when you rename the VM itself?
|
# ? Sep 11, 2015 01:31 |
|
|
# ? Mar 28, 2024 16:14 |
|
Potato Salad posted:Anyone else go to LA VMWorld? Is this LA vmug or something? I went to US vmworld this year and the VMware parts of it were garbage, some good sessions and vendor meet ups though.
|
# ? Sep 11, 2015 04:08 |