Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Locked thread
long-ass nips Diane
Dec 13, 2010

Breathe.

devmd01 posted:

lol just lol, not everyone works in a megacorp, I can throw a pen across the room and hit our network person. I can tear down and rebuild our production vmware cluster all day long; i'm not going to break anything on the network by running a separate vmware cluster on a completely different network segment, calm the gently caress down.

Someone studying for their CCNA is totally capable of doing this responsibly, though

Adbot
ADBOT LOVES YOU

YOLOsubmarine
Oct 19, 2004

When asked which Pokemon he evolved into, Kamara pauses.

"Motherfucking, what's that big dragon shit? That orange motherfucker. Charizard."

What is the great security threat you're trying to prevent by letting someone set up an isolated test environment? Put them on their own network, firewall everything but ingress management traffic, and let them go nuts.

People are probably already experimenting on your network whether you want it not. You can do a whole lot of experimenting in VMware workstation or Fusion.

some kinda jackal
Feb 25, 2003

 
 
Make sure you adhere to all corporate information security policies if you go this route. They can be far-reaching and just putting something on a segregated subnet may not be enough to put your org out of compliance of certifications/standards or policies.

To be fair, I don't think anyone said "just do it without telling it anyone" but when you get permission make sure you get it in writing and be expressly clear about what your intended use of the lab will be. That way if you get audited you can at least CYA.

Just my two cents as someone who writes corporate policies to prevent exactly what you guys are talking about :v:

MC Fruit Stripe
Nov 26, 2002

around and around we go
The only problem with letting someone set up a lab environment is that it uses company resources in the way of datacenter rack, power and cooling. If you're concerned about it from a network standpoint then your network guys aren't on top of things.

SEKCobra
Feb 28, 2011

Hi
:saddowns: Don't look at my site :saddowns:

MC Fruit Stripe posted:

The only problem with letting someone set up a lab environment is that it uses company resources in the way of datacenter rack, power and cooling. If you're concerned about it from a network standpoint then your network guys aren't on top of things.

You're not gonna run a lab without any internet access. The only internet access in the data center is gonna be the companies.

YOLOsubmarine
Oct 19, 2004

When asked which Pokemon he evolved into, Kamara pauses.

"Motherfucking, what's that big dragon shit? That orange motherfucker. Charizard."

SEKCobra posted:

You're not gonna run a lab without any internet access. The only internet access in the data center is gonna be the companies.

Of course you can. Especially if all you're doing is setting up GNS3 or packet tracer or whatever. If you really want to lock it down then you just force them to use a jump box to move software into the lab, or even make them put it on removable media.

Of course, even with Internet access a virtual lab running a few server OSes is still a much smaller attack surface than your user workstations.

SEKCobra
Feb 28, 2011

Hi
:saddowns: Don't look at my site :saddowns:

NippleFloss posted:

Of course you can. Especially if all you're doing is setting up GNS3 or packet tracer or whatever. If you really want to lock it down then you just force them to use a jump box to move software into the lab, or even make them put it on removable media.

Of course, even with Internet access a virtual lab running a few server OSes is still a much smaller attack surface than your user workstations.

Meh, maybe I'm paranoid from hospital work, but I wouldn't allow that.

YOLOsubmarine
Oct 19, 2004

When asked which Pokemon he evolved into, Kamara pauses.

"Motherfucking, what's that big dragon shit? That orange motherfucker. Charizard."

SEKCobra posted:

Meh, maybe I'm paranoid from hospital work, but I wouldn't allow that.

Can you articulate your actual concerns beyond "not on my network!"?

Sepist
Dec 26, 2005

FUCK BITCHES, ROUTE PACKETS

Gravy Boat 2k
While we're on the topic of network labs, I'll plug my own work here.

Currently I'm building up http://routeandswit.ch/ , it's one-click virtual labs for CCNA, CCNP, and CCIE topics that you can troubleshoot without having to install any hardware emulation software, just use your favorite telnet client to connect to our hosted devices.

While it's not exactly aimed at people studying for certs, it will be useful for those wanting to get exposure to troubleshooting real network issues without having to come up with your own scenarios that you already know the answer to, plus it will be updated frequently with more labs as I come up with more evil ways to break a network.

You need to register on the site to see the "start lab" button, otherwise you can only access the topologies of each scenario and their troubleshooting task list.

iajanus
Aug 17, 2004

NUMBER 1 QUEENSLAND SUPPORTER
MAROONS 2023 STATE OF ORIGIN CHAMPIONS FOR LIFE



Thanks for the help, I agree that the virtual solution is best but I like playing with my hardware to get comfortable with it. I have taken advantage of gns3 to play with more complicated set-ups, though :)

skooma512
Feb 8, 2012

You couldn't grok my race car, but you dug the roadside blur.
We have bad routers/switches at work. I wonder if they're working just enough for me to grab images from them...

Is there any reason for me to not to grab an image from these things for GNS corporate security wise?

Docjowles
Apr 9, 2009

In terms of just the firmware, nah. It's totally generic. The configs are stored separately.

edit: the legality of grabbing that firmware is a separate issue, but you didn't ask about that :ssh:

Docjowles fucked around with this message at 19:14 on Jul 17, 2016

MC Fruit Stripe
Nov 26, 2002

around and around we go
Think of all those people using GNS3 and all the trouble they're going through to purchase every piece of hardware they're emulating, and you pretty much have your answer.

fordan
Mar 9, 2009

Clue: Zero

skooma512 posted:

We have bad routers/switches at work. I wonder if they're working just enough for me to grab images from them...

Is there any reason for me to not to grab an image from these things for GNS corporate security wise?

But are those switches & routers models that GNS can emulate?

Thanks Ants
May 21, 2004

#essereFerrari


I cannot get my head around subnetting and it would suck to fail the CCNA (taking v2 next week) because I need to know how to do something on paper that I currently use Google for.

Has anyone got any resources that worked for them to get this to click?

MF_James
May 8, 2008
I CANNOT HANDLE BEING CALLED OUT ON MY DUMBASS OPINIONS ABOUT ANTI-VIRUS AND SECURITY. I REALLY LIKE TO THINK THAT I KNOW THINGS HERE

INSTEAD I AM GOING TO WHINE ABOUT IT IN OTHER THREADS SO MY OPINION CAN FEEL VALIDATED IN AN ECHO CHAMBER I LIKE

Thanks Ants posted:

I cannot get my head around subnetting and it would suck to fail the CCNA (taking v2 next week) because I need to know how to do something on paper that I currently use Google for.

Has anyone got any resources that worked for them to get this to click?

Actually, Lammle I thought explained it well, he gave you the long version and how to do it, and the next chapter (or maybe 2 later I forget) gives you the ol' "here's how to do this in a sane and faster manner"

MC Fruit Stripe
Nov 26, 2002

around and around we go
I am now running my home lab on Windows 10 and have a problem I didn't account for. The new Windows Update model of "you're getting this patch. you're rebooting." is already proving to be a bit of an issue with keeping my lab up. Go to log into server, no response. Or I try to log into a server hosted on another box, authentication issue because AD has disappeared. This, because the Windows 10 computer running VMware Workstation has rebooted and all of those VMs have been powered off.

Obviously I'm not the first person to run into this. I feel like I've got a couple of options here, and I'm not wild about any of them. Disable Windows updates. Set my account to auto-login, then start VMware workstation and the VMs automatically. Neither of those thrill me.

Anyone else running their VMware lab on Windows 10 and found a graceful solution to the rather constant reboots?

evobatman
Jul 30, 2006

it means nothing, but says everything!
Pillbug

MC Fruit Stripe posted:

I am now running my home lab on Windows 10 and have a problem I didn't account for. The new Windows Update model of "you're getting this patch. you're rebooting." is already proving to be a bit of an issue with keeping my lab up. Go to log into server, no response. Or I try to log into a server hosted on another box, authentication issue because AD has disappeared. This, because the Windows 10 computer running VMware Workstation has rebooted and all of those VMs have been powered off.

Obviously I'm not the first person to run into this. I feel like I've got a couple of options here, and I'm not wild about any of them. Disable Windows updates. Set my account to auto-login, then start VMware workstation and the VMs automatically. Neither of those thrill me.

Anyone else running their VMware lab on Windows 10 and found a graceful solution to the rather constant reboots?

15 seconds of Googling makes it seem like it might be possible to run VMWare workstation as a Windows Service, but I can't confirm it for sure.

Perplx
Jun 26, 2004


Best viewed on Orgasma Plasma
Lipstick Apathy

MC Fruit Stripe posted:

I am now running my home lab on Windows 10 and have a problem I didn't account for. The new Windows Update model of "you're getting this patch. you're rebooting." is already proving to be a bit of an issue with keeping my lab up. Go to log into server, no response. Or I try to log into a server hosted on another box, authentication issue because AD has disappeared. This, because the Windows 10 computer running VMware Workstation has rebooted and all of those VMs have been powered off.

Obviously I'm not the first person to run into this. I feel like I've got a couple of options here, and I'm not wild about any of them. Disable Windows updates. Set my account to auto-login, then start VMware workstation and the VMs automatically. Neither of those thrill me.

Anyone else running their VMware lab on Windows 10 and found a graceful solution to the rather constant reboots?

Ideally you want something else as your hypervisor, windows server, esxi, linux etc.
You can also setup a wsus server on your vm so you can install when you want, and lastly windows updates shouldn't be a surprise, they come out on the 2nd tuesday of the month at 10am PST just set a reminder for yourself.

ErIog
Jul 11, 2001

:nsacloud:

Sepist posted:

While we're on the topic of network labs, I'll plug my own work here.

Currently I'm building up http://routeandswit.ch/ , it's one-click virtual labs for CCNA, CCNP, and CCIE topics that you can troubleshoot without having to install any hardware emulation software, just use your favorite telnet client to connect to our hosted devices.

While it's not exactly aimed at people studying for certs, it will be useful for those wanting to get exposure to troubleshooting real network issues without having to come up with your own scenarios that you already know the answer to, plus it will be updated frequently with more labs as I come up with more evil ways to break a network.

You need to register on the site to see the "start lab" button, otherwise you can only access the topologies of each scenario and their troubleshooting task list.

Wow, this is exactly the kind of thing I've been looking for since beginning to study for the CCNA. I wish there were a lot more labs, though. Do you have more in the works?

Sepist
Dec 26, 2005

FUCK BITCHES, ROUTE PACKETS

Gravy Boat 2k
Eventually. I built that site when working at my previous job which offered a lot more free time to dick around on side projects. Right now I'm busy about 600% of the day so I don't have the time to even finish the dmvpn lab.

If you signed up using a real email I'll probably send out an email when I start adding more labs

Dennis Kucinich
Apr 28, 2013
Just got my CCNA r/s, and looking at tackling CCNA Security next.

Anyone have any tips on setting up a lab environment for this? I was about to grab an ASA 5505 for $115 off ebay, but it sounds like the 5506 is recommended since FirePOWER is now on the exam. The 5506s are running more like $400+, which is prohibitively expensive for me right now. I work in a NOC and have some firewall experience (SonicWALL, pfSense, and iptables), but since none of it is Cisco, I'm sure I'll need some hands-on lab time to have a shot at passing.

Aside from that, I have a couple of 2821 routers and 2950 switches that I purchased for my CCNA studies, and I'm pretty comfortable with GNS3. I'm willing to drop the extra money, if that's what I have to do, but I'm wondering if anyone that's familiar with the 210-260 exam can give me some input.

Sepist
Dec 26, 2005

FUCK BITCHES, ROUTE PACKETS

Gravy Boat 2k
The ASA sends traffic to firepower using a service policy, outside of that they are basically two different devices with no overlap. That said, if you wanna play around with firepower without an asa you can download NGIPSv, install it on a vm and get a demo license from cisco.

MC Fruit Stripe
Nov 26, 2002

around and around we go

Perplx posted:

Ideally you want something else as your hypervisor, windows server, esxi, linux etc.
You can also setup a wsus server on your vm so you can install when you want, and lastly windows updates shouldn't be a surprise, they come out on the 2nd tuesday of the month at 10am PST just set a reminder for yourself.
Yeah it looks like I just got surprised by how many updates and reboots it needed to do. Over the first 3 days of running this PC, I'd guess it rebooted 5 times for updates. None since, now that it's caught up. The initial blast of reboots definitely gave me a "whoa, this is not going to be a very reliable lab" concern, but we're fine.

TheShazbot
Feb 20, 2011

So I have a DL320 G6 1U server at home I purchased from university surplus ages ago for 20 dollars with an E5504 processor in it. (2ghz, quad-core) and it currently has 16GB of ECC DDR3 in it and 4 1TB 3.5" SATA enterprise-grade spinning disks I was given (in a RAID 5 array) and ESXi installed.

Would it be worth it to try and find a more robust processor for the server, or other upgrades? My main machine is a FX-8350 with only 8GB of ram currently, and a battery of various sized disks.

I know this chassis has a different backplane I could purchase and install for 2.5" disks and SAS.

I'm not doing a whole lot with it yet, but will be setting up the following:

A bunch of linux servers (openvpn, webserver, monitoring, automation tools, deployment, possibly mail server using postfix?)
And then a windows domain structure (the usual suspects and SCCM, possibly SQL or exchange depending on how much I hate myself)

I'm a budding sysadmin and need to actually practice and break things. I also have a ton of "ancient" laptops I could press into service as client machines as well.

Any tips or ideas? I'd like to get better kit, but not spend a ton of money given what I already have.

Ciaphas
Nov 20, 2005

> BEWARE, COWARD :ovr:


On a half whim half personal challenge (with no networking or real virtualization experience whatsoever, because I'm a moron) I bought a managed switch today--a Netgear GS108T--with the goal of setting up my Intel NUC, an ESXi host, to have a pfsense router as one of its VMs, that would handle the routing from my cable modem to the rest of my home network. The NUC only having one physical NIC meant that I either had to get a USB3 ethernet dongle that would work in an ESXi VM, or try to set up what googling revealed to me was called a router-on-a-stick topology. I opted for the latter, so:

pre:
                                        /- (port 3)-> NUC (containing pfsense VM and other VMs)
internet->cable modem->(port 2) switch->|
                                        \- (ports 4+)-> other network devices (gaming PC, wireless AP, whatever)
Dunno how dumb or practical that is, but I feel I got pretty close considering I know gently caress all about networking, VLANs, what have you :v:

By the time I quit for the evening and put everything back, I'd gotten the pfsense VM to get a WAN IP address and could communicate with the Internet, but could not communicate with any other devices on the local network. Likewise, local network devices couldn't see it even if it's on the same subnet as the VM host, which other devices COULD see (else I wouldn't be able to log in to the VMs through vSphere). So I screwed something up with either my understanding of subnets, or VLANs, or... something else?

Here's screenshots of what I'd got to before I reset everything. Can anyone tell me what I hosed up?

http://imgur.com/a/dHDKh

(e: apparently imgur loving loves rearranging the album on me, it should be read in order of VLAN names and IDs, VLAN 4 - WAN, VLAN 5 - LAN, PVID Config, and ESXi settings... :mad:)

SEKCobra
Feb 28, 2011

Hi
:saddowns: Don't look at my site :saddowns:
So I took a look at the config and your first mistake is having Port 3 tagged & untagged. Really you should just make it a trunkport (if netgear lets you) or at least have it tagged for all vlans that go into the vm host. The VM host then needs to put the two virtual adapters (plugged into virtual switch) on the two vlans respectively.

E:
I couldn't quite figure out the PVID menu, but you shouldn't have to set anything there.

Thanks Ants
May 21, 2004

#essereFerrari


If you want VLAN 30 (for example) untagged on a port, and VLANs 40 and 50 tagged then in Netgear land you need to change the port PVID to 30 as well or weird things happen.

I hate cheap switches because they have weird quirks like this, poor documentation and a requirement to use a web browser.

Edit: Just saw the photos, the switch config looks right. Where you've gone wrong is tagging VLAN 5 onto the NUC port - that's the untagged VLAN so that pfSense NIC should be in the same virtual switch as the other VMs.

Thanks Ants fucked around with this message at 13:05 on Aug 17, 2016

Ciaphas
Nov 20, 2005

> BEWARE, COWARD :ovr:


SEKCobra posted:

So I took a look at the config and your first mistake is having Port 3 tagged & untagged. Really you should just make it a trunkport (if netgear lets you) or at least have it tagged for all vlans that go into the vm host. The VM host then needs to put the two virtual adapters (plugged into virtual switch) on the two vlans respectively.

E:
I couldn't quite figure out the PVID menu, but you shouldn't have to set anything there.

I was led to believe that allowing multiple vlans on a single port (in this case, vlan 4 tagged and vlan 5 untagged) was the definition of trunking. Was I wrong and that it means something else?

As for PVID, I'm led to understand that if I don't set, say, port 4 (my main PC) to VLAN 5 any ethernet packets it generates don't get tagged VLAN 5 and so can't communicate with anything else on the network--NUC-with-router included. Am I wrong again?

Thanks Ants posted:

If you want VLAN 30 (for example) untagged on a port, and VLANs 40 and 50 tagged then in Netgear land you need to change the port PVID to 30 as well or weird things happen.

I hate cheap switches because they have weird quirks like this, poor documentation and a requirement to use a web browser.

Edit: Just saw the photos, the switch config looks right. Where you've gone wrong is tagging VLAN 5 onto the NUC port - that's the untagged VLAN so that pfSense NIC should be in the same virtual switch as the other VMs.

Yeah it's all purely experimental and for-fun (though if I get it right I may leave it permanently such rather than plug the RT-AC66U back into router mode) so I just went to Fry's and grabbed the first cheapish switch that seemed to review well. I'm guessing I could have done better.

Where do you see that I've tagged VLAN 5 onto the NUC port? Do you mean something's wrong with my vSphere config? Switch said VLAN 5 is untagged across ports 3-8.

Ciaphas fucked around with this message at 16:22 on Aug 17, 2016

YOLOsubmarine
Oct 19, 2004

When asked which Pokemon he evolved into, Kamara pauses.

"Motherfucking, what's that big dragon shit? That orange motherfucker. Charizard."

Ciaphas posted:

Where do you see that I've tagged VLAN 5 onto the NUC port? Do you mean something's wrong with my vSphere config? Switch said VLAN 5 is untagged across ports 3-8.

You have VSphere configured to accept frames tagged for VLAN 5, but the switch is sending frames destined for VLAN 5 as untagged. Look at the VLAN ID for the LAN port group that contains one of your pfsense interfaces. If you're sending untagged then the port group should not have a VLAN ID. Alternately, you should send traffic destined to VLAN 5 as tagged out of port 3. Mixing tagged and untagged traffic on the same port can be confusing.

Ciaphas
Nov 20, 2005

> BEWARE, COWARD :ovr:


NippleFloss posted:

You have VSphere configured to accept frames tagged for VLAN 5, but the switch is sending frames destined for VLAN 5 as untagged. Look at the VLAN ID for the LAN port group that contains one of your pfsense interfaces. If you're sending untagged then the port group should not have a VLAN ID. Alternately, you should send traffic destined to VLAN 5 as tagged out of port 3. Mixing tagged and untagged traffic on the same port can be confusing.

For your alternate, what I found last night when I set port 3 vlan 5 to tagged on the Netgear switch was that I could no longer communcate from my desktop PC (port 4) to the ESXi box on port 3, so I dropped that approach out of sheer confusion. (And I think I can't tag all traffic coming out of ports 4-8 because most local devices don't understand tagged frames, right?)

Other than that... You're suggesting I move the LAN virtual NIC on the vSwitch from the vlan 5 port group to the no-vlan port group, correct? Does that change how I would configure pfSense, i.e. whether I set it up to use VLAN interfaces or use the default em0 (WAN) and em1 (LAN) interfaces?

evol262
Nov 30, 2010
#!/usr/bin/perl

Ciaphas posted:

I was led to believe that allowing multiple vlans on a single port (in this case, vlan 4 tagged and vlan 5 untagged) was the definition of trunking. Was I wrong and that it means something else?
Thats a port with a PVID set, basically.

Networking terminology is kind of brand-specific. What you want is for all frames passing that port to have a tag.

Connect your cable modem to some port. Set that port as untagged, with a PVID of some VLAN (2, whatever).

Port 3 (your NUC) should be configured to accept tagged frames with that VLAN. It should also be configured to allow another VLAN (3, for example). Configure a vSwitch which listens for VLAN 2 (pulling this out inside a vswitch will make your initial configuration easier -- feel free to go wild once it works), and pass that to your pfsense VM.

Create another vswitch for your other VLAN (3), and assign ports from that switch to your other VMs. Also assign a port from that to your pfsense VM, so you can access it.

Configure the management network on ESXi's management console (the yellow and gray/black screen) to use VLAN3.

Set ports 4-8 as untagged, with a PVID of 3.

Ciaphas posted:

As for PVID, I'm led to understand that if I don't set, say, port 4 (my main PC) to VLAN 5 any ethernet packets it generates don't get tagged VLAN 5 and so can't communicate with anything else on the network--NUC-with-router included. Am I wrong again?
That's correct.

Ciaphas
Nov 20, 2005

> BEWARE, COWARD :ovr:


When I get home I'll look into that, but as far as I know I can't create more than the one vswitch when my ESXi host only has one physical NIC. Is that wrong?

YOLOsubmarine
Oct 19, 2004

When asked which Pokemon he evolved into, Kamara pauses.

"Motherfucking, what's that big dragon shit? That orange motherfucker. Charizard."

Ciaphas posted:

For your alternate, what I found last night when I set port 3 vlan 5 to tagged on the Netgear switch was that I could no longer communcate from my desktop PC (port 4) to the ESXi box on port 3, so I dropped that approach out of sheer confusion. (And I think I can't tag all traffic coming out of ports 4-8 because most local devices don't understand tagged frames, right?)

Other than that... You're suggesting I move the LAN virtual NIC on the vSwitch from the vlan 5 port group to the no-vlan port group, correct? Does that change how I would configure pfSense, i.e. whether I set it up to use VLAN interfaces or use the default em0 (WAN) and em1 (LAN) interfaces?

Which network is VLAN 5? 192.168.0.x? Which network is VLAN 4? What network should your VMs be a part of? You've got some mismatches on your port groups between tagged and untagged but it's hard to know where the problem is without knowing what's supposed to be where.

edit: I said it's usually best to tag everything if you're tagging anything on a port. So a single device passing traffic over a single VLAN can sit on an untagged port with the appropriate PVID. That's what most of your edge devices will do because they likely aren't capable of sending 802.1q tagged frames. ESXi can send tagged frames though, and you are using that on the port groups that are tagged as VLAN ID 4 and VLAN ID 5. However you have other port groups on that vSwitch that do not have a VLAN ID associated, so they are sending untagged traffic. This is what's causing confusion, as you've got both tagged and untagged traffic hitting the same switch port and you're trying to handle them in different ways despite them all ultimately having the same goal, which is to get tagged. A better solution is to set the VLANs you want as trunks on the switch (allow tagged frames on those VLANs) and then make sure all of you port groups in ESX have the appropriate VLAN tags. Then there's no confusion about what's lives on what VLAN or where it's tag is being applied. The switch is just accepting tagged frames and forwarding them to other ports that allow those frames.

YOLOsubmarine fucked around with this message at 21:28 on Aug 17, 2016

Thanks Ants
May 21, 2004

#essereFerrari


Also number your VLANs in some sort of resemblance to the subnet (third octet or whatever). No technical reason but it helps with readability.

Ciaphas
Nov 20, 2005

> BEWARE, COWARD :ovr:


Still not home from work but I think as soon as both of you said 'subnet' I maybe realized my problem. For the sake of testing I assigned the switch IP address 192.168.0.2, the pfsense router 192.168.0.1, and my working PC 192.168.0.3. Now that I've had more time to think on it, that sounds like a whoopsie? I'm not really sure though, since the router needs to be able to communicate between the WAN VLAN and the LAN VLAN. AN AN AN.

To answer your question, NippleFloss, my reckoning was that my cable modem and ESXi host (specifically, the pfSense VM) should be on the WAN VLAN (4, right now) and all of the VMs (in the same ESXi host machine! [ed: also including the router!]) plus other network devices (like my desktop PC) would be on the LAN VLAN (5). I guess the ESXi host needing to be on both VLANs while plugged into one switch port is what's causing the confusion here.

Ciaphas fucked around with this message at 00:19 on Aug 18, 2016

Ciaphas
Nov 20, 2005

> BEWARE, COWARD :ovr:


Also I decided to go look up how IP address schemes and VLAN numbers are chosen--pursuant to your post, Thanks Ants--and I guess I didn't realize it was another Holy Scripture thing. IT is weird.

Ciaphas fucked around with this message at 00:29 on Aug 18, 2016

Ciaphas
Nov 20, 2005

> BEWARE, COWARD :ovr:


Going for the triple post wombo combo to respond to an edit~

NippleFloss posted:

edit: I said it's usually best to tag everything if you're tagging anything on a port. So a single device passing traffic over a single VLAN can sit on an untagged port with the appropriate PVID. That's what most of your edge devices will do because they likely aren't capable of sending 802.1q tagged frames. ESXi can send tagged frames though, and you are using that on the port groups that are tagged as VLAN ID 4 and VLAN ID 5. However you have other port groups on that vSwitch that do not have a VLAN ID associated, so they are sending untagged traffic. This is what's causing confusion, as you've got both tagged and untagged traffic hitting the same switch port and you're trying to handle them in different ways despite them all ultimately having the same goal, which is to get tagged. A better solution is to set the VLANs you want as trunks on the switch (allow tagged frames on those VLANs) and then make sure all of you port groups in ESX have the appropriate VLAN tags. Then there's no confusion about what's lives on what VLAN or where it's tag is being applied. The switch is just accepting tagged frames and forwarding them to other ports that allow those frames.

Ah, I misunderstood. I should have been clearer in my screenshot: All those other VMs (linux, media server and vpn) are irrelevant, they were shut down the whole time I was testing and I wasn't gonna bring them up until either a) I'd gotten my PC on the internet through the pfSense VM--in which case I'd move all those other VMs to the LAN VLAN on the vSwitch--or b) I'd given up and put everything back behind the Asus router, so I saw no point in reconfiguring them while I was trying things :v:

1000101
May 14, 2003

BIRTHDAY BIRTHDAY BIRTHDAY BIRTHDAY BIRTHDAY BIRTHDAY FRUITCAKE!

Ciaphas posted:

Also I decided to go look up how IP address schemes and VLAN numbers are chosen--pursuant to your post, Thanks Ants--and I guess I didn't realize it was another Holy Scripture thing. IT is weird.

I'd generally recommend customers pick VLAN IDs however makes them happy. For example I tend to make inside VLANs be something < 1000 and DMZ and outside VLANs might be north of 2000.

The more important thing is how you lay out your subnets. You typically want to assign them on easily summarized boundaries so if you have say 3 sites you might say that 10.0.0.0/12 is HQ, 10.16.0.0/12 is San Jose, 10.32.0.0/12 is Seattle, etc.

Then in there carve up a little bit more. Maybe 10.0.0.0/24 is used for loopbacks, 10.15.0.0/16 is used for DMZs and everything in between is carved up into /16s. Then those /16s get carved up into /24s that get allocated.

This comes in handy if you're using static routes or dynamic routing (which you should be doing!) as you can quickly ID where a network is and they're all neatly summarized.

edit: just realized this is the home lab thread so just pretend you're doing this in GNS3 or something.

1000101 fucked around with this message at 01:12 on Aug 18, 2016

Adbot
ADBOT LOVES YOU

YOLOsubmarine
Oct 19, 2004

When asked which Pokemon he evolved into, Kamara pauses.

"Motherfucking, what's that big dragon shit? That orange motherfucker. Charizard."

Ciaphas posted:

Still not home from work but I think as soon as both of you said 'subnet' I maybe realized my problem. For the sake of testing I assigned the switch IP address 192.168.0.2, the pfsense router 192.168.0.1, and my working PC 192.168.0.3. Now that I've had more time to think on it, that sounds like a whoopsie? I'm not really sure though, since the router needs to be able to communicate between the WAN VLAN and the LAN VLAN. AN AN AN.

To answer your question, NippleFloss, my reckoning was that my cable modem and ESXi host (specifically, the pfSense VM) should be on the WAN VLAN (4, right now) and all of the VMs (in the same ESXi host machine! [ed: also including the router!]) plus other network devices (like my desktop PC) would be on the LAN VLAN (5). I guess the ESXi host needing to be on both VLANs while plugged into one switch port is what's causing the confusion here.

So, like I said, your issue is that your port groups don't match. Your ESXi host management network is on a port group on you vSwitch that is sending traffic untagged. The Port group that your pfsense router lives on is expecting tagged traffic. These two port groups need to have matching VLAN settings because the interfaces connected to them are on the same network and connected to the same switchport. If one is expecting to send/receive untagged traffic (the management port group) and the other is expecting to send/receive tagged traffic (the pfsense LAN interface) then one of them is going to be unhappy irrespective of the switch config. The quickest fix would be to remove the VLAN ID from the LAN port group. It's less clean than explicitly tagging all port group traffic, but it will get things correct quicker.

The way router on a stick would work in this scenario :

1) Traffic comes into your network through your cable modem
2) The modem forwards this to the PF sense VM's WAN IP address. When the frame arrives at the switch on port 2 it will add the 802.1q VLAN header, tagging it as belonging to VLAN 4, because that port is configured with with a PVID of 4 (untagged frames belong to VLAN 4).
3) The switch will then forward this frame out of port 3 (the arp table will tell it where the packet it going) with the VLAN 4 tag still in place, because port 3 is set to forward traffic destined to VLAN 4 with tags in place (T port not U port in the GUI). It will arrive at the ESX host and get forwarded to the vSwitch that contains the PFsense WAN interface. ESXi network stack will look at the VLAN tag, see that it is tagged for VLAN 4, verify that the interface being targeted belongs to a port group that allows VLAN 4, and so it will strip the header and forward the packet along to the PFsense WAN interface. The important thing here is that the tag gets applied at one end of the communication (ingress to the switch from the modem) and stripped at the other end (leaving the vSwitch destined for the VM).
4) The PFsense VM receives the (no longer tagged) frame and sees that it is destined for your internal network. It routes the VM onto your LAN segment and forwards it out of the PFsense LAN interface.
5) The LAN interface belongs to a port group with no VLAN ID, so it the vSwitch forwards the frame back to your Netgear switch with no VLAN tag applied.
6) The switch accepts the frame on port 3 and tags it for VLAN 5, because the switch is configured with a PVID of 5 on port 3.
7) It then forwards the frame out of port whatever towards your PC, on whatever port it is connected to. That port is set to pass traffic to VLAN 5 as untagged (U port, not T port) so the tag is stripped off and the traffic is delivered to your PC as untagged ethernet traffic, which it can read and understand. When it responds the frame is retagged as VLAN 5 when it enters the netgear switch, then forwarded back toward the PFsense WAN interface out of port 3 with the tags stripped, because port 3 is set to send traffic to VLAN 5 as untagged. The routing happens in reverse and then the frame is forwarded out of the ESXi host now with tags for VLAN 4 because it is leaving the PFsense WAN interface. It arrives at the switch with VLAN tag 4, and is forwarded on to the modem with the tags being stripped as it leaves port 2.

The crux of this is you must be mindful of what the netgear switch and the vswitch are expecting. If one is sending tagged traffic then the other must be configured to expect tagged traffic. If one is sending untagged then the other must be configured to expect untagged. The endpoints (physical or virtual) don't really know or care about any of this, the tags are added and removes as the packets move through the switching layer. It's helpful to remember that the ESXi host isn't an endpoint, it's a switch with endpoints behind it. You've got two switches that are sending VLAN tagged traffic back and forth between them, and they need to agree.

  • Locked thread