What Are You Doing Right Now
-
Took a break from writing to do some quick photography. Back to writing.
-
My street
-
RHCE / Ansible practice time.
-
New Test Page on Fedora 34
-
Doing a Detailed DR Plan, like a step by step on how to recover from different scenarios.
Like Building burns down, or we loose both our clusters.
Another is Ransomware.
What would your first reaction/actions be if someone phones up and says "a pop up just happened saying everything is encrypted". Would you unplug and switch off everything, then look into it?
Or would be just switch their machine off and isolate it to investigate.Personally think my reaction would be switch all servers off VMs and physical. But is that a bit OTT?
-
Happy fucking Monday…..
Forwarded From +1314: Jared, this is Curt at M. We are getting this message throughout the warehouse.
-
While me…..
-
@jaredbusch Don't worry, google didn't let their cert lapse.
-
So walking somebody through stuff on the phone while I drive.
I can ping the DNS server which is CF and Google. But no look up’s work.
-
@jaredbusch said in What Are You Doing Right Now:
So walking somebody through stuff on the phone while I drive.
I can ping the DNS server which is CF and Google. But no look up’s work.
If it were only CF, I'd figure the ISP was doing something to 1.1.1.1, but Google's DNS addresses generally aren't effected by those shenanigans. Good luck!
-
@jaredbusch said in What Are You Doing Right Now:
So walking somebody through stuff on the phone while I drive.
I can ping the DNS server which is CF and Google. But no look up’s work.
What is doing the lookups?
-
If the gateway is the DNS for the internal clients, try having one of them go direct for lookups.
I'm assuming rebooting the firewall didn't solve anything?
-
Trying out official preview build of W11 on a VM
-
@dashrender said in What Are You Doing Right Now:
I'm assuming rebooting the firewall didn't solve anything?
No.
-
It looks great, very fast and smooth!
https://timothygruber.com/wp-content/uploads/2021/06/img_60da37dbd5be2.png
-
@dashrender said in What Are You Doing Right Now:
If the gateway is the DNS for the internal clients, try having one of them go direct for lookups.
OMG so much broken shit...
So the problem is that their AT&T is down due to a power failure at the DMARC that they have no access to. I popped the door open with a screwdriver. Nothing in there has power. Fire alarm, other monitoring gear, nothing.
Rest of the building has power though. So fun times.
On to what's so fucked up.
The private network 192.168.1.0/24 is working and DHCP is a Windows 2012 R2 DC. But it hands out and old SBS server that does not exist as the primary DNS, then itself as the secondary DNS. The gateway is a pfSense box on 192.168.1.5 and uses a Charter Coax connection to get to the internet.
But that's not all! The DC is not the owner of the FSMO roles. I can't access the DNS MMC because of that. The old SBS server is still the FSMO role holder of ALL the roles.
But the network not working was the warehouse. This network gets its DHCP from the pfSense box and is on 172.16.44.0/24 and routes out the AT&T pipe. This DHCP only had 1.1.1.1 and 8.8.8.8 as the DNS entries.
I had hit the firewall to route the warehouse to the charter network early on, but it was still failing. No idea.
Once I arrived on site and found out that the AT&T service was down, I changed the routing again and this time it worked. Everything is going out the Charter pipe. I did also disable some weird blocker service module that was installed in pfSense so that might have helped.
Enough said, they are online and I left to dela with the customer I was supposed ot be coming to St Louis to deal with.
-
@obsolesce oh no, not already.
-
Working on a video, then RHCE practice time.
-
@hobbit666 said in What Are You Doing Right Now:
Doing a Detailed DR Plan, like a step by step on how to recover from different scenarios.
Like Building burns down, or we loose both our clusters.
Another is Ransomware.
What would your first reaction/actions be if someone phones up and says "a pop up just happened saying everything is encrypted". Would you unplug and switch off everything, then look into it?
Or would be just switch their machine off and isolate it to investigate.Personally think my reaction would be switch all servers off VMs and physical. But is that a bit OTT?
What's worth more, a few hours of unplanned but controlled downtime or recovery?
-
@jaredbusch said in What Are You Doing Right Now:
@dashrender said in What Are You Doing Right Now:
If the gateway is the DNS for the internal clients, try having one of them go direct for lookups.
OMG so much broken shit...
So the problem is that their AT&T is down due to a power failure at the DMARC that they have no access to. I popped the door open with a screwdriver. Nothing in there has power. Fire alarm, other monitoring gear, nothing.
Rest of the building has power though. So fun times.
On to what's so fucked up.
The private network 192.168.1.0/24 is working and DHCP is a Windows 2012 R2 DC. But it hands out and old SBS server that does not exist as the primary DNS, then itself as the secondary DNS. The gateway is a pfSense box on 192.168.1.5 and uses a Charter Coax connection to get to the internet.
But that's not all! The DC is not the owner of the FSMO roles. I can't access the DNS MMC because of that. The old SBS server is still the FSMO role holder of ALL the roles.
But the network not working was the warehouse. This network gets its DHCP from the pfSense box and is on 172.16.44.0/24 and routes out the AT&T pipe. This DHCP only had 1.1.1.1 and 8.8.8.8 as the DNS entries.
I had hit the firewall to route the warehouse to the charter network early on, but it was still failing. No idea.
Once I arrived on site and found out that the AT&T service was down, I changed the routing again and this time it worked. Everything is going out the Charter pipe. I did also disable some weird blocker service module that was installed in pfSense so that might have helped.
Enough said, they are online and I left to dela with the customer I was supposed ot be coming to St Louis to deal with.
WOW sounds fun
