It’s one of those times when the culmination of things either rumbling in perpetuity or looming in the distance have me feeling very overwhelmed and woefully unproductive. A perfect storm of anxiety and depression and a little despair, and as much as I push to just get through each day, it’s there waiting for me again whenever I finally wake up the next day…
I’m currently in the process of replacing my entire team (of 2) at work after one sadly passed away this summer and the other resigned to move on to something new. This is the first time I’ve ever hired anyone, so I’m a little apprehensive about that, but I think more so trying to figure out how my team will work going forward as I transition from experienced engineers to people who are brand new and will require a lot more hands on support from me.
On top of that, it seems like everything is breaking at work quicker than we can fix it, which is equally daunting with my last teammate leaving in just a couple of short weeks.
Next week I also have to do a sleep study, which is basically an insurance technicality to confirm that I still have the sleep apnea that was diagnosed 10+ years ago so that they’ll pay for a replacement CPAP machine for me. This has me anxious because in order for the test to be accurate, I have to go off my machine for three nights, meaning that I’m going to be tired and groggy and miserable because by now I know that I get absolute garbage sleep without it.
Seriously, as much as I hate using the thing, I know that I can’t sleep without it – even naps on the couch leave me feeling more tired than when I lay down – but it’s something that I have to endure because I’ve had just the one machine this entire time and apparently I’m lucky that it didn’t die on me years ago!
And of course, all of this has driven both my time as well as my desire for writing into the ground, which sucks because before it really hit a few weeks ago, I was just starting to make some progress on a few things again. 🙁
It hasn’t all been negative recently, however, as I’ve managed to get a handful of random projects done in an effort to distract myself from all of the other stress:
I rewired and did some necessary cable management behind the TV in our living room.
I added 10-gigabit NICs to two of my servers at home, and was reminded something important about troubleshooting in the process. “Correlation does not mean causation!”
I made a small, but pivotal dent in starting to clean our garage.
And I think I might’ve finally found a solution for managing our kids’ iPads to make that whole process a little less cumbersome. (more on this soon!)
We’re also just today getting a bunch of deliveries to give our playroom/Sara’s school room a much needed overhaul, so even though it’s going to mean a lot of work for me this weekend tearing down all of my old Lego shelves and putting together new bookshelves from Ikea, I’m looking forward to the end result and seeing that room transform into something a little more organized and less cluttered.
I feel like right now the best I can do is choose one or two things a day to focus on, and then in the meantime just do what I can to keep everything else at bay until such time that I can give them the attention that they actually need. I know that with some of this stuff, in particular hiring, I realistically won’t feel great about until probably six months after I hire people and build up a new rapport working with them, so some of this stress is here for the long haul.
Albeit more slowly than I would like, things will eventually get done and one by one I can start to put more of this stuff behind me … and then new things will present themselves to take their place, and we can start this whole mess all over again!
Well, hopefully not as overwhelmingly next time, mind you. 😛
Today is the first day that I’ve ever been in our attic.
Well, I’ve stood on a ladder with my head in the little hole before, typically just long enough to decide, “Nope!” and figure out something else to do that didn’t involve combining my fears of heights and enclosed spaces and falling through the ceiling and breaking any number of limbs…
But today I actually had to go inside because I wanted to finally get the cable for our wifi access point off the floor and mount it to the ceiling properly.
I’ve watched a lot of videos of people running cabling through their attics, so I figured it wouldn’t be that big of a deal, but little did I know that falling through the ceiling is only one of many things one has to worry about in the attic! The big one is, well, attics aren’t really designed for larger people to fumble about up there – between all of the insulation and air ducts and random ceiling joists, it’s not like there’s just a straight shot across the house up there and there certainly wasn’t any standing room like I’d seen on YouTube! I found tons of dividers (firewalls?) and just extremely cramped quarters and exposed nails and sharp metal to avoid, but by the time I’d mustered the nerve to get up there I was determined to see it through.
Even though I had drilled my hole from the other side and shoved a bunch of the wire through, I hadn’t thought about the insulation hiding everything so I had a hard time finding where I was supposed to be going until I finally had my wife grab a pole and tap on the ceiling where I needed to go … which at that point was almost comical because not only had I been climbing in the wrong direction in the attic, but it was literally only a couple of feet from the hole whereas I had to have been somewhere in the middle of the living room by that point.
The other end of the cable needed to run into our bedroom closet which is where all of the servers are located, which also ended up being a chore because I guess you never think about how all of these unique ledges and shapes are represented in the actual walls behind the drywall. In this case, there’s this weird “inverse ledge” in our closet where part of the ceiling is lower than the rest … which is where I had originally wanted to run the wire through because it would put it in the corner and as out of the way as possible.
It turned out, though, that I think said weird ledge is actually a wooden box inside of the wall, so again I couldn’t find the wire that I had fished through. After more tapping, Sara finally suggested that I just make a new whole a lot closer where it would be easy to find so that I could get out of the attic already and be done with it, so I listened to my wife and about five minutes later I was done!
By then I was thoroughly covered with fiberglass particles that are even now after a shower still lingering, but it’s done and I’m very glad that I don’t have to do anything else up there. In fact, I’ve got another cable running to the TV that I want to hide and I honestly think that I might just drill a hole right through the wall like I did into my office on the other side of the closet rather than deal with the attic anymore.
And when we build our next house, I’m definitely having the builders pre-wire everything with ethernet in every room at the highest grade we can to try and future proof it because if I struggled this much going into the attic at 41, I can only imagine trying to do it again at 55!!!
It’s been a while since I’ve done much tinkering with my home servers – frankly because I’ve been pretty satisfied how they’ve been running as-is!
I think it was back in 2018 that I migrated the first half of my setup to a rackmount (old enterprise) hardware, with Plex and everything else running under virtualization for the first time; and then in 2019 I added 4U of NAS boxes to move all of my media and put running on desktop hardware to an end.
For the most part, it’s been great – I’ve got a couple of beefy batteries to help endure blips with the power, and Unraid is so much better at managing disks than me shuffling stuff from one external hard drive to another … particularly when you’re looking at upwards of a dozen disks…
I actually believe I’m running 15 now and I’ll probably add more if I can find a good sale in a few months for Black Friday, but regardless…
The limitation of using old enterprise hardware is that there’s only so much you can do to upgrade them. Dell considers all of the machines that I’m running to be 11th generation, whereas currently they’re on 15th generation, so when it comes to things like processor upgrades and even drive support, there’s not a ton of wiggle room.
Not a big deal, considering that as I said they’re running pretty smoothly, but still … I guess you could say I got bored and felt like seeing what I could tweak anyways? 😛
About a month ago, I settled on three relatively simple upgrades that I could do to bulk my main server that runs my VMs up a bit to at least help prolong until I’m able to replace it with a new build altogether…
New CPUs – This sounds like a big endeavor, but I literally found a pair of matched CPUs on eBay for like $40.
Add SSDs – This is the one I just finished, and it wasn’t as easy as I had assumed!
10 gigabit ethernet – I’ll do this one later on this fall … I don’t have a 10-gig switch yet, so I figured in the short term I could run 10-gig just between the two servers and at least see some speed boosts there.
Graphics Card for Plex Transcoding – Probably my last update because A) the card alone runs about $500, and B) I have to do some other mods like replacing a riser in the case just to get the thing to fit. Still, this will be the biggest impact because it should render all Plex transcoding a walk in the park…
Swapping out the CPUs probably made me the most nervous – I mean, you’re basically doing brain surgery on your computer – but aside from a brief scare where I couldn’t find my thermal paste, but the replacement went super smoothly and it booted up no problem with the new chips a few minutes later!
I went from 4 cores each running 2.5 Ghz (16 cores total w/hyperthreading) to 6 cores each running at 3 Ghz (24 cores total), which will basically give me a little more overhead for Plex transcoding while I wait to add a real graphics card.
And now on to today’s adventure, or should we say last week’s adventure???
My Plex libraries have gotten pretty massive – think thousands of movies and TV shows across over 100 TB of media – and one recommendation that I’ve heard to improve performance as you grow bigger is to move your metadata and database over to an SSD. I guess it’s not so much the interface speed itself, but the IOPS because with the metadata you’re dealing with tons of tiny files with all of the images.
Anyways, I originally wanted to go nVME because I understand they’re by far the fastest drives available right now. Unfortunately, there was no good way to get them into my system. Sure, I could buy a card to add them, but RAID wouldn’t be supported and I’ve come to appreciate the redundancy because I know how easy it is for drives to fail…
So instead I went with just regular, old SSDs because I could fit them right into my normal drive bays and then my existing RAID controller could just manage them like normal. Easy, right?!
Well, apparently the PERC6i RAID controller that I had in my Dell R610 doesn’t like SSDs, or at least doesn’t like Western Digital Red SSDs, because it would detect the drives at startup but refuse to do anything with them, reporting failures.
I tried swapping drive bays, thinking it was a bad cable.
I tried upgrading the firmware for the card, which OMG took forever because the server is so old that the online firmware update process no longer works.
Finally I decided to try replacing the card altogether because I knew that the PERC6i couldn’t use larger drives, either … not a problem in this case, but still. I ordered an H700 off of eBay instead and waited a week, then waited another week while I tried to figure out how to backup all of my VMs just in case!
As of now, everything is up and running as expected, so luckily I didn’t end up needing those backups, but it sure was close for a minute this afternoon…
Installing the card itself was pretty easy – thanks to this video which walks through the whole process…
That said, the cables that shipped with the card that I bought weren’t long enough to route through my case correctly, so now it won’t close! I have replacements coming in a couple of days that cost me another $20 on top of the $40 I spent for the card, cables, and new battery.
Once I booted up the new card, the new SSDs showed up immediately as Online!
Not so much for the existing drives, but that was an error on my part because I missed a prompt to “Import the Foreign Array”. A second reboot to correct that showed everything and a few minutes later I was initializing the new array and doing a background initialization on the old array which finished by the time I was done with lunch.
But we weren’t out of the woods yet!
Upon booting back into ESXi, it too saw the new SSDs and I was able to create my new datastore where I wanted to move my Plex VM to … but the old array was nowhere in sight and as a result, it thought that all of my existing VMs were invalid.
That sort of made sense because I figured that the identifiers for the drives probably changed with the introduction of the new card. After a healthy amount of digging, I found that the trick was to SSH into ESXi itself and then do this…
esxcli storage vmfs snapshot list
esxcli storage vmfs snapshot mount -n -l "SNAPSHOT_NAME"
The first command confirmed that my datastore still existed and the second one mounted it so that ESXi would recognize it as usual. (this link explains it all)
From here it was simply a matter of unregistering each VM, moving them one by one to the new SSD datastore, and then re-registering them as existing VMs. Apparently the one downside of doing it this way is that the disk provisioning type for each VM changes from thin to thick, which means they take up their full disk allocation instead of only what they’re using at the time … but I’m really only using the new SSD datastore for two VMs and they’ll both probably get rebuilt eventually for OS upgrades anyways, so it wasn’t worth the hassle with vCenter/vSphere/whatever that I honestly don’t understand yet anyways.
They’re moved, they’re running again, and we’re done!
Now can I tell much of a difference???
I was hoping to see faster download speeds on the one VM, but not so much there.
For Plex itself, the interface both on my laptop, phone, and a TV locally as well as my phone over cellular data does seem a lot snappier. Artwork seems to pop onto the screen in maybe a second per page on the TV when it would take a couple to browse through pages of large libraries, so that’s a win in my book.
And again, I wasn’t exactly expecting huge gains with any of these, but aside from pounding my head against the wall for the last week about the stupid RAID card, both were fun, little upgrades that added incremental gains to the performance, so that’s cool in my book.
Next up will likely be the 10-gig upgrade because I know what parts I need and just need to get them ordered. The goal is to string the two servers together for starters, and then maybe add a 10-gig switch next year along the same time I upgrade to a new Macbook that can also support 10-gig, too!
My longer term goal, I’ve decided, is going to be to eventually build new servers myself to replace all of these ones. Still rackmount, but I want to do them myself because new enterprise gear is ridiculously expensive and I think I’m comfortable enough with it now to go at it alone. I’ve gotten a lot of inspiration from watching Linus Tech Tips and the evolution of their own server room at their office, so ideally it would wait until we build a new house in a couple of years where I can:
Install a full rack in a dedicated room (that isn’t our bedroom closet!)
Build two identical servers so that I can upgrade to a cluster to make it easier to make changes without taking Plex down
Build a replacement NAS, probably in a single 4U case
Along with a matching backup NAS
As you can see, it’ll be a big endeavor that will take some time to put together, but I’m in no rush and I think my current rig will keep us moving forward just fine until then.
We’ll see if the other two upgrades are crazy enough to warrant a blog post of their own!
This week, amid Tropical Storm/Hurricane Elsa, we did something that we honestly should’ve done years ago – added a solar heater to our pool.
Once the bad weather passed and the sun had a chance to come out for more than an hour, the effect was seriously like swimming in bath water!
Yesterday I set the pool at 92 degrees and it was almost too warm, compared to the 82-85 degrees we’ve been seeing the last couple of weeks in June since I started measuring.
Today I backed it off to 90 and at 11:30am when we first went, it was just about perfect.
I’ve found that 80+ is enough to enjoy during the summer and feels refreshing when the sun is out, but you still get that shock when you get in for the first time – particularly when the boys in your swim trunks first hit the water – and after dark once the sun is down and the kids are asleep, it’s a bit chilly to relax and unwind in.
90, on the other hand, is like stepping into a giant bathtub and brings back memories of swimming down in the Florida Keys in the middle of July … without the salt water, thankfully. I was even able to use the heater to get the hot tub up to 102 degrees during the day, which fell to around 97-98 by the time I got the kids to bed and could actually enjoy it for myself.
Considering that our spa heater has literally been broken for years, this was a nice little bonus!
The only downside that’s going to take some getting used to is that Matthew is a lot more comfortable in the warmer water, which has kept me on my toes the last couple of days. Lately I’ve been taking all three kids swimming by myself because Sara has been having problems with her ear, and I can mostly handle it because Christopher and David can usually handle themselves and Matthew would spend a lot of time playing on the pool deck, so I could relax and just catch him when he wants to jump in…
…now, however, he’s gotten a lot braver between jumping in without waiting for me and swimming over his head where he can’t make it across without assistance. He’s also been doing this thing that Christopher used to do where he can’t judge when he’s tired and needs a break, so he just keeps going and going and struggling more along the way. ☹️
Hopefully we can push him to build up his skills, just like his brothers did, to where these won’t be as big of a deal, and once Sara’s feeling better that will give me an extra set of hands, too.
Anywho, I’m really curious to see how these solar panels manage once we get out of the hottest months of the year because the selling point that we heard wasn’t so much warmer temps but a longer swimming season – as in, this should in theory add another month or two to each end of when we normally use our pool, which is typically around May – October for the kids and maybe June – September for us thin-skinned adults!
If it could keep the temps into the high 80s or even 90 for March/April and October/November, that would be amazing, but there are a lot of other factors like the position of the sun in the sky that I’m just not familiar enough to say, so we’ll see what happens.
In the meantime, though, the consensus is that solar heat is a win! 😉
Writing Prompt – What’s the last movie you saw and how would you make it better?
This is an easy one – Fatherhood, with Kevin Hart, and I’d improve it by not killing off the wife in the first act.
Aside from the fact that it’s been done a thousand times, I kind of shook my head when it played out on the screen because I was expecting a solid comedy with Kevin Hart and this plot device just made it way too sad too quick. Plus, as a father myself, there are plenty of wacky fatherhood scenarios that the movie could’ve played out without him needing to be a single Dad. Or if you want him to explore that helplessness anyways, make the wife a career woman and put him in the stay at home Dad role.
It just really set the movie off on the wrong foot for me, which admittedly was already going kind of slow, to have that traumatic scene, and the funeral, and everyone gathered at the house, and then all of the pressure from his mother-in-law about them wanting to take the baby home with them?!
Maybe it wasn’t meant to be nothing but laughs like his other movies, but I was definitely looking for something lighthearted and funny that night so to be hit with more of a drama was a little disappointing.
By my best estimation, I think we spent something like 3-4 hours in the pool today. It was basically all we did today, and it was fantastic.
The kids admittedly didn’t get as much pool time as we would’ve liked last week due to scheduling and rain conflicts. I think the last two nights we had to cancel on account of thunderstorms, which of course is very common this time of year in Florida, so out of fear that the same could happen again today for a third day in a row, I ended up taking them out this morning to get in an hour or two before lunch…
…which also has the added bonus of quieting down the house so that Sara could sleep because she both worked overnight last night and is working again tonight…
Anyways, arguably the pool is one of my easiest ways of entertaining the kids because they’re allowed to get a little wild (within reason) on account of being outside, it wears them out like crazy which makes naptime/bedtime so much easier, and it’s a good workout for me, too.
And it’s fun, of course!
Usually I’ll spend some time diving with David & Christopher while Matthew plays around on the pool deck, and then once his brothers start jumping that’s what he wants to do which turns me into the aquatic version of an air traffic controller to both keep his head above water and at least try to prevent everyone from bumping into each other … the latter of which is still very much a work in progress!
After our pool time, Christopher asked if we could eat lunch outside on the patio – another thing that we’ve also been enjoying lately, so we ate and watched iPads for a while before one by one I started peeling them off for naptimes. Christopher and I then sat around by ourselves a while longer, with me doing some work and him playing with his Mario Kart cars, until we eventually made a quick trip to the gas station for ice and snacks which resulted in us getting a bit drenched by the afternoon downpour.
Thankfully, however, it only lasted maybe twenty minutes and the heat outside still made the pool a welcoming place, so after naps and saying goodbye to Mom, we headed back out for round #2 and swam for another hour while waiting for pizza to get delivered for dinner!
In reflection, it just reminded me of one of those classic, lazy summer days where all of your time is either spent in the water or eating nearby it. It would’ve been nice to have a grill or something for maybe some shrimp or fish, but then again it would’ve been nice for Sara to be able to join us, too, because I certainly can’t man a grill and supervise three kids in the pool at the same time anyways!
We need more days like this over the summer. I’ll circle back to the grill point at a later date… 😉
I honestly just don’t know what to think about this pandemic anymore.
It frustrates me that vaccinations have slowed down so significantly – a month ago on May 22nd, 2.3 million doses were given out, compared to just 647k doses yesterday on June 22nd.
It also confuses me that the case and death numbers have fallen as much as they have, which don’t get me wrong is obviously a good thing. It just leaves me wondering if we’re missing something … are states getting lax about their reporting (Florida switched from daily to weekly recently) or are more cases going unreported???
I wish I knew what it’s supposed to look like for me to believe that it’s over because although the graphs above look promising even despite vaccinations running at only 45% of people vaccinated instead of the 70% that I thought we were aiming for a few months ago.
I guess I just worry that after we’ve all been through and sacrificed so much, what if there’s another surge that we could’ve prevented by taking precautions just a little bit longer or by more people being eager to get vaccinated themselves. My wife and I are both vaccinated and we still wear our masks everywhere because we’re worried about bringing it home to our kids, and yet it’s hard to see mask rules pretty much on a whim at this point where you know that those most adamantly against them just aren’t wearing them and still refuse to get vaccinated.
It’s weird for me to see Disney not only removing the last of its precautions around the parks, but also absolutely packing them in with people who are ready to move on with their lives … despite today seeing 365k cases around the world and 8,600 deaths.
And it’s damn scary to see Republican politicians going out of their way to pass legislature against things like schools requiring vaccinations or even masks, or businesses being allowed to require proof of vaccination for going on cruises in the name of “personal liberty.”
I’ve been taking my shoes off at the airport now for twenty years because one time a guy tried to blow up a plane with bombs in his shoes – what about that personal liberty?!
I think a lot of it makes me uncomfortable because it forces me to be more judgmental of the people around me for the sake of protecting my family. If everyone still wore masks or even if the vaccination rates were still up to show that the public trust was high around everyone just being careful, it would make me feel better about trying to ease back into some of the things that we did before COVID-19 changed all of our lives.
I saw one person joking about becoming agoraphobic over the last year and I don’t know if I’d go that far, but my anxiety is definitely a bit more on edge when I have to be around people who I don’t know if I can entirely trust because I don’t know if they’ve shared the same attitude towards the virus and staying safe as I have.
It leaves me wondering what back to normal will look like for me and just how long it’s going to take regardless of whatever the world chooses to do around me… 🙁
I’ve despised this 30-some-odd foot monstrosity pretty much since we bought this house back in 2012. It drops leaves and acorns like it’s slowly dying on my front lawn, even though it’s definitely not, it’s impossible for me to prune because of how huge it’s gotten, and it’s even started to tear up the sidewalk and our driveway.
If left to its own devices, I have little doubt that before too long this innocent-looking oak tree would’ve displaced the entire house, leaving just one gargantuan behemoth of a tree in the center of our tiny, suburban lot and the family that once lived there forced out onto the area formerly known as our sidewalk before it ripped that whole thing to absolute shreds, too!
Admittedly it actually was pretty impressive to see the crew work. It wasn’t cheap, but these guys rolled up with three trucks and two massive cranes along with an industrial-sized wood chipper that buzzed through 8.5 years of my oak dread like a hot knife through a tree.
I griped a lot about our HOA over the years, but the process was surprisingly easy. I probably spent a week or two getting quotes from various tree removal companies – the one we picked wasn’t the cheapest option, mind you, but I was more confident in these guys than the next quote that was about $300 cheaper and when I watched them work, I knew we’d made the right choice. We didn’t have to get a local permit because the tree was considered a nuisance (tell me about it!) due to the damage its roots were doing, and I think even the HOA had their approval back to me in less than a week!
From the final approval, it took about a month for scheduling to get these guys out, and once they arrived the whole thing was GONE in maybe an hour?!
They still need to come back in a few days to grind out the stump so that we can try to grow grass over the oak tree’s unmarked grave, but all in all I was pretty impressed.
And relieved. We literally just had the AC worked on in our van because acorns were clogging up the pipes and causing water to back up into the system. Just before that, I raked 4-5 bags of leaves myself and then paid somebody who raked another 10 bags out of my tiny, little front yard!
Good riddance – don’t let the wood chipper hit ya where the lumberjack split ya… 😛
Writing Prompt – Where would you fly right now if you could hop in a plane?
Let’s assume that this implies we’re not in the middle of a global pandemic because my travel anxiety is high enough without adding in the threat of communicable disease!
I’ve always wanted to travel to Europe to check out all of the castles of the Middle Ages. I’m not sure the extent to which they let you just randomly wander around – if it’s simply a look at the castle walls or you can actually walk the dungeons and towers and explore all sorts of secret passages that for all I know are based more in movies than reality.
When I was a teenager, we had a camping trip at Northern Michigan’s closest equivalent to a castle and I got to ditch my lowly tent on the grounds for an opportunity to actually sleep inside! I don’t remember a whole lot about it, except that the room we stayed in was made of stone and it definitely felt the part to a 14 year-old who’d always looked up to the likes of Robin Hood and the Three Musketeers as veritable role models…
It does make me wonder with so many castles over there, does looking at them ever get “old” as in, “If you’ve seen one castle, you’ve seen ’em all…” or are there enough varieties and details to explore that it’s like people who tour classic churches or museums where there’s always something new to discover?
I don’t know if it will happen before we retire, but it’s definitely a trip that I’d like to make eventually … and if there’s an opportunity to spend the night inside, well, then that’s just a bonus! 😉
I honestly wasn’t really swayed last fall when this was released because I didn’t have any of these things as a kid and admittedly most handhelds are just too small for my big hands and old eyes to play for any length of time these days. But then Nintendo just announced that this year they’re releasing a Zelda version – which is easily in my Top 5 favorite games of all-time – and I also happened to find the Mario one on sale, so here we are…
Even though by now I’ve played the original Super Mario Bros on probably half a dozen different platforms (NES, SNES, Wii, WiiU, Switch, Arcade Cab, Computer Emulator), for some reason it still amused me to see that iconic screen in full color on a tiny, 2.5-inch portable screen. And it’s not like I don’t have scads more graphically impressive games on my phone, or that I could carry around literally thousands of games on a 1 TB micro-SD card the size of my thumbnail.
Fun Fact – Super Mario Bros on the NES was only 40 KB in size!
I think what really makes me smile is that Christopher has gotten a kick out of playing it, even though it’s not nearly as shiny and feature-rich as Super Mario Odyssey and Super Mario 3D World that he’s playing right now. Here’s a pic from 2018 of him trying to play, but more or less just running into pits and Goombas…
And yet three years later he can play along with the game that not only introduced his Dad to video games, but also was the predecessor for the Switch games that he loves exploring today!
I’m curious to see if the next one will spark a little interest in the Zelda franchise for him because although he hasn’t had much interest to date, you never know what will catch their eye next! 😉
…either way, I’m sure I’ll enjoy playing through the first dungeon randomly before my eyes start seeing double or my hands cramp up on these tiny, tiny buttons…