I think I’m making progress, albeit in a number of avenues that I wasn’t necessarily expecting!
One thing standing out that is seeming to be more of a problem than I would’ve thought is that simply put – I’ve got a lot of WordPress installs on this server!!! Sorting through them all, I came up with something like 20 installs in total, which is amusing because I’ve only got like 10 domains currently registered … apparently I’ve built up a healthy list of test installs in addition to some really old ones that I never post to and thus don’t really think about.
Now I wouldn’t have thought this to be much of an issue until I was able to start digging into specific processes running slow along with their associated URLs and database queries, and it turns out that WordPress’s own cron function has been at least part of the source of my problems, for a couple of different reasons:
A) Across those 20 installs, a number of them weren’t up to date – some of them being so old that I had to update them manually (gasp!), and more prominently some outdated plugins that either also needed to be brought current or in some cases removed altogether for instances where I’m not even using them anymore (i.e. I used to heavily use a backup to Dropbox plugin, but I’ve since come to rely more on system-wide backups and I don’t even have Dropbox installed on my laptop today).
B) Also, I still need to learn more about how WP-Cron actually functions, but I think there may have been some cases where many sites were trying to all run at the same time, which is just a general recipe for disaster! From what I’ve read so far, it sounds like WP-Cron actually triggers based on page views … which confuses me how my test sites were even triggering … but one option here might be to disable WP-Cron and instead tie them into real cron jobs at the OS level so that I can scatter them throughout each hour instead of triggering arbitrarily.
I’m not entirely sold on B just yet, but realizing that I have so many separate installs definitely reinforces my curiosity around WordPress multi-site, which I was playing around with earlier this summer and actually abandoned when I decided not to redesign some of my sites. But from a resource management perspective it still might make sense, even if I just try to pull some of the like-minded sites into one install, or possibly even a bunch of the test sites, just to help get the numbers down!
All in all it’s a work in progress, but so far my last high load notification was at 5:50 pm last night and I updated a bunch of my installs in the hours since, so hopefully I’m starting to work through the issues and load is at least going down a bit! Mind you, it doesn’t help that I don’t really know what’s a reasonable daily page view volume that my current server should be able to handle … and granted, now that I’ve started playing around with routing some of my traffic through Cloudflare, I’m sadly reminded about how much junk traffic a server gets that never even becomes legitimate pages served (like brute force attacks, scanning, etc…).
One other tool that I’ve found that’s been helpful specifically in pinpointing the cron issues has been New Relic, which is actually a probe that I had to install under PHP on the server itself but then in turn monitors all sorts of neat stats about processing times and whatnot. I’m just using the free Lite version they offer now and it’s already been enlightening – definitely worth checking out!
