UPDATE 2015-11-22 11:59:44: I believe the previous developer changed the wp-config file to point to a staging DB. When I deleted the staging DB, it brought down the entire site. Media Temple is not at fault, and actually have gone a bit above and beyond helping me resolve this.
Concerning my @mediatemple frustrations last week: looks like a previous dev did some wonky things to wp-config. MT was helpful in all this
— Casey Driscoll (@caseydriscoll) November 22, 2015
This last week I had some absurd problems with a Media Temple installation on a project I just inherited. It was on their premium WordPress tier. As of last night it entered nightmare mode.
I’m not even looking for fault or blame. I just want my sanity back and find the best tool for my clients.
TL;DR: I THINK SOMEHOW, SOMEONE OR SOMETHING CHANGED THE NAME OF OUR MEDIATEMPLE DATABASE. THE INFORMATION IN THE WP-CONFIG DOES NOT MATCH PHPMYADMIN, EVEN IF I DO A ROLLBACK TO THREE WEEKS AGO TO WHEN THE SITE WAS WORKING
— Casey Driscoll (@caseydriscoll) November 21, 2015
The original goal was quite simple, build a staging instance so the client may build a new ‘magazine issue’ every 6 months or so, before pushing live to the production server.
Being new to the Media Temple environment, and given the shortness of time to launch the feature, I chose to build the staging site on a personal server, as I new that environment well. I can usually create a full copy of a site in under 30 min. I do this often.
The typical steps are:
- Install staging site on personal Digital Ocean (DO) server [worked as expected]
- Zip themes and plugins directories on production server (MT) [worked as expected]
- wget the zip files from prod to DO. Something like http://website.com/wp-content/plugins.zip [worked as expected]
- Unzip new plugins.zip and themes.zip in new DO instance [worked as expected]
- Install wp-migrate-db-pro on new DO server and production server [worked as expected]
- On the DO server, PULL a migration from the production server [NOPE, BORKEN]
It turns out WP-Migrate-DB-Pro is not supported on the basic WordPress server packages. As confirmed by support, this is because ‘there is a migration tool in place’. This is fine. I’m not sure if the failed PULL caused the issues. I don’t see how it could, but it may have.
I also took a stab at using their staging installer, which seems fine, if not a bit slow. As far as I understand it now, you can push and pull files between the two installs, but you cannot sync the DB in either direction. Additionally, I’m not yet sure how to control the subdomain on the staging site (to make it ‘new.website.com’ for example), but perhaps it is easier than the docs I found suggest. I only first noticed the connect to DB errors on Wednesday night, after deleting an older staging site. I’m not sure if they are related.
Either way, the site was down and this standard error was coming up:
A couple of frustrations came up while troubleshooting.
First, I just moved to a new apartment, and the wifi was super slow (1MB/sec). This was causing ssh timeout errors and lag, and generally decreasing my patience. (I updated to an xfinity day pass, which overlapping my apartment and is consistently 10MB down. Worth the $8).
Second, to get chat support for the Media Temple install, I had to create my own account, even though I was trusted by the client with the admin credentials for the main account. This is probably just policy, and that’s fine. But it was a pain in the ass as I was trying to put a fire out, to be told I had to jump through a number of (seemingly) extra hoops. More frustrating, once I created an account, cookies made me appear as the admin account still to the support chat and only worked when I created a private browsing session.
Third, Google Chrome seems to cache a version of a site, for certain errors, or in certain locations, or something? As a web developer, I’m used to hitting command-shift-R, for the “I’m serious, refresh without using cache!” and then using command-shift-delete to move immediately to delete the last hour of history. Or something (see number four).
Fourth, commically, when I finally get in the support chat, as I was going through all these steps to see why I was getting ‘Error establishing a database connection’, the support agent informs me the site is up and well. I had JUST confirmed with the client and with http://downforeveryoneorjustme.com/ that the site was down globally. On his statement, I fired up Safari and low and behold the damn site was up. HOW. I hit refresh on Chrome and it still said ‘Error establishing a database connection’. Finally I think I went into Chrome Dev tools and deleted some sort of resource frame or something? I forget That managed to do a full refresh and I saw the site in Chrome.
So at this point Wednesday night I have confirmation from myself, support agent and client that site is up. Everything previously must have just been a fluke? OK. Let’s just walk away, as I’m sure it will continue to run just fine.
I was busy for 48 hours and didn’t get another chance to look up until Friday evening. Apparently the site was down since sometime Thursday, just reporting the same ‘Error establishing a database connection’.
Fifth, there is no access to php or server error logs (that I could immediately tell). This is my standard troubleshooting method, as the log will explicitly tell you why it is failing. Chat agent recommended turning on WP_DEBUG in the wp-config, which I did. That turns the standard ‘Error establishing a database connection’ into a slightly better one.
Sixth, tweaking the damn wp-config file. At this point, I’m at the end of my rope. As I dig deeper to fix an otherwise VERY SIMPLE AND ROUTINE ISSUE, I keep creating more and more problems. I burning precious time, and falling farther behind. AND the problem is not only worse that before but I don’t know why. This is happened before, and I typically have the skill set and patience to deal. At this point in the troubleshooting routine, I start employing simple sanity checks. Namely: kill it all. If all the pipes are correctly in place (as I’m currently presuming them to be), then I should be able to fatally break the install at will. For example, placing a random char in the wp-config.php file SHOULD (from my experience) bring down the whole site, as php will fatally error out and give a white screen. NOPE. Ok, try moving the entire wp-config file to wp-config.old. That should make /wp-admin/setup-config.php appear. KINDA NOPE. Ok, after a few refreshes you get the screen below. BUT IF I ADD WP-CONFIG.PHP BACK IT STILL SHOWS.
Chrome just keeps reshowing the old ‘need config file error’ while Safari is showing ‘Error establishing a database connection’. There is some sort of weird caching between me in the server. Is it Chrome, xfinity, THE PIPES, or Media Temple?
AND NOW I’M JUST CELEBRATING THE FACT IM SEEING A DATABASE ERROR. AHHHAHAHAHAH. CUE EMMA WATSON AWKWARD FACE TWEET.
It’s late, I’m exhausted and prone to mistakes (and grouchy) so I go to bed and attack on Saturday morning. @mediatemplehelp says they may be able to help, so I submit official ticket and go to bed.
I wake up and nothing is really resolved, although I do get a reply back. They recommend I restore from previous back up, so I restore from November 1, well before I had access to the site. I’m not sure if the DB is restored or just the files. I notice a new wp-config.php though, dated on the server for October 16. But the contents inside are even weirder than expected.
:clingspillow: (You can do this Casey, you’re a professional)
As I am completely lost at this point, I decide to purchase a SECOND WordPress install to see how the defaults are set up.
New WordPress Install
To prevent myself from going crazy I bought a new $30/month WordPress site package. I needed to see how the default are set up. At this point my sanity is worth the cost, and I’m hoping I can get it refunded after they read this (hi guys!) as it was just a troubleshooting tool.
The top two images are the phpMyAdmin and wp-config files for the new blank install. You’ll notice the Database Name and $table_prefix match in both images, as expected. Even the server ports are the same (:3310), which is doubly expected.
In the next two images at the bottom for the current production site, you’ll notice that neither the database name or the prefix match between phpMyAdmin and wp-config. EVEN THE DAMN PORT DOESN’T MATCH. ¯\_(ツ)_/¯
I’m not sure what is an alias and/or what is standard procedure. As far as I know, to fix this, we have to get the wp-config to match the database info.
I THINK WE HAVE TO CHANGE THE DATABASE NAME FROM newe2600404616 TO e4022600405903 AND THE TABLE_PREFIX FROM na_ TO wp_52ks29b2z8_
And when this is all over change all the prefixes and everything as I’ve exposed them here.
I forgot to hide the salt keys on the new site, but it doesn’t matter as it is temp (we’ll just overwrite/delete them anyways).
Current Production WordPress Site