Related Posts

Why care about email?

I got my first email address in the very late 80s. I was an intern at a government agency. I learned a lot there: how to sequence DNA, how to handle radioactive material, how to handle human pathogens, and how to send email. I got my first non-work non-school address in the mid-90s. One of the first things I did was join some mailing lists.
One of them was a list for folks who had pet rabbits. I met a lot of people there, both online and in person. As with many people we meet through a shared interest as our interest wanes the relationships change. Some relationships were maintained, but some of us lost touch with one another. Moves, job changes, email address changes, they all affect our ability to maintain relationships online. I kept in touch with some, one was the maid of honor at my wedding and a few years ago I was the maid of honor at hers. I lost track of others.
 

Read More

Network glitches and corrupted VMs

I had a bit of a interesting Friday. I was so glad it was finally the weekend. Saturday we did a bunch of errands, including go visit our servers. See, we’ve been upgrading infrastructure to implement a second type of backup system. Saturday we were doing the last set of upgrades so we could install over the weekend.
Yes, we do all our own networking and racking.
12974536_10206263292444901_7498678361263518784_n
Saturday evening Steve is installing the new backup software. This is awesome backup software. It backs up the entire virtual machine. If we lose a virtual machine, we can just reload the entire thing and it will be back again.
Except while installing the software, there is a weird network glitch. Said network glitch caused the system to crash. The system crashes hard. The system crash corrupts some of the data on disk. The data on disk is our virtual machine files. Files are in read only mode and won’t fsck automatically.
We lose most of our production virtual machines.  We’re off the air.
IronyBlog
Possibly this was tragic, not ironic. I dunno, it’s been a long weekend.
We lost a bunch of production virtual machines to the disc corruption. We haven’t lost any data, but it’s taking some time to rebuild the machines and pull data from the other backup system and get it installed.
That means some of our websites and services, like tools.wordtothewise.com are down. It may mean you saw some bounces if you sent us mail over the weekend. Mail is back and we are communicating with the outside world again.
Steve’s working through our other services as fast as possible to get them back up and running.
(If massive server issues weren’t enough, one of the cats got a UTI so we’re having to pill her twice a day. Then last night managed to puke so hard she passed out briefly. Poor thing. She’s doing better this morning.)

Read More

Friday blogging… or lack of it

It seems the last few Friday’s I’ve been lax on posting. Some of that is just by Friday I’m frantically trying to complete all my client deliverables before the weekend. The rest of it is by Friday I’m just tired. Today had the added complication of watching the Trumpcare debate and following how (and how soon) it would affect my company if it passed.
That’s been a bit distracting, along with the other stuff I posted about yesterday. I wish everyone a great weekend.

Read More