The weekend project has me downloading a bunch of weather forecast data and working on some processes for analyzing it. I’ve been trying to not EVER delete any raw data because I never know I’ll need to redo something I screwed up.
This habit has saved my can a few times.
Somewhat recently, I learned SQLITE so I could stop storing everything on the hard drive in flat files, which is probably ridiculously inneficient for the computer and is definitely a pain in the ass to monitor.
But here’s the thing with SQL, it’s a tricky and brutally unforgiving language. You can ACTUALLY delete things by mistake and never see them again. And last week that’s what happened to ALL of my backup data. I still don’t know what on earth I did.
Luckily it was only the backup data (still had the main database) and even more luckily I backed up the backup a week earlier when I left town on a trip.
In the end, I lost a few days worth of backup info forever. The upshot is that I quickly moved backup management to the top of the to-do list. The thought of losing all this work gives me the willies. I’d probably have a nervous breakdown.
So, here’s my ‘backup plan’:
I recently bought this nifty router that can host an external hard drive on an ftp server. I just plug it in and presto, it’s accessible online from anywhere. Perfect.
But netowrk programming is finicky. Connections interrupt for reasons I cannot fathom, which means that the old drag and drop method fails when the interweb gods frown upon me.
So I spent all day today figuring out a way to cycle through my directory and compare all the files to the existing backups on the ftp site and keep the new ones. If the process gets interrupted it just waits until I get it back. No restarts.
Cycling through the installation-related junk slows me down, but brtue-force backupping is good for peace of mind. I know it’s all going to be there.
This is the latest in a long series of programming detours I’ve had to make. It’s amazing how little time I spend doing the real analysis that is the ENTIRE POINT of this project. If I knew how hard this whole thing was going to be, I’d have been a bit less cavalier in committing myself to it.
Two months and I’m not even remotely close to a working prototype.
My target is the last weekend in July, now. We shall see.
Not Unreasonable 