I had a dream last night that the apartment beneath ours caught on fire, we had to rush out of the building, and my computer and all of its data was destroyed.

I’ve been pondering a formal backup system for a while now.  (My current system involves making sure important files are in a version control system and exist on at least my laptop and desktop.  This is pretty ad-hoc, inconsistently updated, and not entirely comprehensive.)  I’m taking my dream as impetus to actually set something up.  This post is to help me organize my thoughts and see if anyone has any comments or suggestions.

§ My Requirements

I want to have a full rotating versioned backup system, where I have complete daily backups for a recent time span (say a week or so) and more sporadic backups back to as much as a year in the past.  Ideally, the backups should be stored in a space-efficient manner so unchanged files don’t take up more space than a single copy would require.  The backups should have off-site redundancy.  They should be relatively easy to use; they should be fully automated on a day-to-day basis, with notification when things go wrong.  Ease of setup would be nice but not necessary.

§ My Data

I currently have about 720 GB of data in my home directory, plus a few hundred MB elsewhere on the computer that I’d want to back up.  I also have about 11GB in a bzr repository, but all of that should remain duplicated in my home directory.  Most of the data in my home directory is in media files that I can either replace (rerip CDs, etc.) or live without; only 25 GB of it is stuff that I must back up.  (A further 130 GB is stuff that would be nice to back up, but I can just burn it to DVD and consider those my backups; the data is essentially static.)

§ JWZ Backups

The easiest approach is the JWZ backup solution.  For all of my data, that would be two 1 TB external hard drives, for about $220.  If I restrict myself to the “must backup” data, I could make do with two 60 GB external hard drives for about $80.  In either case, I’d keep one drive at the office and swap them periodically.

The advantage of this approach is that I control everything.  I can put encrypted volumes on the drives, so if they get lost or stolen, my data isn’t usable to other people.  I can use rsync with hardlinks between datestamped directories to get versioned backups with efficient disk usage.  The drawbacks are a modest initial monetary outlay and the need to coordinate shuttling drives back and forth.

§ Amazon S3

Another approach is to use Amazon S3 to store my data.  It’s offsite by definition (and stored among multiple data centers; if I write data to it, I can reasonably trust that I’ll get that data back).  It’s not too expensive: at $0.17/GB-month, my minimal backup will cost about $3.85/month.  Throw in transfer costs and churn, and I doubt I’d exceed $6/month.  (The initial upload would be $2.56.  A full restore would cost me $4.36.)  With S3, I would only back up the minimal data; the 130 GB of optional backups would cost an additional $20/month, which would exceed the cost of the full do-it-myself hard drive backups in one year.

The complication to S3 is that it’s just a web-based data storage service; you need additional software to make a reasonable backup solution.

# Jungle Disk

From everything I’ve read, Jungle Disk is currently the best software for storing filesystem data on S3.  It runs on Windows, Mac OSX, and Linux, and exports your S3 buckets as a WebDAV disk, which you can then mount and treat like an ordinary (unlimited capacity) disk drive.  All data is encrypted before it’s sent to S3.

I like this approach.  Since it looks like a disk, I can use the same rsync setup I would with my own disks, and since the data is encrypted, I don’t need to worry too much about it being transported over the Internet and stored on someone else’s servers.  The main drawback is that it’s proprietary software.  In addition to my principled preference of open source software to proprietary, there’s also the issue that, especially because the data’s encrypted, this software would be my only access to my backups.  If something went wrong and I couldn’t get support from the company (e.g. they went out of business), I’d be out of luck.

The software costs $20.  Assuming $5/month on S3, it would take one year for this approach to cost more than the minimal get-my-own-disks approach.

# Other S3 software

I haven’t seen anything else that will let me back up to S3 and keep versioned backups in a space-efficient manner.  Most of the S3 backup software I’ve seen doesn’t do versions, and the few that do don’t appear to do it space-efficiently.  As always, I have the option of writing my own, but that would take a fair amount of time and effort, and I’d be likely to give up partway through, continuing to leave myself without good backups.

§ Conclusion

Barring any better suggestions from others, I’m leaning towards the two smallish hard drives.  They’d pay for themselves after a year of use, and I get complete control of my data (for better or worse).  I like the idea of using S3, but it’s more expensive in the long run, and I’m not completely happy with any of the software I’ve found to use with it.

I made this for Thanksgiving this year, and have already been asked for the recipe, even though I haven’t had any yet.  So here goes.

• Brownie Layer
  • 1 cup sugar
  • 1 cup butter
  • 4 Tablespoons water
  • 2 cups semi-sweet chocolate morsels (~300g)
  • 1/2 cup amaretto
  • 2 teaspoons vanilla extract
  • 4 eggs
  • 1 1/2 cups all-purpose flour (~190g)
  • 1/2 teaspoon baking soda
  • 1/2 teaspoon salt
  • 1 cup chopped or sliced almonds (optionally toasted)
  • 1/2 cup chopped maraschino cherries
• Truffle Layer
  • 8 oz cream cheese, softened
  • 1/4 cup powdered sugar (30g)
  • 1 cup semi-sweet chocolate morsels (~150g)
  • 2-3 Tablespoons amaretto
• Topping
  • 1 cup semi-sweet chocolate morsels (~150g)
  • 1/2 cup whipping cream
  • 1 cup sliced almonds, lightly toasted
  • maraschino cherries for garnish

§ Brownie Layer

Preheat oven to 325°F.  Prepare a 9×13 baking dish.  (I line it with a sling of parchment paper and then spray it with Baker’s Joy.)

In a saucepan, bring the sugar, butter, and water to a boil.  Remove from heat.  Add chocolate, amaretto, and vanilla extract, stirring until chocolate is melted.  Add eggs, one at a time, stirring until blended.

Whisk together flour, baking soda, and salt.  Add to chocolate mixture, stirring well.  Stir in almonds and cherries.

Pour mixture into baking dish and bake for 42-48 minutes.

Leave them in the dish to cool.

§ Truffle Layer

While the brownies are cooling, beat the cream cheese and powdered sugar in a stand mixer on medium speed until the mixture is smooth.  Melt chocolate and add with the amaretto to the cream cheese, mixing until well-blended.

Spread over brownies and refrigerate until firm, at least 1 hour.

§ Topping

In a saucepan, melt the chocolate in the whipping cream.  Spread evenly over the brownies.  Sprinkle with almonds and cherries.  Refrigerate until set, at least 1 hour.

Cut into bars and serve.

I have simple needs.  I have a base class with some generic behavior and subclasses with specific information for that generic behavior.  More concretely, the subclasses need to provide the generic behavior with an ordered list of things that designate key fields on database tables.  The best representation of those “things” in Delphi seems to be members of an enumeration:

type
  TKeyField = (kfFoo, kfBar, kfBaz, kfQuux);

Since I need the list of fields to be ordered, I need them in an array:

type
  TKeyFieldArray = array of TKeyField;

The declaration of the base class is pretty simple:

type
  TBaseClass = class
   protected
    function GetKeyFieldList : TKeyFieldArray; virtual; abstract;
   public
    procedure DoSomethingWithKeyFields;
  end;

As is the declaration of the subclass:

type
  TSubClass = class(TBaseClass)
   protected
    function GetKeyFieldList : TKeyFieldArray; override;
  end;

So where’s the problem?  Where’s the hate?  The hate is in the implementation.  If Delphi had array literals, this would be easy.  Something like:

function TSubClass.GetKeyFieldList : TKeyFieldArray;
begin
  Result := [kfBar, kfFoo, kfQuux];
end;

But it doesn’t.  It has some special magic for array literals if they’re the parameter to a function, but not anywhere else.  It does, however, have a syntax for array constants.  Perhaps this will work:

function TSubClass.GetKeyFieldList : TKeyFieldArray;
  const
    keyFieldList : TKeyFieldArray = (kfBar, kfFoo, kfQuux);
begin
  Result := keyFieldList;
end;

But no.  That TKeyFieldArray is a dynamic array; Delphi doesn’t allocate any space for it, so it can’t be a constant value.  You have to tell Delphi how big each constant array is, even though you’re already telling it how many elements are in the array.  So perhaps this is the solution:

function TSubClass.GetKeyFieldList : TKeyFieldArray;
  const
    keyFieldList : array[0..2] of TKeyField = (kfBar, kfFoo, kfQuux);
begin
  Result := keyFieldList;
end;

But no.  Because of Delphi’s approach to static typing, those are actually different types, and are therefore not assignment-compatible.  (See previous hates on this subject.)  No, here is the code that Delphi makes me type for what should be a one-line function implementation:

function TSubClass.GetKeyFieldList : TKeyFieldArray;
begin
  SetLength(Result, 3);
  Result[0] := kfBar;
  Result[1] := kfFoo;
  Result[2] := kfQuux;
end;

And just earlier this morning I was pleased because I read that Delphi 2007 (to which I’ll soon be upgrading from Delphi 5) has for...in loops, so I can finally have foreach.  (Can’t get the generics and anonymous functions in Delphi 2009, because we need .NET and that’s not yet available for Delphi 2009.)  Oh, Delphi.  The one hand giveth, and the entire rest of the stupid, anemic, pox-ridden language taketh away.

I’m divesting myself of a lot of computer-related things that are taking up too much space in my apartment.  Take a look at my stuff and let me know if there’s anything you’d like to take off of my hands.  Anything left after a couple of weeks will go to the electronics recycling center.

On September 27th, 2008, I got married.  As a consequence of my marriage, I gave up my last name and took my wife’s, going from Phillip Gregory to Phillip Gold.  I’ve been asked about my decision a lot; this is my explanation.  [Note that my name is now Piper Gold; the names in this post are what I was using at the time it was written.]

I feel that the prevaling societal standard—the assumption that the woman must go through all the work to change her name and give up the identity she’s had since birth—is unfair and an example of gender inequality.  Rather than simply make that assumption, Rebecca and I discussed our names a lot before the wedding, starting with what we each wanted out of our married names, and working from there to a mutual decision.

I wanted us to both have the same name, as symbolic of our marriage.  I also didn’t want a hyphenated last name, because I feel that those are cumbersome and unwieldy.  Rebecca also wanted to have a Jewish last name, to honor her cultural heritage.  Finally, I was inclined to have a name that started with “G” so my (and her) initials would stay the same.

Our first thought was that we would find a new name that met all of our criteria and both change to that name.  Unfortunately, there are only really two common Jewish surnames that start with “G”: Gold and Green (plus all the variations thereof), and we couldn’t find a variation on Green that we both liked.  We started looking at other Jewish surnames, and I realized that I really did want to keep my initials, if only because I have the username “phil_g” on a lot of sites, not least of which is the email address I’ve had for over a decade now.

So I offered to just take Rebecca’s name because that approach accomplished everything we wanted.  She was a little hesitant, feeling that doing so would require more of me than her, but we eventually agreed that it seemed the best approach given our requirements.

Postscript: Some people suggested changing my middle name to my old last name, as some married women do.  I opted against that approach, because my middle name is the same as my dad’s.  I don’t really like the idea of giving my child the same first name as myself, but I like the subtle continuity of shared middle names.