Saturday, December 6, 2014

In the beginning - Conceiving CephPi

The idea for this project came from Brian Harvel sharing a link about a new type of hard drives sold by Seagate - Seagate Kinetic.  These drives implement an object store on board, but more importantly have an ethernet port so they can be accessed directly, truly detaching storage from compute.

At the time I knew nothing about distributed file systems.  Sure - I've worked with network storage.  From installing and configuring Samba on my Linux machines to SANs connected to VMWare clusters I have a passing familiarity with how network storage works.  But I have never really thought about the ins and outs of a truly distributed file system.

So I googled 'distributed file systems' and if I recall correctly landed here.  A cursory examination (also the first plausible one in the list) got me to look at Ceph.  This would be a good time for a DISCLAIMER:

I am still a complete dilettante.  I have not completed a full study of the field so any and all qualitative opinions I am about to share are incomplete, partial and subjective, and likely wrong.  The technical details, however, are correct.  Because science.

So lacking access to cool Seagate drives, and a bit disappointed by the fact they do not seem to expose block storage and mostly because I thought it would be cool, I decided to build my own.  The Thanksgiving holiday was quickly approaching as well, and for once I didn't actually have a lot of real take home work.  So motive, opportunity...  time to procure materials.

Having recently moved to Boston, and being of the geek persuasion, I love the fact that Microcenter is only a short drive away.  It was time for some retail therapy.  I compiled a list of all the things I needed to build a small distributed storage cluster with 3 Raspberry Pis, operating as storage cluster, each connected to SATA drive.  The system was to run Linux / Ceph, and thus the name CephPi emerged, because its shorter to type than RaspberryCeph.  And also I am not much of a berry guy... be continued.