The Hobbyist Computing Revolution: Take 2

Well here we are. The Post-PC era. At least that’s what we’re led to believe nowadays. I’ll admit…I’m only 21 years old, but already I’m beginning to miss the common act of picking out custom PC components, building out that bad-ass system, and configuring my software just right. I don’t think that part of me will ever die, but it’s quite evident that times are changing. Younger generations have definitely not had the same exposure or experience with computing. A lot of people are saddened by the fact that younger generations may never have this opportunity. That it’s a generation being handed an iPad without an explanation of how any of it works.

Although this era may be coming to a close, a whole different era is just now beginning. I think we’re right in the middle of the next wave of hobbyist computing. Thinking back to my younger years (again, I admit I’m still a youngling!), I remember putting together my own 486 and Pentium systems. Some of my earliest memories involve using a Compaq Presario running at 75MHz.

With the launch of the latest Arduino Due, it got me thinking. This guy that was just released is clocking in at a whopping 84MHz. Now hold on a second…you’re telling me that this tiny little chip that I can buy for $8 is faster than one of the first $1000 PCs I’ve used? And even better – everything about Arduino is open source. I would argue that before Arduino, hobbyists wishing to get into the microcontroller scene had a pretty high barrier to entry. Those that were very dedicated were able to read through the datasheets, hack on low level C/assembly code, and debug for hours to make their simple embedded apps function. Arduino changed all of that. Arduino put hardware in the hands of everyone and made it intuitive. This is exactly what the hobbyist (or MAKE) culture needed to flourish.

In the age of Apple-esque devices designed to never be disassembled by the mere mortal, this is the alternative. The alternative that puts hardware back in the hands of hackers. With the 32-bit Due released and the $25 Raspberry Pi out in the wild, obtaining hardware is becoming more accessible every single day. We’ve already seen real time operating systems ported to Arduino and it’s only a matter of time until real full blown operating systems are running on the Arduino platform. Now of course, SoCs are essentially just the components of a traditional computer shrunk down and contained within a single package, but the software is what makes the hardware magical. These SoCs are quickly becoming the next generation platform for computing. Destroy a chip while developing? No big deal, just drop another chip on your board and carry on. Hardware is becoming a commodity.

Now it may just be the part of me thinking back to the magic of computing that excites me so much. The part that remembers so fondly the constant CPU upgrades, memory upgrades, and expansion cards swapped out. That magic feels like it’s returning again, though. The magic where every clock cycle counts, every KB of memory means that much more, and the added capability of each PCI card. But it’s happening again. And the best part? Tinkerers and hackers are in the driver seat this time around. We decide what hardware is included in our devices. We write the software; the drivers. Weare completely in control of the device. It’s incredible to see on such a large scale. I can’t wait to see these little chips pave the way for the 2.0 hobbyist PC revolution. One that I’m already a part of.

My Reminder

This is a reminder. A reminder for myself to always come back to when I’m facing a difficult decision about what I want to do with this life. Time is short – none of us are long for this world. The time we are given is just but a tiny sliver of all human history – past, present, and future. The key is to make the most of this time. Of course making the most of time is both a relative and subjective goal.

One thing I feel I have learned during these 21 years of life is that we all have at least one or many unique/natural abilities. Not all of us have discovered what that is yet, but I believe most people are searching. All of these abilities are important – never forget this. Whether that ability is designing 3D models in CAD, growing a garden, raising a child, or just making others laugh. Have you ever stopped to think that even just your presence as a person can make such a positive impact on your friends or others?

I believe myself to be a “maker” at heart. I love software, hardware, and the combination of both that produce meaningful outcomes. I’ve launched my fair share of websites, built “cool” gadgets, and even tried to launch a new social media app. That last one was a failure by the way. A failure that has had such a major and positive impact on my life.

I love the idea of failure. Knowing how to lose is crucial. Although I never became particularly skilled at skateboarding, I believe it to be a great analogy for failure. One of the first things I was taught when learning was how to fall. You learn this because you will fall A LOT. You can’t expect to get on a skateboard and land each trick the first try you attempt it. Only through the process of understanding exactly what you’re doing wrong can you correct yourself. As with just about any skill I’ve attempted to pick up, I find the same exact concept applies.

Another area I’ve been thinking a lot about as well has been focus. I believe my generation suffers greatly from a lack of this. It definitely shows through in all areas of my life, whether that be writing music or developing software. It’s all about being able to deliver. To see something through to the end. Even if that something is rough around the edges, finished is ALWAYS better than perfect. Of course you strive for the best that can be achieved, but getting so caught up in the details that the end goal becomes out of sight is useless. To me, simple completed projects are immensely more impressive than complex half completed projects.

Back to this failed social media app. I teamed up with a few other buddies I met in college to try to build a next generation chat/link sharing website. At the time it felt like the most important thing in my life. 8-10 hours of coding a day and the project never left my mind. My buddies and I ended up in silicon valley for a week to work with another startup company. While out there I had a major realization; so many smart people are pouring so much energy into projects that were more or less incremental improvements on social media sites, music playback, and mobile games. All very cool stuff, for sure. But is this really what we need to be focusing on as a society? It began to hit me that I was no different than all of the others just trying to build something cool without the thought of how it fits into the grand scheme of things. We ended up canning the project shortly after this trip; mostly due to a lack of being able to define the real problem we were solving.

I’m down to my last several months of college and time is quickly running out before my structured life comes to an end. It’s required me to think a lot about what I want to do with the rest of my life. I keep coming back to the same conclusion: I want to change this world for the better. There’s that tricky word again: better. Tying this back in with the unique ability thoughts from above, my answer to this is to link my personal skill set with what I believe are some fundamental issues that we face as a society. Just looking at my notepad, I’ve written down energy production, crime, food supply, and space colonization. I’m sure my skills and this list with both change with time, but it’s important to think about both of these things. To think about what really matters and how to be most effective at solving those things.

Google Drive: Still not ready for Prime Time

Let’s sync some files

In my quest to find the perfect cloud syncing and storage service, I’ve been trying out all the big names (Dropbox, Google Drive, Skydrive). My current setup relies on storing all my data in my Dropbox folder and then backing this directory up locally.

Most recently I decided to give Google Drive a spin (caveat: I’ve only used it on OS X so far). For many, one of the most appealing reasons to use Google Drive is the pricing. $5/mo for 100GB and $10/mo for 200GB. For myself, I already use a lot of Google services so it made perfect sense to use Drive.

Slow Client

One of the major issues that I ran into right away was the speed at which Google Drive runs. Just indexing my home folder which contains approximately 20,000 files took several minutes. Even after the initial indexing, relaunching Google Drive still took several minutes to run through my directory. For comparison, subsequent Dropbox launches typically take no more than a few seconds to “catch up”.

Slow Uploads, Pegged CPU, Crashes

During the upload process I found that Google Drive was significantly slower than Dropbox uploading. The client had a tendency to crash every hour or two while uploading my files and my CPU fan seemed to be running at full tilt. For whatever reason, Google Drive is extremely resource intensive. Both SkyDrive and Dropbox upload for hours without my machine breaking a sweat.

Git Weirdness

As a developer, I have plenty of Git repos. Dropbox and SkyDrive handle these repos without any issues. With Google Drive, I found my repos becoming corrupted, files appearing to be modified when they weren’t, and even occasional ”could not sync” errors for files in the .git directory.

No Access For You!

While in the middle of using the Google Drive web interface, I began encountering 403 Forbidden errors. At one point the web interface was inaccessible for up to 30 minutes. I realize this may have been some random fluke when I was attempting to use the service, but it doesn’t look good when none of your files are accessible from the web.

File Deletion

A bit of an unexpected behavior for this one. With services such as Dropbox, file deletion is compared based on the last modification date. I had an outdated folder “Documents” on my Google Drive. I first quit the Google Drive client, deleted this folder from the web interface, copied my up to date “Documents” folder back into my Google Drive folder, and then fired up the client on my machine.  Dropbox’s behavior would be to sync the new “Documents” folder back into the system. Google Drive simply removed the new “Documents” folder from my machine.

Oh you want to leave?

After encountering all of these issues, I decided Google Drive wasn’t for me. I quickly removed all of the files that I had added to Google Drive and emptied my trash from the web interface. I then attempted to downgrade my storage back to the free 5GB plan so I was no longer paying for my extra storage. To my surprise, I still had 20GB+ of data storage used up by Drive. Searching around online, I found that even after removing files that they will still appear in the “All items” category. There is simply no other way to remove these files but by selecting them 500 at a time and deleting them. With 20,000+ files this took a considerable amount of time. Finally after doing this I could empty my trash again and downgrade my account.

Dropbox, you’re all mine

As much as I want Google Drive to be a great alternative to Dropbox, it’s still far from the polish that makes Dropbox so good. I’m always willing to give services another shot, but for now I’ll be sticking with Dropbox. Oh and did I mention that Dropbox just doubled all of their storage plans for free? $10/mo for 100GB of storage makes it that much more appealing.

Building the open source distributed cloud

I’ve been thinking a lot about cloud computing and storage lately. Moving to the cloud scares me. But at the same time, the convenience of having my files everywhere and always in sync is too good to pass up. Although I’ve been using a multitude of different cloud services, I’m still searching for that perfect solution where I still have control of my data.

I recently received my Raspberry Pi in the mail, which I was really excited for. I stayed up hours on end the night of preorder availability, bashing my F5 key in hopes of getting a hold of one. Yes, I eventually got my preorder in, but not after pulling close to an all-nighter and feeling like a zombie the next day. Worth it.

Of course one of the first things a lot of people do (including myself) is load up some quake 3 or connect the Pi to a TV and play around with it as a media center. Cool stuff for sure, but I kept thinking there has to be more I can get out of this thing. Just the fact that you have a full *nix based system that can fit in the palm of your hand for $25 could (and has proven to) be earth shattering.

Now of course most geeks and sysadmins will tell you they’ve been running their own servers in their homes for ages. They’re awesome for mass file storage, media transcoding/streaming, and backups. Of course there are also the downsides. You’ve dedicated a whole PC to a single task, that PC is sucking down power 24 hours a day, and you need a good place to store it. Now that devices such as the Raspberry Pi exist, everything has changed. We now have a low power, silent, and small PC that can easily handle the task of acting as a home server. Hmm…maybe we can use this to our advantage to solve the cloud issue.

So what am I proposing? An open source distributed cloud. A cloud that everyone contributes storage space to, that everyone can store data on, and that keeps data private unless explicitly shared. Sounds like a tall order, huh? I totally agree, but I think it may be feasible.

A few projects have already attempted to tackle this problem. One of the most prevalent is/was theOFF System. Although originally designed to combat copyright law, the same concept could be used for building a shared cloud. OFF works by breaking out files into small chunks, encrypting them, and storing these chunks between multiple nodes. This “encryption” essentially builds chunks from other chunks, allowing chunks to be shared between different files. No one owns these chunks but a retrieval URL acts as the “recipe” for combining the seemingly random blocks back into the original file.

Unfortunately development has come to an end and the network is severely lacking the number of nodes required to make this much more than an experiment. Because the intent of the network was essentially to skirt around copyright, this may have doomed the network from the start. Repurposed for something like a distributed storage cloud, OFF really could have taken off.

So how does the Raspberry Pi fit in? They’re cheap, they’re low power, and most owners already using them as servers/media centers keep them on 24 hours a day. Perfect.

There is still an issue, though. We aren’t quite there with storage yet. With the Raspberry Pi, everything is currently based around SD cards. Sure, you can connect USB drives but now we’ve just made that perfect server a bulkier package. In the next few years we should see PCs like the Raspberry Pi ship with an SSD, allowing for mass data storage out of the box. This is going to be important for this open source cloud idea to succeed.

If we could build something like this, we truly would have a massive open cloud storage system. Backups would be a thing of the past. All data would be replicated across several nodes and you could even control how redundant your data is. I’m not sure about the technical implementation, but there would need to be some sort of limits on the system. Something along the lines of only allowing users to store as much data as the storage space they have contributed.

Another problem this could potentially solve – bandwidth. When you host all of your data from your single Cable/DSL connection, it can take forever to grab files or stream some media from outside your network. With a distributed storage system like this, you’d be able to pull your media from multiple sources at a time. Technically, I’m not sure how well this would scale. Perhaps it wouldn’t scale at all. I definitely think it’s worth looking into though.

Imagine if the official Raspberry Pi image eventually shipped with something like this. By default each Pi that is brought online could contribute to the network and continue to grow the storage space available. Just as the web has come to embrace open standards, we need open data. Storing data locally has become a chore. Problems like retrieving data, out of sync files, and running backups should be a thing of the past.

I’d love to keep this an open dialogue. Leave a comment and let me know your thoughts! If you’re a dev, would you be interested in working on a project like this?

Well this is it

Digging deep within the abyss that is my hard drive, I stumbled upon a file named “Blog.txt” earlier this week. A file I had completely forgotten. 40 pages of 11 pt text that essentially summarizes my life from 2003-2008. Migrated across multiple versions of my very first HTML site I built and eventually gave up on. Scrolling all the way to the bottom I see my very first entry.

“Today I started my blog. I’m sure a lot of you have no idea what a blog is so I’ll tell you now. A blog is a place where I post what is going on with me and my server and stuff like that. I might tell you what I ate at different times of the days, If my server is all messed up and I have no idea what I’m gonna do, and whatever I may have on my mind. Well this is it.“

Dated June 13th, 2003 – I was 12 years old at the time. It hit me as I was reading through the various entries just how much I’ve changed these past 9 years. At the same time it made me really miss writing for fun. Since getting into college it seems like all of my time is spent shooting off “buisnessy” emails, writing papers about topics I’m not particularly interested in, and trying to finish my thesis.

I think it’s time to get back into creative writing. Writing about the things I’m passionate about.

Well this is it.