I purchased a pair of Raspberry Pis (Model B) from ModMyPi to use for my data gathering. My doctoral work requires data from bike-share systems such as Velib’ in Paris. I will be scraping many sites repeatedly for the next couple of years. I wanted a distributed system that could resist theft/fire, hard-drive failure and internet service interruptions while still being affordable. Going with my current host would not have been possible due to my long term capacity requirements of about 40GB and the costs associated with those requirements. I considered using Amazon’s Web Service but don’t want to lose control of my data and I’m not sure the cost is worthwhile compared to using a couple Raspberry Pis (RPi). I wanted to share my process because most existing tutorials ask you to connect your RPi to a monitor/TV for setup. That’s no longer required since SSH is enabled by default. I also desired documenting my steps so that I can exactly replicated my setup for my second RPi.
Setting up a development environment locally can save a lot of time, but the setup can be quite painful. I needed to modify a few things to getting my website running in OS X Snow Leopard. Setting up apache is wonderfully easy with OS X but further customization took more time. I wanted to get the .htaccess files and apache rewrite module functioning functioning locally.