Archive for the ‘Big Data’ Category

Compiling Hadoop example

I’m working through some of the examples in this Hadoop book. I’m a little rusty on compiling java programs and had a little trouble with this one so I’m documenting it here for anyone else how might be having issues. Firstly, I tried compiling the examples like this; javac That wasn’t too successful; […]

Hadoop VersionInfo Issue on OpenSuSE 12

I was getting the following error when attempting to run hadoop version. The java class is not found: org.apache.hadoop.util.VersionInfo Unable to determine Hadoop version information. ‘hadoop version’ returned: The java class is not found: org.apache.hadoop.util.VersionInfo This was due to having the OpenJDK installed rather than the one from Sun/Oracle. To resolve this simply uninstall the […]

Preparing the NCDC Weather Data for Hadoop

I’m exploring Hadoop with the book Hadoop: The Definitive Guide. Appendix A shows how to download NCDC Weather data from S3 and put it into Hadoop. I didn’t want to download from S3 or load the entire dataset so here’s what I did instead. Here’s a little bash script I used to download the data. You […]

Getting started with Hadoop

I wanted to get started playing about with Hadoop but had trouble installing Cloudera’s CDH. As I only wanted to have a working version of Hadoop for development purposes I decided to skip using Cloudera’s distribution and go direct to the Apache Hadoop release. Here’s the process I went through to set it up on OpenSuSE […]

Tech plans for 2013

Just a quick post on my technical plans for 2013… SQL Server 2012 – I moved our primary SQL Server system across to a SQL Server 2008 R2 Cluster this year so I can’t see us upgrading for a while. That said it’ll be good to get ahead of the game and start experimenting with […]