Archive for the ‘Data’ Category

EFK: Free Alternative to Splunk Using Fluentd

Here is an updated version of the instructions given at Free Alternative to Splunk Using Fluentd. The installation was performed in CentOS 6.5. 1. Install ElasticSearch mkdir /opt/src cd /opt/src wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.2.1.noarch.rpm rpm -ivh elasticsearch-1.2.1.noarch.rpm /sbin/chkconfig –add elasticsearch service elasticsearch start # Move default file locations if required mkdir /data/elasticsearch mkdir /data/elasticsearch/data mkdir /data/elasticsearch/tmp mkdir /data/elasticsearch/logs […]

Get TFL Tube data with Powershell

The London Datastore has loads of datasets available that we can use for free. One of the datasets available is a list of TFL Station Locations. The station location feed is a geo-coded KML feed of most of London Underground, DLR and London Overground stations. Here’s Powershell script that will extract this data from a […]

British Government launches free data site

Just a quick post about the British Governments launch of data.gov.uk offering free access to a large number of datasets. It’s good to see this data, that we as taxpayers have funded, available for free use. Hats off to The Guardian’s long running free our data campaign. The site is being officially launched by Sir […]

Primary Keys do not always have to be ‘Id’!

We’ve all more than likely spotted tables in databases with no primary keys. But does a primary key always have to be defined something like… ?View Code TSQLALTER TABLE MyTable ADD Id INTEGER NOT NULL IDENTITY(1,1) PRIMARY KEY CLUSTERED; I think we have ORM to blame for this widespread practice of using auto-incrementing integers as […]

London Postcode District Data

The requirement to work with postcodes, and specifically London postcode districts, has followed me around during my professional life. Below is a TSQL script to create a table for the postal districts in London. Enjoy! ?View Code TSQLCREATE TABLE london_postcodes (postcode VARCHAR(4) NOT NULL PRIMARY KEY CLUSTERED) GO   SET NOCOUNT ON; SET XACT_ABORT ON; […]