Goodbye, reader

Since September 10, 2008 you have read a total of 80,236 items.

Goodbye, reader. We had some fun times, you and I, but we both knew it had to end eventually. Still, we had a good run; 4.5 years is practically a lifetime in the internet world.

I’m taking my subscriptions out of the cloud, back to something self-hosted. Frustration with the quality of desktop RSS feed readers was what originally drove me to Reader. Hopefully the rest of the RSS world has caught up in the last five years, and I will not lose my ability to use multiple devices or have reasonable performance.

Interview with Eben Moglen at Re:Publica

Excellent interview with Eben Moglen about the problems of centralization on the internet – probably the best explanation I’ve seen – and various other topics, including WikiLeaks and copyright reform.

Help! We’re drowning in data

The age we live in is the first with an overabundance of data. It’s had a tremendous influence on our daily lives.

Cheap electronics and digital processing have made it possible to gather and organize all data imaginable. All online stores, and most regular ones have easy access to their transactions in digital form. Video cameras keep an eye on dangerous corners and other interesting or boring views. The integrated GPS, GSM and Wi-Fi in phones is used to pinpoint the location of its user. (Radio)telescopes capture vast amounts of images. And, of course, web browsers and web servers are aware of the web pages we visit.

In addition, the extremely low cost of digital storage, compared to the technology available to previous generations, has made it possible to record this information and keep most of it around for a long time. At least for as long as it could possibly be useful.

Gathering digital information usually is not hard, nor expensive. There is often barely any effort involved at all, because the data is already generated or digitized as part of some other process – it just needs to be stored. Storing large amounts of data is not trivial, but also not unreasonably hard. This is also one of the reasons why most internet giants are reluctant to throw away data; it can come in handy later, but once it’s gone it’s completely gone or at least hard to regain.

The real challenge is in organizing and accessing the information in a useful and simple manner. This is a hard problem. Not only does the information need to be available, it needs to be available in a structured manner. Having millions of petabytes of raw data at your disposal isn’t very useful without knowing where to look or how to interpret what you find. Likewise, it isn’t possible to iterate over all available data for every single query. If you’ve used the web in its early days you will remember that search engines only provided a very simple subtext match over all pages everywhere and returned results in no particular (useful) order. They were some help in navigating your way through the web, but there was still a skill set (and a lot of patience) required to actually finding what you were looking for. This challenge – making all information everywhere available in a usable manner, and fast – seems to be what Google is all about.

Having all of this data available, and the ability to access it in a meaningful way, is absolutely awesome.  It’s an information engineers’ wet dream. It helps eliminate information asymmetry. It’s what allows my phone to know where I am without having GPS enabled (it uses the location of the nearby WiFi access points instead). It’s what makes it possible for me to find the full details and contents of any book in mere seconds. It’s what makes it possible for last.fm to predict the concerts I’m interested in in my direct vicinity.

It’s also pretty fucking scary, considering the effect omni-accessible information has on our privacy.

Perhaps the current use of the centrally available data by the state is reasonable, but there is little or no monitoring of how this data is used. The mere availability enables easy widespread abuse by any (future) totalitarian state.

My circadian rhythm, as seen by Google

For the last 5 years I have consciously had the “track search requests” feature in Google enabled when I was logged in. I use my Google account for all search requests except for those I don’t want tracked. Apart from some interesting statistics on the search terms I have most often used during the last 5 years (my own name is among them – yes, I’m that vain), Google’s ad profile is very interesting. From my searches, it has correctly deduced most of my various hobbies and other interests, down to specific music styles. It still lacks a few, but those it did find are all correct.

Interestingly, the data (roughly 27k requests) over the last 6 years also does a very good job of showing my circadian rhythm when graphed:

Combined with the location data from my Android phone, Google.com logins and Youtube views, this gives Google a pretty clear idea of what I’ve been up to. I’m considering clearing it all before the big privacy policy change in March.