So this tweet from Dominic Campbell and the article it links to felt timely, and worth flagging.

“A Case Study In How #Infographics Can Bend The Truth” fastcodesign.com/node/1669222 via @FastCoDesign #dataviz

— Dominic Campbell (@dominiccampbell) March 8, 2012

There’s nothing revolutionary in the way that Kestrel’s being used here, from the post it seems to be a fairly standard use case for it, but it’s a lovely example of detailing the day-to-day work of growing a web app that blogs are so good for. And it’s particularly good to read about the stages they went through as they tested each component in turn, in production, before switching over to them completely.

It’s a strange release for me—it was the first of GDS’ launches that I didn’t press the button for, or even attend as I’m currently out on paternity leave—but I’m really delighted to see it out there for the world to feed back on. Since I first heard about the vision for the single domain I’ve been excited about the possibility that government information could be published in a way that allows it to be sliced along axes other than ‘department’ and “INSIDE GOVERNMENT” begins to give some life to that.

It’s also a really interesting counter to the citizen end of our emerging publishing platform. Where the citizen content is conceptually very flat with very few formal connections, the content in the corporate system is very much a series of nodes interconnected in numerous ways. Free Range’s work on the app, guided by Neil and co’s careful analysis, is especially fascinating for the way it lays out an encoded domain model for such a notoriously complex world.

This commit is a good example of the level of thought going into the process:

“First Secretary of State” isn’t really a role, so we can’t use its name directly to order.

Ideally we’d model which minister was also the “First Secretary”, but that introduces more problems. If we’d made “First Secretary of State” a ministerial role, William Hague would appear twice in the list of cabinet members. If we’d made it a non-ministerial role, it wouldn’t have been sortable in the set that MinisterialRole.cabinet returned.

This is a quick, pragmatic solution, but more thought probably needs to go into whether or not the lists of ministers are lists of roles, or should instead be lists of people (along with the roles they currently have).

My only involvement in the code to date is a fledgling branch that started to map out what our API for this side of gov.uk might look like. As it worked out there wasn’t time to do that justice and ship the website so we left the API to return to later (Paul Battley’s working on part of that now, and taking a slightly different tack). My time on it was enough to really appreciate the amount of insight captured in the code.

It’s also been a delight to have Free Range in leading the development of the ‘whitehall’ app that drives “INSIDE GOVERNMENT”. GDS is committed to building a world-class team of in-house developers, designers, and the like, but we’re not going to achieve the transformation we’re aiming for on our own. So we need talented outside teams, small companies, freelancers, and the like to help us. This release is our first evidence of just how well that can work.

The talk is an exploration of what we might do if flickr disappeared tomorrow; it’s a topic many of us have been pondering at least since the news broke of yahoo’s decision to “sunset” delicious. Two elements of the talk really grabbed my attention.

The first was the detailed exploration of how difficult it is to back up or transfer the “social” aspect of a social website. Copying my files and some related metadata is relatively easy. Preserving experiences and relationships is a lot harder:

“Privacy is genuinely important no matter what people are passing off as industry best practices. It is doubly important for anything that archives Flickr because a respect for privacy remains core to what the site is about and the ways that people use it.”

“This is the actual hard part of the personal archiving problem: How to deal with authentication and authorization controls defined by a third-party site that may or not exist anymore.”

“This problem is also why parallel-flickr is not the mythical archiving of all of Flickr. Because you can’t back up Flickr. Or rather: The only way to back up Flickr with any kind of credibility or ethics is to swallow the thing, whole.”

But in the middle of the piece there’s also a little something about a past event to backup flickr which involved building out complex metadata and wrapping them in lots of “standards”: “all the best practices around XML, the Semantic Web and static, linkable resources”.

It is incomprehensible gibberish.

Worse, it’s hard to do anything with. Not only are the data models overly-complex but all the stricter-than-strict, standards-compliant tools that grew up around them are hard to use.

This is a really important point, especially if we’re going to talk about personal archiving.

If I can’t stand to look at this stuff seven years later then what hope is there that someone who does not live and breathe the technical details will?

For those of us living in a world where there’s a lot of excitement around ideas of “linked data” and associated standards, experiences like this one are very important to hear.

It’s been about ten days and it feels a lot longer, but recently we unveiled a rather important beta: GOV.UK. That beta is a “live operational test” of a new single-domain for government. It’s a radically simplified way for people needing UK government information and services, built in-house with a set of publishing tools that lay the groundwork for a broader platform.

This beta came out of the work a team of us did to build alpha.gov.uk, itself a deeply unusual creation for a government website: built by an in house team, ruthless in scope and relentless in user focus, and above all a prototype designed to trigger conversations. The alpha worked: it triggered good, constructive conversations, it helped us identify things that worked and others that didn’t. It paved the way for the creation of the Government Digital Service and to the beta of GOV.UK.

Around the release I’ve been writing quite a bit on the GovUK blog: an explanation of our hosting choices (AWS/EC2), there’s a colophon to list our key tools, a high level overview of how we’re using puppet and provisioning servers, a status update on the APIs we’re building. I also did a little interview for wired.co.uk, answered a few questions for O’Reilly Radar and found myself on stage (at the end of the day, having only had 3 hours’ sleep) at monkigras.

Which is really all to say that there’s not a huge amount more to say here other than to signpost all that content and to say “watch this space” cause we’re far from done!

Oh, and we’re hiring (not just ruby devs)

So, you saw all the gov.uk @govuk news this week, exciting huh! Why not come and join the team? Ruby devs, YOUR COUNTRY NEEDS YOU. Email me.

— Thayer Prime (@Thayer) February 3, 2012

What I’d not explored for a while was the simplest way to implement app-to-app oauth where there’s no UI for user interaction so over the New Year break I pulled something together for another project. It’s all pretty straightforward but not very well documented so I thought I’d better share.

The easiest thing to do if you want to allow an oauth client to work with your app is just to generate the ID, secret and access token for whoever’s responsible for the app and to provide them (securely) for direct use.

In order to do that in the rails app I was focussed on I knocked up a class to help me with that when using the aforementioned oauth2-provider:

and then a few rake tasks for interacting with it:

In the oauth-provider world, any “authorization” can be owned by a resource, which is any other model in your app. In a standard app like our SSO solution that’ll probably be a user, but in the app I’m working on here it’s an organisation that may have many users. You get access to that resource in your controllers with, eg:

And with that I had my API protected using everyone’s favourite standard authentication protocol.

The section on “API-centric architectures” particularly chimes with our approach and the idea of an “outside-in model” resembles what I was getting at in “Building APIs, building on APIs“. I quite like the use of the phrase “outside-in”, and the iterative approach implied in:

In an outside-in model, one would start with easy consumption (read REST) of perhaps “chatty” APIs and then improve upon them. This is in contrast to thinking performance first and ease of use second.

As with anything user-centered this approach does require some sense of who those users are, and as ever that’s going to be the biggest challenge in most cases. To follow through, organisations are going to need to be proactive in understanding the value for others in our systems and try even harder to approach them as outsiders might.

All three are exciting changes, but it’s the last that I sat down to write about here. For several years I’ve been working on building Ket Lai, initially alone, then with James Weiner, and gradually with a wider selection of collaborators. Things were going well, and we were building up a solid base of clients and getting close to having a couple of products of our own to release. But when the call to the public sector came, it didn’t take long to agree that we’d put Ket Lai on the back-burner and jump on board.

It’s a really exciting time to be joining, becoming part of the new Government Digital Service team and working (as tech lead) on the new Single Domain beta. We’ve had some great new people sign up to join us over the past few weeks, building a team I’m really enjoying working with.

For those who want a little more detail I’ve written a couple of pieces for the GDS blog: one about our approach to APIs and another about our platform choices.

I’ve not slept much since then, but so far everything seems to be running smoothly. Apart from my email and twitter clients which are swimming in a deluge of feedback.

There’s a quick post from me on the Alphagov blog exploring the way we’re handling geographic information and place names. A longer post is coming later in the week with an outline of the technical architecture of the site, and a few more will follow exploring more nitty gritty details.

For now, please take a look, and let us know what you think!

As I was writing a few links crossed my radar that felt relevant but didn’t fit in the post itself, so I thought I’d compile them here. None of them have had a direct impact on my post, but there are tangential connections aplenty.

Varnish 3.0 Changes: We’re using Varnish quite heavily so it’s good to see some momentum behind its new version. We were bitten by the behaviour of Edge-Side-Includes with gzip’ed content so I’m particularly pleased that that’s getting some love.

If all you have is a hammer…: Rachel Andrew explains why she uses wordpress rather than one of her company’s CMS products. It’s a very good response, and I’m hoping we’ve exhibited a similar pragmatism in architecting (which sounds overly grandiose, perhaps “piecing together” would be better) Alpha.gov.uk.

What is #devexp?: Adewale Oshineye’s write up of a set of ideas around how to improve the experience of using development tools and libaries: “Developer Experience (#devexp) is an aspirational movement that seeks to apply the techniques of User Experience (UX) professionals to the tools and services that we offer to developers.” It’s a far cry from where we currently are with Alpha.gov.uk tools but I’d hope some of this thinking will be included in any future development programmes.

Summary of the Amazon EC2 and Amazon RDS Service Disruption in the US East Region: We’re heavily dependent on Amazon EC2 for alpha.gov.uk so I was glad they published such a thorough explanation of what caused their recent outage and what they’re doing about it.

Alpha Conversation: Richard’s post on accessibility kicked off a flurry of discussion on Saturday morning. Public Strategist pulled some of it together. It’s great to see our work triggering public debates in just the way it should.

Cloud Foundry Blog: VMWare’s work on CloudFoundry is really impressive and if it had come along a couple of months earlier we might well have been tempted to make use of it. As it is, I’m looking forward to playing with it more once alpha.gov.uk settles down a bit.