Notes


3
Jan 12

Simple behind-the-scenes API authentication with OAuth2

Like many others I’ve been spending a lot of time with OAuth2 lately. The single-sign-on system we’ve built at GDS acts as a very simple oauth provider for our other apps (effectively just joining up the oauth2-provider and devise gems), and we’re probably going to be extending our API adapter code so that we can use it for those apps whose APIs need authentication.

What I’d not explored for a while was the simplest way to implement app-to-app oauth where there’s no UI for user interaction so over the New Year break I pulled something together for another project. It’s all pretty straightforward but not very well documented so I thought I’d better share.

The easiest thing to do if you want to allow an oauth client to work with your app is just to generate the ID, secret and access token for whoever’s responsible for the app and to provide them (securely) for direct use.

In order to do that in the rails app I was focussed on I knocked up a class to help me with that when using the aforementioned oauth2-provider:

and then a few rake tasks for interacting with it:

In the oauth-provider world, any “authorization” can be owned by a resource, which is any other model in your app. In a standard app like our SSO solution that’ll probably be a user, but in the app I’m working on here it’s an organisation that may have many users. You get access to that resource in your controllers with, eg:

And with that I had my API protected using everyone’s favourite standard authentication protocol.


2
Jan 12

Outside-In APIs

We spend a lot of time at work talking about APIs so Anant Jhingran’s “Six API predictions for 2012” was a particularly relevant read among the current glut of review/prediction pieces.

The section on “API-centric architectures” particularly chimes with our approach and the idea of an “outside-in model” resembles what I was getting at in “Building APIs, building on APIs“. I quite like the use of the phrase “outside-in”, and the iterative approach implied in:

In an outside-in model, one would start with easy consumption (read REST) of perhaps “chatty” APIs and then improve upon them. This is in contrast to thinking performance first and ease of use second.

As with anything user-centered this approach does require some sense of who those users are, and as ever that’s going to be the biggest challenge in most cases. To follow through, organisations are going to need to be proactive in understanding the value for others in our systems and try even harder to approach them as outsiders might.


5
Oct 11

Where Is He Now?

Nearly five months ago we revealed Alpha.gov.uk. And then this blog became even quieter. It’s been a few months of big transitions, two of which had to be kept quiet for a while. Since I last wrote here we’ve been busy arranging a move of house (from Harringay to Homerton), have discovered we’ve got another child on the way, and I’ve become a Civil Servant.

All three are exciting changes, but it’s the last that I sat down to write about here. For several years I’ve been working on building Ket Lai, initially alone, then with James Weiner, and gradually with a wider selection of collaborators. Things were going well, and we were building up a solid base of clients and getting close to having a couple of products of our own to release. But when the call to the public sector came, it didn’t take long to agree that we’d put Ket Lai on the back-burner and jump on board.

It’s a really exciting time to be joining, becoming part of the new Government Digital Service team and working (as tech lead) on the new Single Domain beta. We’ve had some great new people sign up to join us over the past few weeks, building a team I’m really enjoying working with.

For those who want a little more detail I’ve written a couple of pieces for the GDS blog: one about our approach to APIs and another about our platform choices.


9
May 11

Reading before writing (about alpha.gov.uk)

I’ve been working on an entry for the Alpha.gov.uk blog for publication later this week. The idea is to give a quick overview of how we’ve approached the technical side of building that prototype. It’s been tricky as we have a very diverse audience and a lot of ground to cover, but hopefully it’ll be a helpful start and the coming (post-reveal) weeks will allow a bit more space to expand on some of the key components. Maybe even open source a thing or two?

As I was writing a few links crossed my radar that felt relevant but didn’t fit in the post itself, so I thought I’d compile them here. None of them have had a direct impact on my post, but there are tangential connections aplenty.

Varnish 3.0 Changes: We’re using Varnish quite heavily so it’s good to see some momentum behind its new version. We were bitten by the behaviour of Edge-Side-Includes with gzip’ed content so I’m particularly pleased that that’s getting some love.

If all you have is a hammer…: Rachel Andrew explains why she uses wordpress rather than one of her company’s CMS products. It’s a very good response, and I’m hoping we’ve exhibited a similar pragmatism in architecting (which sounds overly grandiose, perhaps “piecing together” would be better) Alpha.gov.uk.

What is #devexp?: Adewale Oshineye’s write up of a set of ideas around how to improve the experience of using development tools and libaries: “Developer Experience (#devexp) is an aspirational movement that seeks to apply the techniques of User Experience (UX) professionals to the tools and services that we offer to developers.” It’s a far cry from where we currently are with Alpha.gov.uk tools but I’d hope some of this thinking will be included in any future development programmes.

Summary of the Amazon EC2 and Amazon RDS Service Disruption in the US East Region: We’re heavily dependent on Amazon EC2 for alpha.gov.uk so I was glad they published such a thorough explanation of what caused their recent outage and what they’re doing about it.

Alpha Conversation: Richard’s post on accessibility kicked off a flurry of discussion on Saturday morning. Public Strategist pulled some of it together. It’s great to see our work triggering public debates in just the way it should.

Cloud Foundry Blog: VMWare’s work on CloudFoundry is really impressive and if it had come along a couple of months earlier we might well have been tempted to make use of it. As it is, I’m looking forward to playing with it more once alpha.gov.uk settles down a bit.


11
Mar 11

Empowered, engaged adults

Matt Thompson’s “A 5-minute framework for fostering better conversations in comments sections” has cropped up in my twitter feed several times over the past few days but it wasn’t until the flight to SxSW that I got a chance to read it. It collects together lots of sensible stuff, and distills it quite helpfully. Definitely something I’ll come back to next time we’re designing commenting systems, or their like.

But the line that really leapt out at me was:

“The very best filter is an empowered, engaged adult.”

(It comes along as part of a response to Clay Shirky’s comment that “There is no such thing as information overload, there’s only filter failure.”)

It seems to me like a very handy telling of something we don’t hear often enough. In all the chatter about what living in the flow, and what being surrounded by twitter and the like will do to our brains, there’s a lot of taking sides, some discussion of automated filters, but not really enough stepping back and wondering what it means to be an “empowered, engaged adult” in the midst of it. We can entirely disengage, we can build better software, but at some point we always need to fall back on self-awareness and self-restraint to mediate whatever we’re surrounded by.

(it feels quite appropriate to be saying that while on a plane (and hence disconnected) and at the start of Lent)


6
Feb 11

Week 178

This week has followed in much the same vein as the last. Lots of rushing between meetings, squeezing in the time to do the work that needs to fit around them. There were a couple of exciting meetings about some projects for the coming months, and a slightly harder one trying to figure out whether there’s a realistic business model around a product we’ve been exploring over the past few months. And an afternoon learning a little about Nokia’s new developer platform (on which more, later).

There was also a day on site with a client trying to figure out how to simplify the way they describe a number of their processes. We’re working on a new system to replace the suite of spreadsheets and pile of papers that they currently use to track their work in progress, and that’s coincided with some bit internal changes for them. Figuring out how to describe and simplify their processes is a challenge, but an enjoyable one.

We’ve got a few new projects coming up–one for a composer, one for a new record label, another tbc–that I’m very pleased about. They’re small jobs that won’t bring in lots of money, or even really challenge us technically, but it’s a style of project we enjoy and that we’re keen to keep working on. This is a chance to not only serve those clients but do a bit more work to figure out how we can make that work sustainable.

The next week isn’t meeting-free, but should be a more settled one. I’m rather looking forward to that.


30
Jan 11

Continuing to consider Drupal

A little over three years ago I wrote a piece entitled “Assessing Drupal as a Rails developer.” In it I attempted to lay out a few of the reasons why I found Ruby on Rails a much more comfortable platform for building web applications than Drupal.

Over the intervening years, Drupal has continued to grow in popularity, and recently we’ve seen the release of Drupal 7. Rails has seen some radical changes with the release of its third major version, but I must confess I’ve no idea what’s happened with its profile and adoption rate.

Over that time I’ve built and deployed many, many Rails applications, and also brought a couple of Drupal sites to the world. Most notably recently there was News Sauce. Drupal’s become the go-to CMS for much of the not-for-profit/charity world and so it’s something we’re constantly monitoring and experimenting with.

The experience of tidying up News Sauce (for the version currently in beta) I was able to get a clearer sense of progress in the Drupal world. One of the major pain points referenced in that older blog entry was the duplication involved in deploying new features, and I’m delighted to see that there’s been huge progress on that front in the form of the features module. Where once you had to go through the cumbersome steps of duplicating a series of interactions, and/or writing a macro, there’s now a relatively easy way to export changes made in the UI to code and to track changes that would affect that code. It’s quite an impressive piece of work, especially when coupled with views and context. It’s great to see that further maturing in the form of the kit specification that begins to give some shape to how the exported features should present themselves (and so how they should interact).

In that context I was pleased to see Jeff Eaton’s recent Drupal 8: The Road Ahead. For Drupal to become a still more compelling platform it’s time for some of the sensibilities that informed the development of features to inform the core system. In general Jeff’s proposals make a lot of sense: some new APIs, a stripped back profile for those who want to build from scratch, and a more developed profile for those who want something that might be described as “a typical drupal site”. It’s all quite good to see.

The changes aren’t enough to make Drupal my first port of call for most of our projects. Ruby—usually with Rails or Sinatra—fits the bill better, and is still my preferred language. But I am far more comfortable with Drupal as a platform than I once was, and I can see it taking a larger role in a number of our upcoming projects.


29
Jan 11

Week 177

I lost my way with these back at the end of August, but it feels like it could be helpful to start again. So here we go…

I’ve spent a fair chunk of the past week travelling around London meeting people for coffee. We’re in a position that’ll be well known to many freelancers and small companies — several jobs coming to an end, but nothing big lined up to replace them — and that’s spurred me to get out and catch up with people, explore possibilities, think about what’s next and expand our networks a little bit.

That’s quite a good way to spend a week, but does conflict with the small matter of actually finishing the current round of projects to deadline, so there’s also lots of evening and weekend working. A sign that we need to work out how to build a bit more redundancy into our processes.

For the past couple of months we’ve been quietly working on a little product we hope to launch very soon. It grew out of some research we did for a client that revealed a little gap in the market. At some point we hope to develop some product ideas that we see as potential money-makers, but this one’s chiefly a learning experience.

We spend a lot of time helping clients build their products, but we don’t see the full process end-to-end. So with this we’re not just doing the build, but we’re setting up the payment systems, we’re meeting with a lawyer, and all that sort of thing. It’s led to a few delays, but will also better equip us to help our clients, and to launch our future products. More on that in due course.

Yesterday saw us at the Design of Understanding conference at St. Bride’s. (Check out these great notes on that. I’m still processing mine) As with all conferences there were some ups and downs, but it was a great day with a good crowd and several very good talks.

Small, focussed conferences are definitely my favourites, but 2011 will also see me going back to SxSWi. And we finally have accommodation sorted out for SxSW. By the time we decided to go all the hotels within 6 miles of the convention centre were booked up, but the wonderful AirBnB came through for us and we’ve got a nice little cottage a couple of miles away.


25
Jan 11

History Hack Day

My officemate Matt has spent the past few weeks putting together the inaugural History Hack Day, which took place at the Guardian offices last weekend. I was only able to attend for the kick-off talks, but they were great, with Matt Sheret‘s exhortation to be timelords fitting especially well.

Jeremy Keith has done a great job of writing up the various hacks that emerged from the weekend and I’m gradually working my way through them. I’ve very much enjoyed watching Simon’s geStation which plots the openings of Britain’s railway stations onto a google map. As Jeremy says:

On the face of it, it sounds like just another mashup of datetimes and lat-long coordinates. But when you run it, you can see the story of the industrial revolution emerge on the map.

I initially opened it in a rather overloaded browser and everything ran very slowly. Switching to a different browser sped things up a lot, but I realised I’d enjoyed the slow pace rather a lot more – there was something quietly engaging about watching each station pin emerge in turn and spending several minutes seeing the country gradually connecting up.


25
Jan 11

Thoughts on the state of the "East London Tech City" initiative

Last Monday I was at the Department for Business, Innovation and Skills to explore the government’s “Tech City” scheme to do something to do with the technology industry in an area loosely defined as “East London”.

I’d been wondering before sitting down to write this how best to sum up my reservations about the scheme, not really realising that just writing that first sentence would actually begin to draw them out quite quickly. An initiative has been announced, a name has been given, but very few people seem to actually be sure what is being proposed, or where or at whom it’s targeted. That made it very hard to work out how to engage with the event.

There were a range of different interests represented, small development shops like ours, a couple of early stage startups, some academics, a few folks who’d been through the startup experience and emerged with successful firms, civil servants, VCs/bankers, and quite a few big tech vendors. It wasn’t hard to discern who fell into each group as time and again it became clear that there were quite distinct agendas in play, not to mention significantly different interpretations of the dress code. And it was a clear a number of those present could do with reading Alex’s politician’s handbook to East London, with requests for “free wifi to enable cafe working” suggesting people hadn’t spent much time in the current crop of cafes around Shoreditch.

There were a few stimulating conversations in the breakout groups but it’s sad that one of the most positive outcomes was that the lack of focus so quickly became apparent! Are we talking about encouraging tech firms to take over the Olympic Park post-2012, or are we talking about changes in Shoreditch? Are we looking to bring in foreign tech giants who are well capitalised and need a European office, or is the primary focus supporting the establishment and growth of indigenous companies?

Those are all valid questions to be asking when exploring what government can be doing to support the British tech sector, but it feels like they’re the questions that come before an initiative is announced with much fanfare and a variety of “commitments.”

It feels like the whole thing is starting on very poor footing. Early on there was a statement (which I’m not meant to attribute because we were under Chatham House Rules) that the name “Silicon Roundabout” had been dropped because “it doesn’t translate around the world.” That was indicative for me that there was no real commitment to exploring the uniqueness of London and the myriad tech communities that thrive within it. I’ve a certain fondness for the Roundabout moniker, though no deep commitment to it, and its key attributes in this context are that it emerged from the community of small tech companies in the Old Street area, and that it relates a certain commitment to and humour about the place. The idea of dropping it didn’t go down well among the locals I was attending with.

Understandably, the consultants from McKinsey running the event were at pains to push those of us there for some concrete answers to “what should the government do?” And given that the initiative is already under way we probably need to come up with some answers… I don’t have many specifics as yet, but I know I’d like them to start by:

  • splitting the initiative along geographical lines: East London isn’t a single entity and different neighbourhoods have different needs. Pick a couple of areas and produce tailored plans for each.
  • acknowledging that small companies generally generate more jobs than big ones (and that “rapid growth” isn’t the only way to be successful): where there’s an existing ecosystem of small companies ask them what they need without the implicit addition of “in order to become big companies”