Liminal Existence

This blog has moved

2010-04-29T15:23:00.001+01:00

This blog is now located at http://blog.romeda.org/. You will be automatically redirected in 30 seconds or you may click here. For feed subscribers, please update your feed subscriptions to http://blog.romeda.org/feeds/posts/default.

Identity

2010-04-26T23:15:00.002+01:00

The web is facing a serious identity crisis. Many have written about it but, having thought a lot about this problem over the past few years, I can't help but feel that something important has been missed in most discussions.

Aza Raskin cuts to the heart of the matter:

“Your identity is too important to be owned by any one company.
Your friends are too important to be owned by any one company.”

I'll go one step further, and say that the centralisation of identity is stifling innovation on the web. Kellan recounts a quote from a friend, on the subject of Facebook's f8 announcements: “Well, [Facebook] gave Foursquare a 6 month reprieve.” This is not a hopeful view.

So what's the alternative? What's missing from the conversation? I think it's important to take a view from the perspective of usability; how are we going to use this new conception of identity? The answer is simple: exactly as we do today. The fundamentals of social networking haven't changed from day one. There's a website, you log on, and you add friends so that you can share content with them. This basic model applies to every successful social internet technology, from email to IM to Friendster to Facebook to FourSquare.

Logging On

Logging on is easy. Whether it's a password, OpenID, @anywhere, Firefox Contacts, or Facebook Connect, the principle remains the same: the user proves to their server that they are a particular individual. In that sense, any protocols or approaches beyond username and password are just icing on the cake. They're ways to make logging in easier, to increase conversion rates (at least, in theory), but they don't fundamentally change what we can do on the web.

So once we've logged on, how do we add friends?

Adding Friends

This is the part that's missing from the conversation. How does it work today? Well, you have two options: either you find someone's profile page and click "add" (which doesn't work across sites, or if you can't find your friend's profile page), or you find other users on the site by entering their email address. Often the latter approach is achieved by the site opening up your email address book and looking for friends in bulk, but fundamentally it boils down to "Find email address of friend, search users database for that email address, add friend."

OpenID doesn't help, because I don't even know my OpenID URL, let alone my friends' OpenID URLs. The "ID" in OpenID is a bit of a misnomer. OAuth doesn't help – Twitter and Facebook use OAuth under the covers for @anywhere and Facebook Connect, but that only helps me, the site using @anywhere/FB Connect, and Twitter and/or Facebook themselves. It also doesn't help us get away from Aza's point, that your identity remains in the hands of a single company.

The solution, as I said to Tim O'Reilly, is Webfinger. Indeed, webfinger was born out of struggling with exactly this problem of representing multi-faceted identity on the web in a way that can't be controlled by any one company. The approach is essentially to invert the currently-closed user databases, and put social network affiliation in the hands of the users, in our hands, all while keeping the user experience the same as with current and past social software.

I won't go into the technical details here, but in essence the workflow from a site builder's perspective looks like this:

Kate (kate@inktopaper.com) meets Fiona (fiona_z_342@gmail.com) at APE, and the two exchange email (re: webfinger) addresses.
Kate wants to stay in touch with Fiona, so she logs on to her social network of choice ("ZineSpace"), and enters Fiona's address.
ZineSpace supports photos, microblogging, and calendaring, and discovers via a webfinger lookup that Fiona has a photostream at Sketchr, tweets at identi.ca, but doesn't share a calendar. ZineSpace uses PubSubHubbub to send subscribe requests to Sketchr and identi.ca on behalf of kate@inktopaper.com.
Sketchr and identi.ca look up Kate's webfinger profile, and use her published photo to show the incoming friend/follow request to Fiona.
Fiona uses her identi.ca microblog for work, so she declines that invitation, but approves the friend request on Sketchr, and adds Kate as a friend (asymmetric follow) on Sketchr.
Even though she declined the identi.ca request, Fiona wants to keep up with Kate's life, and has a personal Tweetter account that's not published on her public webfinger profile. She logs in there and adds fiona_z_342@gmail.com as a friend.
Tweetter sends a subscribe request to ZineSpace (discovered via the webfinger profile) on behalf of kate@inktopaper.com. ZineSpace knows that Fiona wants to see Kate's microblog posts, so auto-approves the request and sends a reverse-follow request (which is again automatically approved).

Now, five things are important to keep in mind when thinking about this process:

Shareable Addresses are what make this exchange possible. Kate and Fiona can't be reasonably expected to remember all of their various profile URLs, nor can they be expected to remember each other's profile URLs. Their webfinger addresses act as mnemonics for their distributed identities.
Webfinger is just a discovery mechanism. HTTP remains the transport mechanism for this approach, which means that everyone can participate.
We have not exposed any personal information about Kate or Fiona, and more importantly, we haven't exposed any information about Kate or Fiona's relationships. They can do that if they wish (e.g., by linking to a FoAF or XFN profile from their Webfinger profiles), but the approach doesn't make any assumptions about what information must or must not be shared.
Subscription = Relationship. The underlying approach doesn't say what kind of relationship the two are creating, but rather allows protocols or data transports on top of the exchange to do so. Fiona could decide that she doesn't like Kate's sketches, or that Kate posts too much, and simply tell Sketchr to hide Fiona's posts. As far as Kate's concerned, Fiona is still subscribed, and still viewing her photos. Alternatively, Fiona could send an unsubscribe request to Kate, signalling that the relationship no longer exists. The semantics are up to the application developer at either end.
No Passwords are involved in the exchange. Kate and Fiona don't need to exchange PGP keys, either, or rely on some complicated "Web Of Trust." All they need to do is trust the servers they use (zinespace, sketchr, identi.ca, and their email providers); if they can't do that, then they have bigger problems. This one's really important, because RSS and Atom were meant to be the future of content exchange. RSS and Atom have completely failed to enable private feeds, because they require passwords to do so. That sucks, and webfinger offers a workable solution to this persistent problem.

Sharing Content

Once relationships have been established, sharing content is entirely up to the individuals and sites involved. Obviously, we need common protocols for this, but we can use Atom and PubSubHubbub, some domain-specific protocol (e.g., Portable Contacts/ActivityStreams), or we'll figure it out as we go along. Direct or private messages are just special forms of general distribution content (i.e., the subscriber has asked to receive content from the publisher, whether that content was broadcast or is a directed message).

There are a lot of things that I haven't covered in this post. The technology is simple, but not trivial, and it is still very new. There aren't yet tools that make this easy (if you'd like to work on building them, contact me!)

Hot Code Loading in Node.js

2010-01-30T23:56:00.005Z

Reading through Fever today, this post by Jack Moffitt caught my eye. In it, he discusses a hack to allow a running Python process to dynamically reload code. While the hack itself, shall we say, lacks subtlety, Jack's post got me thinking. It's true, Erlang's hot code loading is a great feature, enabling Erlang's 99.9999999% uptime claims. It occurred to me that it wouldn't be terribly difficult to implement for node.js' CommonJS-based module loader.

A few hours (and a tasty home-made Paella later), here's my answer: Hotload node branch.

Umm… What does it do?

var requestHandler = require('./myRequestHandler');

process.watchFile('./myRequestHandler', function () {
  module.unCacheModule('./myRequestHandler');
  requestHandler = require('./myRequestHandler');
}

var reqHandlerClosure = function (req, res) {
  requestHandler.handle(req, res);
}

http.createServer(reqHandlerClosure).listen(8000);

Now, any time you modify myRequestHandler.js, the above code will notice and replace the local requestHandler with the new code. Any existing requests will continue to use the old code, while any new incoming requests will use the new code. All without shutting down the server, bouncing any requests, prematurely killing any requests, or even relying on an intelligent load balancer.

Awesome! How does it work?

Basically, all node modules are created as sandboxes, so that as long as you don't use global variables, you can be sure that any modules you write won't stomp on others' code, and vice versa, you can be sure that others' modules won't stomp on your code.

Modules are loaded by require()ing them and assigning the return to a local variable, like so:

var http = require('http');

The important insight is that the return value of require() is a self-contained closure. There's no reason it has to be the same each time. Essentially, require(file) says "read file, seal it in a protective case, and return that protective case." require() is smart, though, and caches modules so that multiple attempts to require() the same module don't waste time (synchronously) reading from disk. Those caches don't get invalidated, though, and even though we can detect when files change, we can't just call require() again, since the cached version takes precedence.

There are a few ways to fix this, but the subtleties rapidly complicate matters. If the ultimate goal is to allow an already-executing module (e.g., an http request handler) to continue executing while new code is loaded, then automatic code reloading is out, since changing one module will change them all. In the approach I've taken here, I tried to achieve two goals:

Make minimal changes to the existing node.js require() logic.
Ensure that any require() calls within an already-loaded module will return functions corresponding to the pre-hot load version of the code.

The latter goal is important because a module expects a specific set of behaviour from the modules on which it depends. Hot loading only works so long as modules have a consistent view of the world.

To accomplish these goals, all I've done is move the module cache from a global one into the module itself. Reloading is minimised by copying parent's caches into child modules (made fast and efficient thanks to V8's approach to variable handling). Any module can load a new version of any loaded modules by first removing that module from its local cache. This doesn't affect any other modules (including dependent modules), but will ensure that any sub-modules are reloaded, as long as they're not in the parent's cache.

By taking a relatively conservative approach to module reloading, I believe this is a flexible and powerful approach to hot code reloading. Most server applications have a strongly hierarchical code structure; as long as code reloading is done at the top-level, before many modules have been required, it can be done simply and efficiently.

While I hope this patch or a modified one will make it into node.js, this approach can be adapted to exist outside of node's core, at the expense of maintaining two require() implementations.

Easy Android

2009-05-05T21:49:00.006+01:00

Let me start by saying that I'm very impressed with Android, and the ease with which I was able to scratch an itch was impressive. The fact that I'm not locked into Apple's app store world is nice; I don't know what the specific terms are for Google's marketplace (I haven't signed up yet), but frankly I trust them more than I do Apple.

Full Disclosure: at Social Web Foo, Joshua very kindly gave me (and about 50 others) a free Android Dev Phone. He gave strict instructions to actually write something for it, and I probably wouldn't have bought a dev phone (nor written <shudder> the Java against the emulator), so the fact that I have done so worked out.

I hadn't written a line of Java before. Shock, horror. It's just as annoying as everyone always said, but developing for Android with Eclipse is pretty straightfoward. The documentation is good. I started with the basics, and pretty quickly moved to example code. Most notably, the NotePad application that comes bundled with the Android SDK.

Once the Eclipse and the SDK was downloaded and installed, it only took about five minutes to get my first app up and running. Getting a list view of hard-coded data took another ten minutes, and modifying the code to display a different view when one of the items was clicked took about two hours (keep in mind that I was learning Java with the patience of a Ruby programmer here).

The Bad

Honestly, there's not a lot that I was upset by. Coming from Rails, defining table structures and setters and getters and all the explicit typing is pretty annoying. Cutting and pasting code meant that I had a few mismatched data structure definitions, and the error messages were less than useful, since all that fancy type matching means that when something doesn't match up in your XML configuration file, Java can't tell you where to go to fix it. Figuring out where I had mismatched strings in XML config files easily took another couple of hours, which sucked, but it seems like the kind of thing that you'd get used to. Functional brain damage, I suppose.

The Good

Android is a developer's platform. The way that content, and well, everything is addressed is fantastic. There are hooks for everything, and the tutorials encourage you to do the right thing out of the box. The documentation really only makes sense once you get it, but for the really simple app I've been working on, the examples were more than sufficient to get things going.

Basically, every data source is addressable through a "content://" URI scheme. What that means is that any application can provide hooks (if they know about your data source) to view, edit, or list bits of data. I expose a set of short recipes (from @cookbook) at content://org.romeda.provider.Cookbook/recipes. That means that as long as, say, Tweetie knows that there's the provider exists, they could add a hook to allow people to add any tweet as a recipe. It also provides hooks for other applications to know when new recipes are published (or, for example, I could tie into a Twitter provider on the phone and piggyback discovery of new updates to that, rather that running my own polling process), in addition to hooks to view, edit, or any action you can imagine.

The best thing about this is that the whole system works the same way. You register an "Intent", and the OS lets you know when something in the system is relevant. The simplest (but seriously awesome) example of this is if you want to intercept an outgoing call and rewrite the number (or just make the call over a voip stack, for example). You just register your intent to handle ACTION_NEW_OUTGOING_CALL, and away you go. A simple data passing interface lets you receive and attach data to the messages.

The other thing that's great about Android that I noticed right away is that the default views are extremely simple to use and customize, and they save their own state. Without writing any special code to remember where a user is in the scroll buffer, and without doing any work to remember which view a user was in (e.g., list or item view, edit, etc), the default behaviour is to remember. It's the embodiment of everything Google's been doing on the web lately — don't save, ever, because saving is stupid. Either you've published/archived/sent/deleted something, or it's in a draft form. The draft is implicitly persistent, and avoids the user ever losing work. This is in stark contrast to the iPhone, where Safari's horrible constant reloading of pages boggles the mind, and burns through roaming data minutes like there's no tomorrow.

The Code

So it's not fancy, and it doesn't even fetch the recipes yet, but I'm posting the code here since it's pretty damned simple at this point, and demonstrates making an app with two views. I'll update it as time allows.

Simple Addressing for the Web, Part 1

2009-05-05T17:05:00.003+01:00

Addressing is important. It's something that many people have tried to solve.

I'm interested in addressing because it's an important piece of web-scale messaging, and of the federated social networks that are an emergent property of verified cross-site communication. In order to communicate with someone, you need to be able to route your communications to them.

The URL is the thing. Except when it's not.

The URL was supposed to become the way that we negotiated identity. We were supposed to have a "home page," a place on the internet to call our own. It didn't quite work out that way, and at the same time as Geocities is shutting down, we're finally facing the need for a strong conception of identity on the web.

It goes without saying these days that everything we do, everything we interact with, has an associated URL. I can give you my blog URL so that you can read my posts, or my calendar URL so that you can invite me to events. However, for the vast majority of users, URLs aren't a viable option. Fundamentally, it's a lack of consistency (or, put another way, unbridled diversity) that makes URLs unusable as identity markers. Take the following URLs as a proof-by-example:

twitter.com/blaine
myspace.com/romeda
flickr.com/lattice
search.twitter.com/search?q=%22Swine+Flu%22+OR+Flu
home.myspace.com/index.cfm?fuseaction=user
blogger.com/post-create.g?blogID=6135683561277543562
amazon.com/gp/pdp/profile/A1GUHSGP27QA4W

All of the above are URLs which I see while interacting with sites on the web. Unlike postal addressing, phone numbers, or email, there's no consistency. The path part of the domain may as well be line noise in the latter four examples. By association, the pattern used by Flickr, MySpace, and Twitter is a fluke. Beyond that, my username doesn't match across the three social networking sites, and as such it's nearly impossible for a friend, relative, or co-worker to guess what my URL is, even given a domain.

I don't see a way to fix URLs across the web so that we can encourage people to use them as identifiers. OpenID has tried, and the results are nothing short of abysmal.

Back to the Future

If not URLs, what should our new web addressable identities look like? The simplest answer is "like an email address." They're universally recognizable. Billions of people have email addresses and know how to use them. All the major IM providers have moved towards email-like addresses as identifiers (gone are the integers of ICQ). Most importantly, email addresses are easy to construct and resolve.

The net result of this line of thought is that instead of @blaine for my Twitter address, I'd be blaine@twitter.com, and on Identi.ca I'd be blaine@identi.ca. I could share my Myspace identity as romeda@myspace.com, and on Facebook I could be blaine.cook@facebook.com.

The problem is that those addresses conflict with an already-existing namespace, specifically email. Which isn't surprising, but it is problematic. Can you send me an email at blaine@twitter.com or romeda@myspace.com? What happens when you do? Unfortunately, there aren't clear answers for those questions, and while some social networks might choose to make "Social Network Addresses" work as email addresses, it would be an uphill battle to convince all providers to do so.

Use What's Already There

I've been thinking about this problem a lot lately, and while the approach of re-using email semantics to provide human-readable web addresses/identities is very attractive, the proliferation of addresses (one for each network) and namespace collisions are less than ideal. After having extensive conversations with Alexis Richardson and Tony Garnock-Jones, the general approach for discovery became clear to me, but I didn't have a more generally applicable form for the addresses themselves.

Eventually, talking over the problem with John Panzer and Breno de Medeiros at Social Web Foo, the solution was there, blazing as bright as the California Sun; Google Profiles means that Google is now providing links to all my social network profiles. They're also my email provider.

My email address is romeda@gmail.com. If you transform that to http://www.google.com/profiles/romeda, you get my profile data, and away we go. Every email provider these days has a website, and Eran's LRDD, new on the scene, provides a discovery mechanism that everyone (i.e., every mail provider, even if they're only hosting static content) can implement in just a few minutes.

This is an important, exciting transformation. Now, with one identifier, I can share all the social bits of myself to anyone I please.

Where are my photos? romeda@gmail.com.

Where's my calendar? romeda@gmail.com.

What's my phone number? Look it up with romeda@gmail.com, and I'll give you permission to see it and store it in your address book.

They're all the same.

If I want to share a different set of social interactions, say, my work identity, I can give my Osmosoft or BT addresses, blaine@osmosoft.com and blaine.cook@bt.com, respectively. Now just photos of conferences come up, and the calendar that you'll find is my work calendar, not my social calendar.

Talking about this problem with others unearthed a post last year by Brad Fitzpatrick and EAUT, which were both aimed at solving the OpenID problem, but both take the same approach as the one that I outline here. EAUT seems to have been lost in the swamps of XRDS-Simple, and Brad's post was probably too early to the races, in true Brad style (if you want to know what's coming to the internet in five years, just read his blog posts).

With a swift and general agreement-in-principle, there's been some very positive movement towards promoting this concept as a way to bring the power of strong identity that email provides to the web. John has an excellent post on the subject, and it seems like a name for the project has emerged: WebFinger.

Part Two (coming tomorrow) goes in depth about how this all works on the tech side. Bits on the wire, as Tim Bray says.

Automatic Bootstrapping of rev=canonical

2009-04-13T09:54:00.004+01:00

Kellan's work on making URL shortening not suck is great, but killing bit.ly and tinyurl just isn't going to happen. Sadly for Joshua, this is the way the internet works, and if he doesn't like it, tough shit.

So far I've only seen one site that (awesomely) shortens URLs with rev=canoncial (I'm sure there are more, but I haven't seen them. So there.) Simon Willison has done some great work on his blog, and throwing http://simonwillison.net/2009/Apr/11/revcanonical/ at Kellan's shortener results in http://swtiny.eu/EZa. Brilliant!

Except no one will use it, because, well, bit.ly is useful for doing things like tracking how far that link you sent got, and there's a degree of muscle memory involved. This is the sort of vi-versus-emacs argument that just isn't going to go away. Also, here's the same post shortened by bit.ly, tinyurl and this Simon-Willison's-post-about-rev=canonical specific URL shortener that I just linked to in the previous paragraph. So why fight it? You really can't win.

There is hope, though. I live in the dark ages, and Blogger publishes my posts as static HTML over SCP or FTP or some other totally inappropriate protocol. Since there's no <$BlogItemShortURL$> tag in Blogger's template syntax, I'm completely unable to do what Simon's done, without migrating to a self-hosted blogging system (contrary to popular belief, not all programmers are compelled to write their own blogging systems (erm, on second thought, ignore twitter)).

Anyhow, it turns out that I can be as cool as Simon, just with one step of indirection:


<link rev="canonical" href="http://bit.ly/?url=<$BlogItemPermalinkURL$>"/>
<link rev="canonical" href="http://tinyurl.com/create.php?url=<$BlogItemPermalinkURL$>"/>

Strike that, Blogger fucking sucks and so I've created a shell script that converts a placeholder into rev=canonical links. Man, that was a pain in the ass. Why do I still use Blogger? Anyhow, the point stands if your blogging software doesn't totally suck and will give you a permalink anywhere in the template engine. Which is probably true unless you're using Blogger. Ugh.

So this is great. Now my blog posts are rev=canonical™ compliant, and I you didn't have to change anything at all, beyond shoving a couple of lines of HTML into your blog template. People that use bit.ly and tinyurl are happy, because they don't need to change their behaviour, and people that use rev=canonical are happy, because they can just by following the links provided for them.

Now of course this doesn't address two of Joshua's concerns. First, I still have no idea where my traffic is coming from, because I don't run my own URL shortener. I don't want to run my own URL shortener. What I see here is an opening for bit.ly and/or tinyurl to allow me to see the stats of redirects (Dear FeedBurnerGoogle: Please purchase bit.ly or tinyurl or build your own), which they should be able to do easily since all I would need to do is prove that I own romeda.org by sticking some secretly named file at the domain root or otherwise.

The second concern that remains unaddressed is what happens when the URL shorteners go away? Well, we have the same problem on the web. The answer to that was/is The Internet Archive, so herewith The (Tiny) Internet Archive.

Currently it's just a proof-of-concept. I don't guarantee in any way that the links posted there will persist. That said, the free quotas that come with Google App Engine allow enough space to store around two million links, and $0.15 per month for every additional two million links, so I'm sure it won't be a problem. Who knows, maybe archive.org will take it over?

The code that's up is available on github here: TinyArchive. If you have suggestions, please send them my way or fork the code and send me patches. It was just a pre-coffee morning hack, my first stab at App Engine, and my first Python code in what seems like forever.

Facebook is Closed for Anything

2009-02-17T11:30:00.000Z

From Mark Zuckerberg's post responding to the criticisms of their new ToS:

“We think this is the right way for Facebook to work, and it is consistent with how other services like email work.”

If that's the case, then why the onerous licensing? I'm quite sure that people running email servers don't need to negotiate licenses to send and receive email to and from one another. Asymmetrical persistence is a feature, not a bug, of email.

Facebook couldn't figure out how to replicate email in a closed, centrally controlled corporate environment (hint: you can't). Resorting to legalese is a by-product of their broken closed model (yes, this includes their support of OpenID and their Facebook PlatformPlantation).

“There is no system today that enables me to share my email address with you and then simultaneously lets me control who you share it with and also lets you control what services you share it with.”

That's because you can't control such things. At the end of the day, I can write down a phone number from a screen, and paste it up on telephone poles anywhere I please, shout it out at the top of my lungs. There's no law that prevents me from doing that, and even if there were, it would be completely unenforceable.

The music and film industries have been fighting this losing battle for years, and frankly it's depressing to see a major web property co-opting the notion of openness while playing essentially the same DRM game.

With legalese like this, I'm expecting the big announcement at f8 to be OpenFacebookHappyDRM. Code name: Hotel California.

You can OpenID in any time of day, but you can never leave…

Velocity

2008-05-12T16:53:00.000+01:00

If you're interested in scalability and performance as it pertains to internet applications, or if you're involved in or looking to build a reliable internet-based system, go to Velocity.

I was fortunate enough to attend the O'Reilly Velocity Summit back in January, which was a Foo-Style event designed to provide some context leading into the actual conference in June. If the conference is one tenth as good as the summit, it will be amazing. Jesse is an amazing organizer and a man with a vision.

Anyhow, instead of reading my off-the-cuff posts about obvious things that are much more fun to talk about while drinking in Miami, go to Velocity and learn this stuff for real.

Scalability

2008-05-12T02:57:00.000+01:00

Updated: Go read Steve's Dynamic Languages Strike Back. It's a longer read, but it's much more interesting, and he's much smarter than I am.

LOL. <-- this is a link. Read Ola's post, first.

For all those who don't get it, languages don't scale, architectures do.

Now, some languages are faster than others. That means that to complete a given operation, it costs less, everything else being equal. Costing less is a good thing. But developers also cost money, so if you have to spend money on developers' time porting from one language to another then you might not be saving any money at all, and really you're just treading water.

Once upon a time, Shell Scripts were used to write CGI applications. With the correct architecture, and enough money, you could build Google with tcsh. No, really. It wouldn't be fun, and you'd be dumb, because there are much cheaper ways to do it. But then again, if you stuck with it, perhaps you'd optimize tcsh to be really fast at spawning and serving up web requests. Faster than Java, faster than <insert your favourite language here>. Faster means cheaper, it doesn't mean more scalable.

I point to exhibit A. Perl used to be slow. Now it beats JoCaml with the bestest concurrency (re: “Scalability”) around. What was Perl built for? Parsing text. Lots of it. All the time. It's fast. Does it mean that you can't build Wide Finder with another language? Absolutely not. Does it mean that you couldn't build Wide Finder to scale out to a trillion documents with gawk? If you answered “yes”, go back to the start of this post and read again! :-) If you're still answering “yes,” try reading some more. Leonard, Ted, Joe, Cal, and Theo are good places to start.

If you answered “no,” congratulations! Pat yourself on the back for knowing what scalability means.

FoWA Miami Rocked.

2008-03-08T14:59:00.000Z

I've been busy getting [Twitter] ready for SXSW, and have completely failed at email and, well, everything except work, since then. Before I actually land get drunk in Austin, I wanted to make a quick post about how great FoWA Miami was.

After giving my workshop, I attended Joe's workshop on scalability, which was an amazingly thorough discussion, and I highly recommend attending anything that Joe does in the future (including the panel that he, Cal, and others are on at SxSW. I didn't catch many of the session talks on Friday, as I spent much of my time talking to attendees, but the "Building a Web App in 45 minutes" panel was a fun experiment, and both Cal and Gary were energetic, brilliant, amazing, all that good stuff. If you get a chance to see either of them speak, don't pass it up. Especially Gary, as you're likely to get free wine, even if he makes you eat dirt beforehand.

I had some hiccups during the demo / discussion portion of my workshop, but the first part went well, I think, and I'm looking forward to some great applications that incorporate Jabber soon. My talk, "Bringing your web app to the masses" went well, except for the part where Twitter went down in the middle of it, and I got a call from work (which I waited until after the talk to answer).

Some heckling ensued, but I was happy to be able to address the audience's questions about Twitter in an open and honest way. The atmosphere that Carsonified has managed to foster at FoWA helped a ton.

Mel, Ryan, Lisa, Keir, and Elliot did a fantastic job organizing everything, and Tantek and Brian put together an amazing lineup of speakers and workshops. Seriously inspiring stuff.

If they'll have me, I'll definitely be going to any conferences they hold in the future. They've come a long way since the FoWA San Francisco in Fall 2006, and it looks like they'll just keep getting better.

Google Entaglement

2008-02-21T18:51:00.000Z

I've done a fair bit of security work, and generally try to care about the finer details of privacy and security. However, one of the things that I've learned is that more often than not, no amount of digital security past a certain point is going to help, since usually the threat model isn't an advanced technological attack, it's a social one.

Thus far, Google has done a pretty good job of keeping private things private and public things public. I've spoken to people on the Google Reader team, and the main reason they haven't added support for private feeds is their acute concern for privacy.

Today Google announced a limited trial of storing health records online. This seems reasonable and doable in a secure way, but I'm sure they'll get lots of unwarranted flak for the long-awaited project.

However, there will and should be some warranted flak. It turns out that they're using your regular Google account to store this information, and will provide access to it using your regular password, no doubt through yet another Google login page. I've heard concerns that OAuth supports phishing (from Google people), but project infighting and power struggles at Google that result in tens of login pages, all slightly (or dramatically) different, all using the same credentials supports phishing much moreso.

I strongly support patients' rights to access their medical information, and Google is probably one of just a handful of organizations that can do the necessary coordination work and stand up to invasive organizations at scale. However, they need to stop thinking of this data as theirs, because it's not — it's your data. Using the same password as your email to access your health records is something that should be actively discouraged. If Google wants to present a unified interface, they should expose an API and use OAuth or AuthSub, just like any other third party that would consume the data.

Now, I may be over-reacting, but I had an interaction yesterday that suggests to me that I'm not. Someone using GTalk sent a chat request to blaine@twitter.com; this email address has an MX record that resolves to mail.twitter.com, and the corresponding JID resolves to jabber01.twitter.com. However, I have claimed my blaine@twitter.com address on GMail, and associated it with my primary GTalk ID (romeda@gmail.com). When I accepted the chat request, the response came from my GTalk account, romeda@gmail.com.

In effect, Google had done something clever, and in so doing broke the Jabber spec, ignored my own self-hosted Jabber server, and exposed my personal email address without asking my permission.

In this case, it wasn't a big deal, I don't care, etc. Others might, though, and I only knew that it was happening because the person on the other end of the chat was tech-savvy enough to realize what had happened. Also, email addresses and connections between them are hardly closely-guarded secrets. The thing I take away from this is that Google is being sloppy. There's a lot going on, and it's hard to keep track of it all. That your health records are being tied to your Google account just reeks of some power struggle where the Google account people want to bolster their product's internal importance (or have managed to do so that they get veto power where they shouldn't have it), and it's simply not a pragmatic choice. There's a reason your health records aren't stored at the DMV, and it's not out of convenience. Just sayin'.

FoWA Miami

2008-01-21T07:19:00.000Z

Just a quick note that I'll be giving a workshop on building real-time web applications using Jabber at the Future of Web Apps in Miami, on February 28th. The conference runs from the 28th to the 1st of March, and should be a lot of fun.

We've been gradually improving the Jabber stack on Twitter, and we're now sending millions of messages every day, doing things that just don't fit into the polling-based world of Atom feeds. There are a ton of extremely awesome things that can be built, and so far we've just scratched the surface.

More to come; if I don't start blogging these things in small pieces, they'll never come.

Stability

2007-09-02T02:39:00.000+01:00

Most of the Twitter team's work in the weeks leading up to launching Blocks was to ensure that it wouldn't fall over as soon as we released it. It's an extremely punishing application, loading 10 timelines on every occasion that someone looks at it. So far, the servers haven't even noticed.

There have been a number of Twitter hiccups in the past few weeks, but they've all been weird, random bugs. Which is not to make excuses, but rather to say that in spite of (very time-consuming) challenges along the way, we've been myopically focused on making the site faster and more reliable. As evidence, here's a graph of page load times, as seen from an external observer:

We're going to keep building a faster and more reliable Twitter. We're also going to add some awesome new features, and soon. Possibly better than contact search and GMail, even! Finally, we'll have more visualizations from the Stamen folks. Britt is off to Berlin for RailsConf mid-September. We'll then have more details about what we're doing to push Rails and Twitter.

These are the people ...

2007-09-01T23:58:00.000+01:00

Folly: "In architecture, a folly is an extravagant, frivolous or fanciful building, designed more for artistic expression than for practicality." – via Tom Coates, by way of Tom Carden.

We just released Twitter Blocks, a nice little visualisation done by the good folks at Stamen Design. It's fun! Go play!

Stamen's recent work highlights the playfulness inherent to Twitter. I can't wait to release more of these interfaces, and hope that it inspires similar work. Sam Ruby, Tim Bray and others have recently weighed in with their long bets. I'm willing to put down that playfulness — of the sort that Stamen, Schulze & Webb and Jane McGonigal explore and invent daily — is so important to who we are as people that the tech world won't be able to ignore it for much longer.

Not exactly a risky bet, but too often the tech industry just ignores these things, so there it is, just for kicks.

SELECT * FROM everything, or why databases are awesome.

2007-06-21T20:44:00.000+01:00

I've just committed a patch to ActiveRecord that prevents a large number of very, very bad queries from hitting your database. Go update your code, ASAP.

We've made some pretty significant progress towards scaling Twitter, and we're now at the point where the majority of requests that hit our site complete in less than 70 ms (mostly API requests), and the really complicated front-end pages that we display complete in less than 160 ms. There are still a lot of hiccups, so the average is higher than that, but we're constantly working on getting it down.

One of the consistent problems we've been facing is errant queries. We've been seeing (off and on) queries like:

SELECT * FROM statuses WHERE user_id = 234223 ORDER BY created_at

If you know anything about relational databases, this is a very bad thing, especially when you have users that have more than 20,000 statuses.

One major downside of having an object-relational mapper is that you don't always control what goes on behind the scenes. In tracking down this problem, first we investigated all our code, and weren't able to find the source of these problems. Switching tactics, we isolated some test cases that replicated the problem and brought out the big guns: print. This pretty quickly brought us to an obscure corner of the ActiveRecord source (three cheers for source code!), where it became apparent that Rails was doing these gigantic loads from the database every time we saved even a single field in a related object. There are a bunch of mitigating circumstances that mean that this bug doesn't get triggered all the time, but it's still really really bad.

Thankfully, the patch ~~will be committed soon~~ has been committed (32 minutes patch-to-commit!), and no-one will have to deal with, as Coda put it: "Arg stabby stab stab stabbity fuck stab" anymore. The fact that no-one noticed really speaks to how freaking awesome relational databases (in our case, MySQL) are these days.

Perhaps underlying all of this is the simple fact that most of the time, ActiveRecord and Rails in general is pretty solid, and Ruby underneath is a fully sound language with which to build high-volume services. Kevin over at PowerSet has more on the topic - they've recently announced that they'll be doing their front-end development in Ruby (up until now, it's just been a glue language internally).

Twitter at RailsConf

2007-05-18T11:30:00.000+01:00

Sadly, I won't be attending RailsConf in Portland this weekend. I'll be finishing up at XTech, followed by a (well-deserved) week relaxing / adventuring in Morocco. Photos will follow.

In the meantime, if you're one of the lucky ~~few~~ many attending RailsConf, Alex and Britt will be speaking 10:45 AM on Sunday. They may or may not being doing a reprise of my Scaling Twitter talk, but I'm sure it will be fantastic in any event.

Wish I were there, have fun all!

Social Software for Robots Slides Up

2007-05-17T09:58:00.000+01:00

I've uploaded the Social Software for Robots slides for the messaging/jabber talk that Kellan and I gave yesterday at XTech 2007.

I think the talk went well, aside from a hiccup getting all the stars aligned to show the visualization that Tom helped us put together. For those who missed it, I'm hoping that Kellan or I or both will give reprises at other upcoming conferences. There's a lot of potential here, and things are finally reaching the point where real-time APIs are not only becoming a reality, but a necessity.

XTech has been great fun so far, and there are a number of talks that I'm looking forward to today and tomorrow. Huge thanks to Edd and everyone else who's helped put it together.

Slides for Scaling Twitter talk, XTech next up.

2007-04-22T20:49:00.000+01:00

The slides from the talk are available on SlideShare. They don't spell out the talk, so if you have questions, please do ask.

I'm really excited about XTech, coming up in a few weeks (May 15th-18th) in Paris. I'll be speaking with Kellan about using Jabber to build Social Software for Robots. Lots of awesome people and talks there, should be a lot of fun.

Scaling Twitter, The Talk.

2007-04-13T20:51:00.000+01:00

Simon Willison linked to an interview with Alex Payne, one of my co-workers on Twitter. This caused a bit of a stir, so apparently there's some interest in our experience scaling Twitter, and Rails.

We've been extremely happy with Rails, and make use of the multitude of helpers that it offers us - like any application on any stack, though, providing fast response times to a (rapidly) growing number of users is a challenge. The solutions are often tightly coupled to the application and its characteristics, and while scaling the most trafficked Rails site in the world, we've run into situations where existing solutions weren't enough.

This process has led us to build a number of tools that help us deal with our load, and just as soon as we find some spare time, we'll be releasing many of them. In the meantime, you can find out first what sorts of challenges we've encountered and solutions we've come up with at my talk at the SDForum Silicon Valley Ruby Conference next weekend (April 21-22nd).

I'll be focusing on ActiveRecord and database optimization, caching, and of course, Messaging. I'll also touch on some areas where we haven't had great successes (yet), and hopefully someone from the audience will shout out that there's some totally obvious and awesome thing that we haven't thought of, and it'll save us weeks of work (no, I'm serious. Does someone want to take bets?).

MapReduce in 36 lines of Ruby

2007-04-01T19:48:00.000+01:00

This has been burning a hole in my head since August, after Joel's post made it blindingly obvious that Ruby is the perfect language for distributed programming. I have some code that properly implements partitioning, etc, but never got around to finishing it sufficiently for a proper release. Here's the core idea; if anyone wants the partitioning code, ping me at romeda@gmail.com. mapreduce_enumerable.rb:


require 'rubygems'
require 'ringy_dingy'
require 'ruby2ruby'

module Enumerable
  def dmap(&block)
    self.each_with_index do |element,idx|
      ring_server.write([:dmap, Process.pid, block.to_ruby, element, idx])
    end

    results = []
    while results.size < self.size
      result, idx = ring_server.take([:dmap, Process.pid, nil, nil]).last(2)
      results[idx] = result
    end

    results
  end

  def ring_server
    return @ring_server if @ring_server

    ringy_dingy = RingyDingy.new nil
    @ring_server = ringy_dingy.ring_server
  end
end

mapreduce_runner.rb:


require 'rubygems'
require 'ruby2ruby'
require 'ringy_dingy'

ringy_dingy = RingyDingy.new nil
ring_server = ringy_dingy.ring_server

loop do
  pid, block, element, idx = ring_server.take([:dmap, nil, nil, nil, nil]).last(4)
  begin
    result = eval(block).call(element)
  rescue Object => err
    result = err
  end
  puts "Got #{result} from #{element} for #{pid}."
  ring_server.write([:dmap, pid, result, idx])
end

From the shell:

$ sudo gem install RingyDingy
$ sudo gem install ruby2ruby
$ ring_server &
$ ruby mapreduce_runner &
$ ruby mapreduce_runner &

From irb:


> require 'mapreduce_enumerable'
> (1..100).to_a.dmap { |v| v * 2 }

Google Code Love

2006-12-22T21:32:00.001Z

Google Code Love
Originally uploaded by liminalists.

My Jabber::Simple project, extracted from Twitter's Jabber/IM support, got some love today from Google Code (thanks, eric!)

The Year in Cities, 2006.

2006-12-22T00:06:00.000Z

Here's the list of cities I stayed in during the course of 2006. Memes away. I've only included cities in which I spent at least one day and one night. I've preemptively added a few spots I'll be visiting over Christmas, too.

San Francisco, CA
Fresno (FresYES!), CA
Tahoe, CA/NV
Seattle, WA
Vancouver, BC
White Rock, BC
Victoria, BC
Whistler, BC
Reykjavik, Iceland
Ólafsvík, Iceland
Belfast, Northern Ireland
Aberystwyth, Wales
Norwich, East Anglia
London, England
Lewes (Brighton & Hove), East Sussex
Barcelona, Spain
Portland, OR
Grande Prairie, AB
Santa Cruz, CA
Mojave Desert & Las Vegas, CA & NV
Zion National Monument, UT
Tucson, AZ
[Near] Calistoga, CA
Chilliwack, BC

All in all, pretty good, though I'm hoping for maybe somewhere more less-European for 2007. We'll see.

The Weather, by Twitter.

2006-12-20T21:12:00.000Z

A few weeks ago, Kellan and I whipped up what is tentatively known as "WeatherBot". It's a simple tool to provide weather updates to Twitter users. Every morning at 7:00 AM and 3:45 PM (local time, that's 07h00 and 15h45 for those of you across the pond), our fleet of WeatherBots will send a weather update to Twitter.

If you follow these bots, you'll receive those updates wherever you normally get your Twitters; IM, Phone, RSS, or just on the web. So far, we have bots for the following cities (links are to the bots' twitter profiles, for easy friending action): Boston, Brighton, Chicago, Helsinki, London, Los Angeles, New York, Paris, Portland, San Francisco, Seattle, Singapore, and Vancouver. If you'd like to see another city, just ask and we'll provide.

The source code is available over on Google Code as Twitter Weather. Kellan and I will be providing follow-up posts (and hopefully talks!) about how we built this, and updates on future improvements, so watch this space, and if you're not already on Twitter, sign up already! We're building an awesome space, and want you to have as much fun as we are.

On Twitter

2006-12-08T19:55:00.000Z

Kathy Sierra has a great post, talking about Continuous Partial Attention and Twitter. Before you read my response, go read Kellan's excellent post on the matter.

Twitter is really nothing new, it's just new to the web (and IM/SMS). IRC has been around for ages, and functions in much the same way. Campfire, 37signals' group chat application is a similar application, but more task-focused. So to say that 2006 is the year of the singularity is probably too much of an exaggeration.

I held off on using Twitter for a long time, even though I work at Obvious, because it was too much of an attention drain for me. I have a 5 year old Nokia cell phone, and the SMS experience is one of mind-numbing pain when you receive 15 or 20 SMSes per day (not to mention expensive!). IM, on the other hand, is seamless. Like Kellan, I have verging-on-sub-conscious Growl notifications for IM, and as such I spend less than a second processing each incoming Twitter message.

A phone call, on the other hand, can easily eat half an hour for me, because I don't "do" the phone very often. I find it intimidating, because most of the time the interactions involve long hold times, anti-human menus, horrible bureaucracy, etc. Email is more distracting than IM or Twitter, and so on. Calling friends and family is also a major time commitment, especially if we haven't talked in a while.

I'm more than willing to put up with certain levels of interruption in order to have that sense that I "know" what's going on around me. I have friends who live far away, and whom I don't speak to on the phone, email, or see regularly. Twitter perfectly fills the gap, and allows us to keep each-other "in the loop", bringing us closer overall.

Sure, I could close myself off, and just work all day - but I feel like all these emerging (emergent?) "ambient" communication technologies actually help to make me happier, and feel less alone in our highly abstract and disconnected world. Having "community" is extremely important; far more so than our productivity. Maybe it's just sad that we allow ourselves to live in this paradoxically disconnected online world, "cyberspace" as it were. I like to think of it as just another progression of human adaptation. To say that people are lonelier or happier, more or less fulfilled, busier, or more productive than they were 10, 50, 100, or 500 years ago is a falsehood. We're different, and we're all just trying to make the best of where we find ourselves.

Announcing Jabber::Simple

2006-11-09T20:14:00.000Z

Jabber::Simple [src, doc] is a simple (duh!) Ruby library that aims to make the implementation of basic Jabber functionality trivial. It is an extraction of the Jabber support that was added to Twitter, and is released under the GPL by Obvious. A line of code is worth a thousand words, so here is the complete code for sending a simple message to a Jabber user:


jabber = Jabber::Simple.new('rex@friendosaurus.com', 'password')
jabber.deliver("bront@friendosaurus.com", "Hey! I'm thinking of going Vegetarian - Any suggestions?")

Getting incoming messages is just as easy:


jabber.received_messages do |msg|
  puts "#{msg.body}" if msg.type == :chat
end

You can also set your status, and get information about your friends' statuses:


jabber.status(:away, "Eating at the Tree Cafe. I need a ladder.")

jabber.presence_updates do |update|
  friend = update[0]
  presence = update[2]
  puts "#{friend.jid} is #{presence.status}"
end

Installation


sudo gem install xmpp4r-simple

or download the package from RubyForge. Source code is also available, licensed under the GPLv2.

Yet Another Jabber Library?

There are a number of existing Jabber libraries for Ruby (jabber4r, xmpp4r, and Net::XMPP), so why Jabber::Simple? First off, Jabber::Simple does not aim to replicate any core XMPP protocol functionality present in these libraries — in fact, Jabber::Simple depends on xmpp4r and the Jabber::Simple#client and Jabber::Simple#roster methods expose all of xmpp4r's awesome functionality. When I started building in Jabber support for Twitter I'd used various Jabber clients, and even set up a simple Jabber server. Writing my own client, however, was a bit more complex. It turns out that the seamless experience of "adding a friend" and chatting with them is (unsurprisingly) comprised of a series of disjoint steps, and fraught with the peril of threads, XML streams, and arcane magic. The available libraries handle these tasks and many more admirably, but lack in elegance. My hope is that Jabber::Simple provides a sufficiently obvious interface with which to develop tools that use the Jabber protocol.

But Wait! There's More!

Now, you might shy away from writing that really cool chat-bot you've been meaning to write, saying "Wow, this is great, but setting up a Jabber server is a pain." --- but fear not! Go over to Google Talk and sign up for an account. Once you're done, use your Google Talk username and password, and start Jabbering. No really, it's that simple.


jabber = Jabber::Simple.new("you@gmail.com", "password")