Category: ideas

Orchestrating streams of data from across the Internet

The liveblog was a revelation for us at the Guardian. The sports desk had been doing them for years experimenting with different styles, methods and tone. And then about 3 years ago the news desk started using them liberally to great effect.

I think it was Matt Wells who suggested that perhaps the liveblog was *the* network-native format for news. I think that’s nearly right…though it’s less the ‘format’ of a liveblog than the activity powering the page that demonstrates where news editing in a networked world is going.

It’s about orchestrating the streams of data flowing across the Internet into a compelling use in one form or another. One way to render that data is the liveblog. Another is a map with placemarks. Another is a RSS feed. A stream of tweets. Storify. Etc.

I’m not talking about Big Data for news. There is certainly a very hairy challenge in big data investigations and intelligent data visualizations to give meaning to complex statistics and databases. But this is different.

I’m talking about telling stories by playing DJ to the beat of human observation pumping across the network.

We’re working on one such experiment with a location-tagging tool we call FeedWax. It creates location-aware streams of data for you by looking across various media sources including Twitter, Instagram, YouTube, Google News, Daylife, etc.

The idea with FeedWax is to unify various types of data through shared contexts, beginning with location. These sources may only have a keyword to join them up or perhaps nothing at all, but when you add location they may begin sharing important meaning and relevance. The context of space and time is natural connective tissue, particularly when the words people use to describe something may vary.

We’ve been conducting experiments in orchestrated stream-based and map-based storytelling on n0tice for a while now. When you start crafting the inputs with tools like FeedWax you have what feels like a more frictionless mechanism for steering the flood of data that comes across Twitter, Instagram, Flickr, etc. into something interesting.

For example, when the space shuttle Endeavour flew its last flight and subsequently labored through the streets of LA there was no shortage of coverage from on-the-ground citizen reporters. I’d bet not one of them considered themselves a citizen reporter. They were just trying to get a photo of this awesome sight and share it, perhaps getting some acknowledgement in the process.

You can see the stream of images and tweets here: http://n0tice.com/search?q=endeavor+OR+endeavour. And you can see them all plotted on a map here: http://goo.gl/maps/osh8T.

Interestingly, the location of the photos gives you a very clear picture of the flight path. This is crowdmapping without requiring that anyone do anything they wouldn’t already do. It’s orchestrating streams that already exist.

This behavior isn’t exclusive to on-the-ground reporting. I’ve got a list of similar types of activities in a blog post here which includes task-based reporting like the search for computer scientist Jim Gray, the use of Ushahidi during the Haiti earthquake, the Guardian’s MPs Expenses project, etc. It’s also interesting to see how people like Jon Udell approach this problem with other data streams out there such as event and venue calendars.

Sometimes people refer to the art of code and code-as-art. What I see in my mind when I hear people say that is a giant global canvas in the form of a connected network, rivers of different colored paints in the form of data streams, and a range of paint brushes and paint strokes in the form of software and hardware.

The savvy editors in today’s world are learning from and working with these artists, using their tools and techniques to tease out the right mix of streams to tell stories that people care about. There’s no lack of material or tools to work with. Becoming network-native sometimes just means looking at the world through a different lens.

Mobilising the web of feeds

I wrote this piece for the Guardian’s Media Network on the role that RSS could play now that the social platforms are becoming more difficult to work with. GeoRSS, in particular, has a lot of potential given the mobile device explosion. I’m not suggesting necessarily that RSS is the answer, but it is something that a lot of people already understand and could help unify the discussion around sharing geotagged information feeds.

Powered by Guardian.co.ukThis article titled “Mobilising the web of feeds” was written by Matt McAlister, for theguardian.com on Monday 10th September 2012 16.43 UTC

While the news that Twitter will no longer support RSS was not really surprising, it was a bit annoying. It served as yet another reminder that the Twitter-as-open-message-utility idea that many early adopters of the service loved was in fact going away.

There are already several projects intending to disrupt Twitter, mostly focused on the idea of a distributed, federated messaging standard and/or platform. But we already have such a service: an open standard adopted by millions of sources; a federated network of all kinds of interesting, useful and entertaining data feeds published in real-time. It’s called RSS.

There was a time when nearly every website was RSS-enabled, and a cacophony of Silicon Valley startups fought to own pieces of this new landscape, hoping to find dotcom gold. But RSS didn’t lead to gold, and most people stopped doing anything with it.

Nobody found an effective advertising or service model (except, ironically, Dick Costolo, CEO of Twitter, who sold Feedburner to Google). The end-user market for RSS reading never took off. Media organisations didn’t fully buy into it, and the standard took a backseat to more robust technologies.

Twitter is still very open in many ways and encourages technology partners to use the Twitter API. That model gives the company much more control over who is able to use tweets outside of the Twitter owned apps, and it’s a more obvious commercial strategy that many have been asking Twitter to work on for a long time now.

But I think we’ve all made a mistake in the media world by turning our backs on RSS. It’s understandable why it happened. But hopefully those who rejected RSS in the past will see the signals demonstrating that an open feed network is a sensible thing to embrace today.

Let’s zoom out for context first. Looking at the macro trends in the internet’s evolution, we can see one or two clear winners as more information and more people appeared on the network in waves over the last 15 years.

Following the initial explosion of new domains, Yahoo! solved the need to surface only the websites that mattered through browsing. The Yahoo! directory became saturated, so Google then surfaced pages that mattered within those websites through searches. Google became saturated, so Facebook and Twitter surfaced things that mattered that live on the webpages within those web sites through connecting with people.

Now that the social filter is saturated, what will be used next to surface things that matter out of all the noise? The answer is location. It is well understood technically. The software-hardware-service stack is done. The user experience is great. We’re already there, right?

No – most media organisations still haven’t caught up yet. There’s a ton of information not yet optimised for this new view of the world and much more yet to be created. This is just the beginning.

Do we want a single platform to be created that catalyses the location filter of the internet and mediates who sees what and when? Or do we want to secure forever a neutral environment where all can participate openly and equally?

If the first option happens, as historically has been the case, then I hope that position is taken by a force that exists because of and reliant on the second option.

What can a media company do to help make that happen? The answer is to mobilise your feeds. As a publisher, being part of the wider network used to mean having a website on a domain that Yahoo! could categorise. Then it meant having webpages on that website optimised for search terms people were using to find things via Google. And more recently it has meant providing sharing hooks that can spread things from those pages on that site from person to person.

Being part of the wider network today suddenly means all of those things above, and, additionally, being location-enabled for location-aware services.

It doesn’t just mean offering a location-specific version of your brand, though that is certainly an important thing to do as well. The major dotcoms use this strategy increasingly across their portfolios, and I’m surprised more publishers don’t do this.

More importantly, though, and this is where it matters in the long run, it means offering location-enabled feeds that everyone can use in order to be relevant in all mobile clients, applications and utilities.

Entrepreneurs are all over this space already. Pure-play location-based apps can be interesting, but many feel very shallow without useful information. The iTunes store is full of travel apps, reference apps, news, sports, utilities and so on that are location-aware, but they are missing some of the depth that you can get on blogs and larger publishers’ sites. They need your feeds.

Some folks have been experimenting in some very interesting ways that demonstrate what is possible with location-enabled feeds. Several mobile services, such as FlipBoard, Pulse and now Prismatic, have really nice and very popular mobile reading apps that all pull RSS feeds, and they are well placed to turn those into location-based news services.

Perhaps a more instructive example of the potential is the augmented reality app hypARlocal at Talk About Local. They are getting location-aware content out of geoRSS feeds published by hyperlocal bloggers around the UK and the citizen journalism platform n0tice.com.

But it’s not just the entrepreneurs that want your location-enabled feeds. Google Now for Android notifies you of local weather and sports scores along with bus times and other local data, and Google Glasses will be dependent on quality location-specific data as well.

Of course, the innovations come with new revenue models that could get big for media organisations. They include direct, advertising, and syndication models, to name a few, but have a look at some of the startups in the rather dense ‘location‘ category on Crunchbase to find commercial innovations too.

Again, this isn’t a new space. Not only has the location stack been well formed, but there are also a number of bloggers who have been evangelising location feeds for years. They already use WordPress, which automatically pumps out RSS. And many of them also geotag their posts today using one of the many useful WordPress mapping plugins.

It would take very little to reinvigorate a movement around open location-based feeds. I wouldn’t be surprised to see Google prioritising geotagged posts in search results, for example. That would probably make Google’s search on mobile devices much more compelling, anyhow.

Many publishers and app developers, large and small, have complained that the social platforms are breaking their promises and closing down access, becoming enemies of the open internet and being difficult to work with. The federated messaging network is being killed off, they say. Maybe it’s just now being born.

Media organisations need to look again at RSS, open APIs, geotagging, open licensing, and better ways of collaborating. You may have abandoned it in the past, but RSS would have you back in a heartbeat. And if RSS is insufficient then any location-aware API standard could be the meeting place where we rebuild the open internet together.

It won’t solve all your problems, but it could certainly solve a few, including new revenue streams. And it’s conceivable that critical mass around open location-based feeds would mean that the internet becomes a stronger force for us all, protected from nascent platforms whose their future selves may not share the same vision that got them off the ground in the first place.

To get more articles like this sent direct to your inbox, sign up for free membership to the Guardian Media Network. This content is brought to you by Guardian Professional.


Ads by The Guardian

guardian.co.uk © Guardian News & Media Limited 2010

Published via the Guardian News Feed plugin for WordPress.

Calling your web site a ‘property’ deprives it of something bigger

BBC offered another history of London documentary the other night, a sort of people’s perspective on how the character of the city has changed over time, obviously inspired by Danny Boyle’s Opening Ceremony at the Olympics.

Some of the sequences were interesting to me particularly as a foreigner – the gentrification of Islington, the anarchist squatters in Camden, the urbanization of the Docklands, etc.  – a running theme of haves vs have-nots.

It’s one of a collection of things inspiring me recently including a book called ‘The Return of the Public‘ by Dan Hind, a sort of extension to the Dewey v Lippman debates, what’s going on with n0tice, such as Sarah Hartley’s adaptation for it called Protest Near You and the dispatch-o-rama hack, and, of course, the Olympics.

I’m becoming reinvigorated and more bullish on where collective action can take us.

At a more macro level these things remind me of the need to challenge the many human constructs and institutions that are reflections of the natural desire to claim things and own them.

Why is it so difficult to embrace a more ‘share and share alike’ attitude?  This is as true for children and their toys as it is for governments and their policies.

The bigger concern for me, of course, is the future of the Internet and how media and journalism thrive and evolve there.

Despite attempts by its founders to shape the Internet so it can’t be owned and controlled, there are many who have tried to change that both intentionally and unwittingly, occasionally with considerable success.

How does this happen?

We’re all complicit.  We buy a domain. We then own it and build a web site on it. That “property” then becomes a thing we use to make money.  We fight to get people there and sell them things when they arrive.  It’s the Internet-as-retailer or Internet-as-distributor view of the world.

That’s how business on the Internet works…or is it?

While many have made that model work for them, it’s my belief that the property model is never going to be as important or meaningful or possibly as lucrative as the platform or service model over time. More specifically, I’m talking about generative media networks.

Here are a few different ways of visualizing this shift in perspective (more):

Even if it works commercially, the property model is always going to be in conflict with the Internet-as-public-utility view of the world.

Much like Britain’s privately owned public spaces issue, many worry that the Internet-as-public-utility will be ruined or, worse, taken from us over time by commercial and government interests.

Playing a zero sum game like that turns everyone and everything into a threat.  Companies can be very effective at fighting and defending their interests even if the people within those companies mean well.

I’m an optimist in this regard.  There may be a pendulum that swings between “own” and “share”, and there are always going to be fights to secure public spaces.  But you can’t put the Internet genie back in the bottle.  And even if you could it would appear somewhere else in another form just as quickly…in some ways it already has.

The smart money, in my mind, is where many interests are joined up regardless of their individual goals, embracing the existence of each other in order to benefit from each other’s successes.

The answer is about cooperation, co-dependency, mutualisation, openness, etc.

We think about this a lot at the Guardian. I recently wrote about how it applies to the recent Twitter issues here. And this presentation by Chris Thorpe below from back in 2009 on how to apply it to the news business is wonderful:

Of course, Alan Rusbridger’s description of a mutualised newspaper in this video is still one of the strongest visions I’ve heard for a collaborative approach to media.

The possibility of collective action at such an incredible scale is what makes the Internet so great.  If we can focus on making collective activities more fruitful for everyone then our problems will become less about haves and have-nots and more about ensuring that everyone participates.

That won’t be an easy thing to tackle, but it would be a great problem to have.

Interesting perspectives from Web 2.0 Expo

Today’s Web 2.0 Expo in San Francisco provided some really good brain food.

Clay Shirky’s keynote was excellent. He talked about architecting a new world for the “cognitive surplus” that’s emerging as people pull themselves out of the historical sitcom hangover and invest their energy online. Matt Jones and Tom Coates shared some neat ideas on design for personal infomatics. And Twitter’s Alex Payne and Michael Migurski of Stamen Design presented learnings from the perspective of an API provider.

One little nugget I really liked was a minor point Migurski made when talking through the Oakland Crimespotting service. He noted that there are several standard formats commonly provided by most web services including HTML, JSON, serliazed PHP, RSS and XML.

But we often forget about simple Excel spreadsheets.

He showed how the Oakland Crimespotting site offers downloadable Excel spreadsheets detailing recent activity from particular police beats, for example.

One of the keys to opening up government data is making the case to the people who are best equipped to provide raw data that it needs to be posted directly to the Internet. Telling them they need to output JSON for data visualizations and mashups will do as much good as a slap in the face. Showing them a regularly updating Excel spreadsheet that is findable on a web page that they can email to their colleagues, friends and families is going to get them thinking differently and perhaps encourage their participation directly.

The crime data issue is going to be a big deal in the not too distant future, I’m sure. And as Mr. Coates and Mr. Jones noted in their talks on personal data design, it’s the details that really matter in this space. You can think about products and features all day, but the specifics that define how data is shared, how it becomes relevant and how it is presented will make or break the intent of any offering.

Designing Your API, Web 2.0 Expo 2008:

The useful convergence of data

I have only one prediction for 2008. I think we’re finally about to see the useful combination of the 4 W’s – Who, What, Where, and When.

Marc Davis has done some interesting research in this area at Yahoo!, and Bradley Horowitz articulated how he sees the future of this space unfolding in a BBC article in June ’07:

“We do a great job as a culture of “when”. Using GMT I can say this particular moment in time and we have a great consensus about what that means…We also do a very good job of “where” – with GPS we have latitude and longitude and can specify a precise location on the planet…The remaining two Ws – we are not doing a great job of.”

I’d argue that the social networks are now really honing in on “who”, and despite having few open standards for “what” data (other than UPC) there is no shortage of “what” data amongst all the “what” providers. Every product vendor has their own version of a product identifier or serial number (such as Amazon’s ASIN, for example).

We’ve seen a lot of online services solving problems in these areas either by isolating specific pieces of data or combining the data in specific ways. But nobody has yet integrated all 4 in a meaningful way.


Jeff Jarvis’ insightful post on social airlines starts to show how these concepts might form in all kinds of markets. When you’re traveling it makes a lot of sense to tap into “who” data to create compelling experiences that will benefit everyone:

“

  • At the simplest level, we could connect while in the air to set up shared cab rides once we land, saving passengers a fortune.
  • We can ask our fellow passengers who live in or frequently visit a destination for their recommendations for restaurants, things to do, ways to get around.
  • We can play games.
  • What if you chose to fly on one airline vs. another because you knew and liked the people better? What if the airline’s brand became its passengers?
  • Imagine if on this onboard social network, you could find people you want to meet – people in the same business going to the same conference, people of similar interests, future husbands and wives – and you can rendezvous in the lounge.
  • The airline can set up an auction marketplace for at least some of the seats: What’s it worth for you to fly to Berlin next Wednesday?

“

Carrying the theme to retail markets, you can imagine that you will walk into H&M and discover that one of your first-degree contacts recently bought the same shirt you were about to purchase. You buy a different one instead. Or people who usually buy the same hair conditioner as you at the Walgreen’s you’re in now are switching to a different hair conditioner this month. Though this wouldn’t help someone like me who has no hair to condition.

Similarly, you can imagine that marketing messages could actually become useful in addition to being relevant. If CostCo would tell me which of the products I often buy are on sale as I’m shopping, or which of the products I’m likely to need given what they know about how much I buy of what and when, then my loyalty there is going to shoot through the roof. They may even be able to identify that I’m likely buying milk elsewhere and give me a one-time coupon for CostCo milk.

Bradley sees it playing out on the phone, too:

“On my phone I see prices for a can of soup in my neighbourhood. It resolves not only that particular can of soup but knows who I am, where I am and where I live and helps me make an intelligent decision about whether or not it is a fair price.

It has to be transparent and it has to be easy because I am not going to invest a lot of effort or time to save 13 cents.”

It may be unrealistic to expect that this trend will explode in 2008, but I expect it to at least appear in a number of places and inspire future implementations as a result. What I’m sure we will see in 2008 is dramatic growth in the behind-the-scenes work that will make this happen, such as the development and customization of CRM-like systems.

Lots of companies have danced around these ideas for years, but I think the ideas and the technologies are finally ready to create something real, something very powerful.

Photo: SophieMuc

The Internet’s secret sauce: surfacing coincidence

What is it that makes my favorite online services so compelling? I’m talking about the whole family of services that includes Dopplr, Wesabe, Twitter, Flickr, and del.icio.us among others.

I find it interesting that people don’t generally refer to any of these as “web sites”. They are “services”.

I was fortunate enough to spend some time with Dopplr’s Matt Biddulph and Matt Jones last week while in London where they described the architecture of what they’ve built in terms of connected data keys. The job of Dopplr, Mr. Jones said, was to “surface coincidence”.

I think that term slipped out accidentally, but I love it. What does it mean to “surface coincidence”?

It starts by enabling people to manufacture the circumstances by which coincidence becomes at least meaningful if not actually useful. Or, as Jon Udell put it years ago now when comparing Internet data signals to cellular biology:

“It looks like serendipity, and in a way it is, but it’s manufactured serendipity.”

All these services allow me to manage fragments of my life without requiring burdensome tasks. They all let me take my data wherever I want. They all enhance my data by connecting it to more data. They all make my data relevant in the context of a larger community.

When my life fragments are managed by an intelligent service, then that service can make observations about my data on my behalf.

Dopplr can show me when a distant friend will be near and vice versa. Twitter can show me what my friends are doing right now. Wesabe can show me what others have learned about saving money at the places where I spend my money. Among many other things Flickr can show me how to look differently at the things I see when I take photos. And del.icio.us can show me things that my friends are reading every day.

There are many many behaviors both implicit and explicit that could be managed using this formula or what is starting to look like a successful formula, anyhow. Someone could capture, manage and enhance the things that I find funny, the things I hate, the things at home I’m trying to get rid of, the things I accomplished at work today, the political issues I support, etc.

But just collecting, managing and enhancing my life fragments isn’t enough. And I think what Matt Jones said is a really important part of how you make data come to life.

You can make information accessible and even fun. You can make the vast pool feel manageable and usable. You can make people feel connected.

And when you can create meaning in people’s lives, you create deep loyalty. That loyalty can be the foundation of larger businesses powered by advertising or subscriptions or affiliate networks or whatever.

The result of surfacing coincidence is a meaningful action. And those actions are where business value is created.

Wikipedia defines coincidence as follows:

“Coincidence is the noteworthy alignment of two or more events or circumstances without obvious causal connection.”

This is, of course, similar and related to the definition of serendipity:

“Serendipity is the effect by which one accidentally discovers something fortunate, especially while looking for something else entirely.”

You might say that this is a criteria against which any new online service should be measured. Though it’s probably so core to getting things right that every other consideration in building a new online service needs to support it.

It’s probably THE criteria.

The business of network effects

The Internet platform business has some unique challenges. It’s very tempting to adopt known models to make sense of it, like the PC business, for example, and think of the Internet platform like an operating system.

The similarities are hard to deny, and who wouldn’t want to control the operating system of the Internet?

In 2005, Jason Kottke proposed a vision for the “WebOS” where users could control their experience with tools that leveraged a combination of local storage and a local server, networked services and rich clients.

“Applications developed for this hypothetical platform have some powerful advantages. Because they run in a Web browser, these applications are cross platform, just like Web apps such as Gmail, Basecamp, and Salesforce.com. You don’t need to be on a specific machine with a specific OS…you just need a browser + local Web server to access your favorite data and apps.”

Prior to that post, Nick Carr offered a view on the role of the browser that surely resonated with the OS perspective for the Internet:

“Forget the traditional user interface. The looming battle in the information technology business is over control of the utility interface…Control over the utility interface will provide an IT vendor with the kind of power that Microsoft has long held through its control of the PC user interface.”

He also responded later to Kottke’s vision saying that the reliance on local web and storage services on a user’s PC may be unnecessary:

“Your personal desktop, residing entirely on a distant server, will be easily accessible from any device wherever you go. Personal computing will have broken free of the personal computer.”

But the client layer is merely a piece of the much larger puzzle, in my opinon.

Dare Obasanjo more recently broke down the different ideas of what “Cloud OS” might mean:

“I think it is a good idea for people to have a clear idea of what they are talking about when they throw around terms like “cloud OS” or “cloud platform” so we don’t end up with another useless term like SOA which means a different thing to each person who talks about it. Below are the three main ideas people often identify as a “Web OS”, “cloud OS” or “cloud platform” and examples of companies executing on that vision.”

He defines them as follows:

  1. WIMP Desktop Environment Implemented as a Rich Internet Application (The YouOS Strategy)
  2. Platform for Building Web-based Applications (The Amazon Strategy)
  3. Web-based Applications and APIs for Integrating with Them (The Google Strategy)

The OS metaphor has lots of powerful implications for business models, as we’ve seen on the PC. The operating system in a PC controls all the connections from the application user experience through the filesystem down through the computer hardware itself out to the interaction with peripheral services. Being the omniscient hub makes the operating system a very effective taxman for every service in the stack. And from there, the revenue streams become very easy to enable and enforce.

But the OS metaphor implies a command-and-control dynamic that doesn’t really work in a global network controlled only by protocols.

Internet software and media businesses don’t have an equivilent choke point. There’s no single processor or function or service that controls the Internet experience. There’s no one technology or one company that owns distribution.

There are lots of stacks that do have choke points on the Internet. And there are choke points that have tremendous value and leverage. Some are built purely and intentionally on top of a distribution point such as the iPod on iTunes, for example.

But no single distribution center touches all the points in any stack. The Internet business is fundamentally made of data vectors, not operational stacks.

Jeremy Zawodny shed light on this concept for me using building construction analogies.

He noted that my building contractor doesn’t exclusively buy Makita or DeWalt or Ryobi tools, though some tools make more sense in bundles. He buys the tool that is best for the job and what he needs.

My contractor doesn’t employ plumbers, roofers and electricians himself. Rather he maintains a network of favorite providers who will serve different needs on different jobs.

He provides value to me as an experienced distribution and aggregation point, but I am not exclusively tied to using him for everything I want to do with my house, either.

Similarly, the Internet market is a network of services. The trick to understanding what the business model looks like is figuring out how to open and connect services in ways that add value to the business.

In a precient viewpoint from 2002 about the Internet platform business, Tim O’Reilly explained why a company that has a large and valuable data store should open it up to the wider network:

“If they don’t ride the horse in the direction it’s going, it will run away from them. The companies that “grasp the nettle firmly” (as my English mother likes to say) will reap the benefits of greater control over their future than those who simply wait for events to overtake them.

There are a number of ways for a company to get benefits out of providing data to remote programmers:

Revenue. The brute force approach imposes costs both on the company whose data is being spidered and on the company doing the spidering. A simple API that makes the operation faster and more efficient is worth money. What’s more, it opens up whole new markets. Amazon-powered library catalogs anyone?

Branding. A company that provides data to remote programmers can request branding as a condition of the service.

Platform lock in. As Microsoft has demonstrated time and time again, a platform strategy beats an application strategy every time. Once you become part of the platform that other applications rely on, you are a key part of the computing infrastructure, and very difficult to dislodge. The companies that knowingly take their data assets and make them indispensable to developers will cement their role as a key part of the computing infrastructure.

Goodwill. Especially in the fast-moving high-tech industry, the “coolness” factor can make a huge difference both in attracting customers and in attracting the best staff.”

That doesn’t clearly translate into traditional business models necessarily, but if you look at key business breakthroughs in the past, the picture today becomes more clear.

  1. The first breakthrough business model was based around page views. The domain created an Apple-like controlled container. Exposure to eyeballs was sold by the thousands per domain. All the software and content was owned and operated by the domain owner, except the user’s browser. All you needed was to get and keep eyeballs on your domain.
  2. The second breakthrough business model emerged out of innovations in distribution. By building a powerful distribution center and direct connections with the user experience, advertising could be sold both where people began their online experiences and at the various independent domain stacks where they landed. Inventory beget spending beget redistribution beget inventory…it started to look a lot like network effects as it matured.
  3. The third breakthrough business model seems to be a riff on its predecessors and looks less and less like an operating system. The next breakthrough is network effects.

Network EffectsNetwork effects happen when the value of the entire network increases with each node added to the network. The telephone is the classic example, where every telephone becomes more valuable with each new phone in the network.

This is in contrast to TVs which don’t care or even notice if more TVs plug in.

Recommendation engines are the ultimate network effect lubricator. The more people shop at Amazon, the better their recommendation engine gets…which, in turn, helps people buy more stuff at Amazon.

Network effects are built around unique and useful nodes with transparent and highly accessible connection points. Social networks are a good example because they use a person’s profile as a node and a person’s email address as a connection point.

Network effects can be built around other things like keyword-tagged URLs (del.icio.us), shared photos (flickr), songs played (last.fm), news items about locations (outside.in).

The contribution of each data point wherever that may happen makes the aggregate pool more valuable. And as long as there are obvious and open ways for those data points to talk to each other and other systems, then network effects are enabled.

Launching successful network effect businesses is no easy task. The value a participant can extract from the network must be higher than the cost of adding a node in the network. The network’s purpose and its output must be indespensible to the node creators.

Massively distributed network effects require some unique characteristics to form. Value not only has to build with each new node, but the value of each node needs to increase as it gets leveraged in other ways in the network.

For example, my email address has become an enabler around the Internet. Every site that requires a login is going to capture my email address. And as I build a relationship with those sites, my email address becomes increasingly important to me. Not only is having an email address adding value to the entire network of email addresses, but the value of my email address increases for me with each service that is able to leverage my investment in my email address.

Then the core services built around my email address start to increase in value, too.

For example, when I turned on my iPhone and discovered that my Yahoo! Address Book was automatically cooked right in without any manual importing, I suddenly realized that my Yahoo! Address Book has been a constant in my life ever since I got my first Yahoo! email address back in the ’90’s. I haven’t kept it current, but it has followed me from job to job in a way that Outlook has never been able to do.

My Yahoo! Address Book is becoming more and more valuable to me. And my iPhone is more compelling because of my investment in my email address and my address book.

Now, if the network was an operating system, there would be taxes to pay. Apple would have to pay a tax for accessing my address book, and I would have to pay a tax to keep my address book at Yahoo!. Nobody wins in that scenario.

User data needs to be open and accessible in meaningful ways, and revenue needs to be built as a result of the effects of having open data rather than as a margin-based cost-control business.

But Dare Obasanjo insightfully exposes the flaw in reducing openness around identity to individual control alone:

“One of the bitter truths about “Web 2.0” is that your data isn’t all that interesting, our data on the other hand is very interesting…A lot of “Web 2.0″ websites provide value to their users via wisdom of the crowds appproaches such as tagging or recommendations which are simply not possible with a single user’s data set or with a small set of users.”

Clearly, one of the most successful revenue-driving opportunities in the networked economy is advertising. It makes sense that it would be since so many of the most powerful network effects are built on people’s profiles and their relationships with other people. No wonder advertisers can’t spend enough money online to reach their targets.

It will be interesting to see how some of the clever startups leveraging network effects such as Wesabe think about advertising.

Wesabe have built network effects around people’s spending behavior. As you track your finances and pull in your personal banking data, Wesabe makes loose connections between your transactions and other people who have made similar transactions. Each new person and each new transaction creates more value in the aggregate pool. You then discover other people who have advice about spending in ways that are highly relevant to you.

I’ve been a fan of Netflix for a long time now, but when Wesabe showed me that lots of Netflix customers were switching to Blockbuster, I had to investigate and before long decided to switch, too. Wesabe knew to advise me based on my purchasing behavior which is a much stronger indicator of my interests than my reading behavior.

Advertisers should be drooling at the prospects of reaching people on Wesabe. No doubt Netflix should encourage their loyal subscribers to use Wesabe, too.

The many explicit clues about my interests I leave around the Internet — my listening behavior at last.fm, my information needs I express in del.icio.us, my address book relationships, my purchasing behavior in Wesabe — are all incredibly fruitful data points that advertisers want access to.

And with managed distribution, a powerful ad platform could form around these explicit behaviors that can be loosely connected everywhere I go.

Netflix could automatically find me while I’m reading a movie review on a friend’s blog or even at The New York Times and offer me a discount to re-subscribe. I’m sure they would love to pay lots of money for an ad that was so precisely targeted.

That blogger and The New York Times would be happy share revenue back to the ad platform provider who enabled such precise targeting that resulted in higher payouts overall.

And I might actually come back to Netflix if I saw that ad. Who knows, I might even start paying more attention to ads if they started to find me rather than interrupt me.

This is why the Internet looks less and less like an operating system to me. Network effects look different to me in the way people participate in them and extract value from them, the way data and technologies connect to them, and the way markets and revenue streams build off of them.

Operating systems are about command-and-control distribution points, whereas network effects are about joining vectors to create leverage.

I know little about the mathematical nuances of chaos theory, but it offers some relevant philosophical approaches to understanding what network effects are about. Wikipedia addresses how chaos theory affects organizational development:

“Most of the focus on chaos theory is primarily rooted in the underlying patterns found in an otherwise chaotic enviornment, more specifically, concepts such as self-organization, bifurcation and self-similarity…

Self-organization, as opposed to natural or social selection, is a dynamic change within the organization where system changes are made by recalculating, re-inventing and modifying its structure in order to adapt, survive, grow and develop. Self-organization is the result of re-invention and creative adaptation due to the introduction of, or being in a constant state of, perturbed equilibrium.”

Yes, my PC is often in a state of ‘perturbed equilibrium’ but not because it wants to be.

How to fix building construction bureaucracy

Sometimes I forget to step outside of our little bubble here and see how people use or in fact don’t use the Internet. When I get that chance I often wonder if anything I’m doing in my career actually matters to anyone.

Usually, however, I’m reminded that even though the Internet isn’t weaved into every aspect of everything, it has great potential in places you might not consider.

For example, I’ve been remodelling my house to make room for a new little roommate due to be delivered in September. I’m trying to do most of the work myself or with help from friends and neighbors. I’m trying to save money, but I also really enjoy it. It’s a fantastic way to reconnect with the things that matter…food, shelter, love and life.

Well, I made the mistake of working without permits fully aware that I probably should have them. It’s my natural inclination to run around bureaucracy whenever possible.


As luck would have it, just as the pile of demolition debris on the sidewalk outside my house was at its worst, a building inspector happened to drive by on his way to another job. He asked to see my permit to which I replied, “The boss isn’t here. Can you come back later?”

The building inspector just laughed. After pleading a bit and failing, I started making calls to get drawings and to sort out the permits.

It was at this moment I realized how much building planning and construction could benefit from the advances made in the Internet market the last few years. The part of construction that people hate most is the one that is perhaps the most important. And it is this part that the Internet is incredibly well-suited to improve.

Admittedly, the permit process was not actually that painful and relatively cheap, too. I have spent in total maybe 1 day dealing with permits and drawings, so far, with a bit more to come, I’m sure.

But the desired effect of permitting jobs is sorely underserved by its process.

At the end of the day what you want is the highest building quality possible. You want builders using proven methods with at least semi-predictable outcomes. You want to make sure nobody gets hurt. And you want incentives for people to share expertise and information.

Rather than be a gatekeeper, the city needs to be an enabler.

One of the brochures I read called “How to Obtain a Permit” includes a whitelist of project types. I’m apparently allowed to put down carpets and hang things on my walls without a permit. Glad to know that.

Strangely, after explaining all the ways the city asserts itself into the process, on the very last page of the brochure it then says, “Remember, we are here to assist you. If you have any questions about your project, please give us a call!” I didn’t meet one person in the 6 queues I waded through the first morning who wanted to help me. They were mostly bored out of their brains.

Instead, the city should be putting that brainpower to work finding ways to lubricate conversation and collaboration around solving building problems. If the building community was in fact a community powered by thoughtful city-employed engineers, then I would be much more interested in working with them. I might even become dependent on them.

For example, if they helped me organize, store, print and even share my plans, then I’d be more than happy to let them keep my most current drawings, the actual plans I’m using to build with. If they could connect me to licensed contractors and certified service providers, I’d gladly give them my budget.

As it stands, my incentive is to avoid them and hide information whenever possible.

Imagine if I was able to submit a simple SketchUp plan to a construction service marketplace. I could then sit back and watch architects and interior designers bid for the planning work. My friends in the network could recommend contractors. Tools and parts suppliers could offer me discounts knowing exactly what I needed for the job. I could rate everything that happens and contribute to the reputation of any node in the ecosystem.

Imagine how much more value would be created in the home buying market if a potential buyer could see all this data on a house that was for sale. I might be able to sell my home for a higher price if my remodel was done using highly reputable providers. There would be a financial incentive for me to document everything and to get the right certifications on the work.

Imagine lenders knowing that I’m an excellent remodeller based on my reputation and sales track record. I might be able to negotiate better terms for a loan or even solicit competing bids for my mortgage on the next house I want to invest in.

At every step in the process, there is a role for the city government to add value and thus become more relevant. Then the more I contribute, the more it knows about what’s happening. The more it knows, the more effective it can be in driving better standards and improving safety and legislating where necessary.

My mind spins at the possibilities in such a world. Of course, when you have a hammer everything looks like a nail. But it seems to me that the building permit and inspection business is broken in exactly the places that the Internet is more than capable of fixing.

Media As A Service

Much like print and tv are becoming marketing vehicles to drive people online, the domain name for an online media service is becoming sort of an abstract utility or maybe just a brand address for media services rather than the real estate upon which the core activity occurs. The service a media vehicle provides matters more than the vehicle itself.

And this isn’t only happening in the content space. Every aspect of the media business is pointing to a services model. Here’s what the key pieces look like, in my mind:

  1. Data is infinitely distributable. All data…not just editorialized words. The RSS standard opened the doors for vast distribution networks, and services like Yahoo! Pipes and Feedburner figured out how to make the distribution methods meaningful. There’s an endless supply of microchunks flying around the Internet, most of them unattached to any domain or URL except as a handy reference point.
  2. Data can be visualized in meaningful ways. AJAX and the many freely available widget kits and javascript libraries such as YUI are rendering these microchunks in the right place at the right time in the right way for people which, again, is not always on a web site. The Internet user experience is no longer held back by the limitations of HTML and the packaging a site owner predefines for their media.
  3. Media is created by everyone. Whether written in long form by a reporter or researcher, captured as video by a mobile phone owner, or simply clicked by a casual web site visitor, expressions of interest are shared, measured and interpreted in many different ways. This results in a seemingly neverending stream of media flowing in and out of every corner of the digital universe.
  4. Distribution technologies are increasingly efficient and inexpensive. Personal media services like instant messaging, blog tools, podcasting and collaborative media services like Wikipedia, del.icio.us, Flickr, etc. are easy to use and often free. Web services and open source software enable people and companies to scale distribution and production functionality for large audiences or groups of users with negligeable costs. Most importantly, these tools enable people to be influential without ever owning a domain.
  5. The distance between buyer and seller is shrinking. There are more and more ways for buyers to find sellers and sellers to find buyers from search engines to recommendation tools to coupon rss feeds, etc. Distributed ad markets like Right Media are enabling marketers and service providers to negotiate both the methods and the value of a marketing message. Advertising can operate as a service, too.

After re-reading this description myself, it looks like I’ve just echoed much of the whole Web 2.0 thing yet again. That makes me think I didn’t articulate the concept properly, as I believe there’s a very different way to visualize how data get created, packaged, distributed and remixed and how the various parts of a media business can be coupled both within the organization and across the wider network. Maybe that’s Web 2.0. Maybe it’s edge economics. SOA. Whatever.

The important thing is to think of how your media business can create for yourself or leverage how others offer Marketing As A Service, Sales As A Service, Operations As A Service, in addition to your editorial and community building efforts. Here’s a quick chart of how a media business might look that hopefully gets the point across:

Staffing Model Source Data Coopted Data Distribution Services
EDITORIAL Reporters, Community Managers, Assemblers (formerly known as ‘Producers’) Original News, Analysis, Columns News Wires, Paid Data Feeds, Free RSS Feeds, Links, Comments, Votes, Ratings, Clicks RSS Feeds, Content API (Read and Write)
MARKETING Customer Service, Evangelists, Event Organizers SEO, SEM, Paid Inclusion, Sponsorships, Staff Blogs Partner Promotion, Customer Evangelist Blogs Customer Help, Usage Policies, SLAs, Traffic/Referrals to favored partners
SALES Sales Engineers, Business Development Customer Data, On-site Inventory Partner Inventory, OEM Partner Services Ad Service API (Read and Write)

We’ve seen Journalism As A Service evolve with a little more clarity, particularly recently. Mark Glaser provides a step-by-step guide on how to structure a community-driven news organization:

“Reach out to the community for bloggers, muckrakers and go-to experts. Each topic area would require more than just reacting to news. The Topic Chief would be sure to enlist as many experts as possible not only to be sources but to also be contributors, commenters, and word-of-mouth marketers. Anyone who possesses the skills that go beyond basic participation can be hired on as freelancers or even full-time staff.”

Similarly, Doc Searls’ “How To Save Newspapers” post also lays out what needs to happen on the editorial side. Here’s step #5 in his list:

“Start looking toward the best of those bloggers as potential stringers. Or at least as partners in shared job of informing the community about What’s Going On and What Matters Around Here. The blogosphere is thick with obsessives who write (often with more authority than anybody inside the paper) on topics like water quality, politics, road improvement, historical preservation, performing artisty and a zillion other topics. These people, these writers, are potentially huge resources for you. They are not competitors. The whole “bloggers vs. journalism” thing is a red herring, and a rotten one at that. There’s a symbiosis that needs to happen, and it’s barely beginning. Get in front of it, and everybody will benefit.”

There is lots of guidance for the newsroom, but all parts of a media business can become services.

For example, the ultimate in Marketing As A Service is the customer evangelist. It’s not about branded banners, as Valleywag points out,

“When paid-for banner ads lead to another site that’s supported by banner ads, you know that something’s wrong. Anyone who relies on that circular spending is asking for trouble.”

Marketing should be about enabling customer evangelists whether your customer is simply promoting your stuff for you or actually distributing and reselling it. Fred Wilson thinks of this in terms of “Superdistribution“:

“Superdistribution means turning every consumer into a distribution partner. Every person who buys a record, a movie, reads a newspaper, a book, every person who buys a Sonos or a Vespa becomes a retailer of that item. It’s word of mouth marketing, referral marketing, but with one important difference. The consumer is the retailer.”

None of this needs to happen on a single domain. The domain chain in any of these actions probably should be invisible to people, anyhow, except maybe to ground the events in trusted relationships.

Now, there are many domains that can create wonderfully useful and valuable destinations once they reach a certain critical mass. Invoking another over-used dotcom jargon word, this is what happens at the head of the long tail. And there are obviously lots of nice advantages of being in that position.

Most media companies want to be in that position and fight tooth and nail for it even if it just means being at the head of a niche curve. But instead of or maybe in addition to competing for position on the curve, most media companies need to think about how they provide relevant services outside of their domains that do something useful or valuable in meaningful ways across the entire spectrum.

Posting articles on your domain isn’t good enough any more. The constant fight for page views should be positive proof of that. There’s a bigger, deeper, longer term position out there as a critical part of a network. Sun Microsystems’ mantra “The Network is the Computer” is still meaningful in this context. What is your role if “The Network is the Media”?

Similarly, is Marshall Mcluhan’s widely adopted view that “The Medium Is The Message” still true? Or, like many have asked about the IT market, does the medium matter anymore?

If we are moving to an intention economy, then those who best enable and capture intention will win. And that doesn’t have to happen on a domain any more.

Membership has its privileges

Mark Glaser asks his readers this week to submit the answer to the following question:

“What would motivate you to contribute to a citizen media site?”

I can’t imagine that anyone is going to be able to answer that question in an interesting way. It’s the wrong question. It’s kind of like asking why do people sing at church? Or why do people meet their friends at the pub?


Photo: -bartimaeus-

If the church asks you to sing, you sing. If your friends tell you to meet at the pub, you go to the pub. The community and purpose of doing things together is already implied, so you do whatever everyone else in that community does if you want to be a part of it.

Jon Udell starts to dig into the critical mass hurdles for social networks in a recent post where he quotes Gary McGraw saying:

“People keep asking me to join the LinkedIn network, but I’m already part of a network. It’s called the Internet.”

The real question is not about getting people to do things. There are too many things to do and too many people to socialize with in a day already.

The question is about forming meaningful communities and the kinds of things that will help a community flourish. Meaning comes in millions of different shapes and sizes, but there are lots of precedents in terms of ideologies, aesthetics, and methods.

News, for example, is inherently about being first to report on an event. Successful community-based news sites enable people who care enough about a topic to either be the first to report on it or be clued in before less speedy outlets pick up on something. It feeds into a competitive and sometimes gossipy human nature. Just ask your best reporters why they became reporters. Digg appeals to the reporter in all of us.

I used to attend a charity event called Rebuilding Together where groups of people would assemble and fix up houses and schools around the city of San Francisco. There was a core team who selected applications for fix-it team deployments. Then there was a leader who would drive the work to be done by each team at each site. On the chosen date, people would jump on a project and invite their friends to join. It was impressive to see what a focused group could accomplish in a day, fixing plumbing, painting, cleaning, rebuilding fences, etc.

Why did people do it?

There was a purpose. We were helping people truly in need. The commitment was lightweight. It was 1 day a year. It was well organized. I didn’t have to debate with people about how things should be done. The result was impactful, a total overhaul of a building. It was fun. I had a laugh with my friends and met new people.

Often when people start asking how you get to critical mass, they’re losing the plot. Sure, it would be great to worry about scaling a site rather than fighting for a Digg. But if you and your community are doing something unique and valuable, then size really shouldn’t matter. And in many cases, it makes sense to make the community exclusive and smaller rather than bigger and diluted, anyhow.

The question then becomes, “Are you offering a service that a lot of people find unique and valuable?”

I think a lot of publishers fail to understand the size of a potential market, what’s unique about an offering, and the value of that offering to the people who do actually care about it.

Then there’s also the issue of recognizing what you can actually deliver. You have to play to your strengths.

Yahoo! Answers is a good example of that. The idea of getting immediate answers to any question you can think of from real humans is outrageously ambitious. There are lots of ways to get answers to questions out there. But Yahoo! played to its strengths to get it off the ground, then it just took off. It’s easy. It’s fun. It works. And, therefore, it’s meaningful. And now there’s nothing like it out there anywhere.

Of course, not everybody can point a firehose of traffic at a domain, but there are plenty of cases where Yahoo! failed to create a community by pointing a firehose of traffic at it.

So, what makes a meaningful community that has a definitive purpose? Yeah, well, that’s an answer you can get from Cameron Marlow, danah boyd, and a lot of people a lot smarter than me.

Though perhaps this is all just echo blogging and the real question gets to something people already understand. Maybe the question is simply: “How do you make membership in your community desirable?”

Wikipedia defines “privilege” as follows:

A privilege—etymologically “private law” or law relating to a specific individual—is an honour, or permissive activity granted by another person or a government. A privilege is not a right and in some cases can be revoked.

I think the answer is in there somewhere for everyone who is struggling to get their community to do stuff.


Photo:Manne