// The Song Remains The Same

Once the young lava fields of Iceland cool down, life begins anew little by little. Ice, wind and water flatten and carve out shapes to begin with, then, during the summer, bacteria, lichen and fungi prepare the soil for plants, in particular mosses which adapt to an environment which remains difficult. These plants colonize the most favourable sites and terrain little by little, forming a new ecosystem.

— from the book Out of Control: The New Biology of Machines, Social Systems, & the Economic World by Kevin Kelly

I believe we are building a unified ecosystem of web applications that will evolve and grow just like biological ecosystems do. In order to allow this ecosystem of data and applications to co-evolve and interlink over time, we need to provide a way for this ecosystem to discover and self organize itself. I believe XRDS-Simple can be one of the technologies that allows us to “boot up” the emerging web data ecosystem.

As we move into the economic downturn, startups (and grassroots efforts) will shift from focusing on the last epoch of social graph and media sites to looking at how we can do something with all these disparate pockets of our media around the web. Just like in the last economic downturn (2000), we will see a new set of seed ideas take hold in the web ecosystem and spring forth many new hits for this next epoch of the web. I believe one aspect of the next evolution of the web will be based around dynamic discovery of relative linked data.

In previous articles we saw how the natural world employs self organization to drive emergent processes. One of the underpinning mechanics of self organization is its decentralized nature. We saw how in ad-hoc networks a network could adapt and thrive under difficult conditions with no central leader. The internet is a loosely coupled set of computers, just like an ad-hoc network. Its an ad-hoc network of clients and servers within a system that faces many of the same obstacles a mesh network does — and has a tremendous amount of linked data contained inside.

This linked data is coming online at a rapid pace, but it continues to be pooled in silos; the data is linked, yet we still have to manually connect those links and use our data in seperate sandboxes (ex: my facebook and myspace data does not play together). Just as google found out, a tremendous amount of value lies in how information is linked together. Today users have their photos in a couple different services and the majority of the US has its social graph split between myspace and facebook. There are all sorts of bookmarking services, and we have accounts for all sorts of shopping sites scattered everywhere. As these clusters of data come online and grow, more value can be had in how they are connected and what can be done with these new connections. With respect to how we’re allowed to use our data, we’re sorta in the “Hotel California” phase of the web —

you can check in, but you can never check out.

In future iterations of the web I see users taking ownership of their data and creating a more personal user experience based on their existing data no matter what device they use or their location. This depends on how we link our data, how we allow our data to be discovered. So that begs the question, just how will we leverage this linked data and aggregate our information in more intelligent ways? I think a key lies in ad-hoc networks, social insects, and self organization — with one fundamental aspect being dynamic discovery.

Discovery is the key mechanic that connects relevant parts of a decentralized system. As the web and linked data get more complex, we can’t predict the ways in which will need to discover and use data. Search engines can only give us part of the solution as they make connections based on their own assumptions and biases. We saw how ants discover food sources using stigmergy yet with no central source of control. We also saw how an ad-hoc network leverages a similar stigmergy based mechanic to find quality routes through a chaotic network. Discovery needs a little more structure than we currently employ — it needs a “road sign” to know where and how to get to your data. Ants do this with pheromone build up, MANETs with routing tables, but how can we do this in a linked data world? I believe XRDS-Simple can provide us with the basic “n+1″ evolution of dynamic discovery we need for this emerging web data ecosystem.

XRDS-Simple (or XRDS-S - google group) as described by Eran Hammer-Lahav:

XRDS came out of the XRI world and is focused on service endpoint selection – connecting a list of concrete resources to an abstract identifier. When expending XRDS to HTTP URLs - non-abstract web resources - it providers a simple format for tagging resources and related services. These tags, called types, are URI-formatted strings which inform machines about the capabilities and characteristics of the resource they are associated with.

This means XRDS-S allows a service to discover attributes about a resource which can redirect a service towards other related or linked resources. XRDS-S is essentially an XML format for pointing towards a set of services related to a url. The current case I am most focused on is a user’s personal set of services, as in the places a user

• Keeps their images
• Keeps their bookmarks
• Delegates their identity provider to
• etc

If a user can control this index of their service “endpoints” (XRDS-S resource), the user can then provide specific levels of access to specific subsets of interested outside parties (a longer term goal). And when we have a robust technology that allows users to do this, I believe web data can interconnect and auto discover in ways we haven’t even through of yet.

I think XRDS-S can create more value for the user by giving control and ownership of data flow to the user, back to the owner, and ultimately give us a better computing experience. However, thats big pie in the sky talk. Let’s think more along the lines of building up the technology in terms of “n+1″, in terms of adding a basic new mechanic and see how it effects the emerging web data ecosystem.

In the short term, I’d like to explore more on how XRDS-S can be used to do some basic things like

• Allow a 3rd Party Service to discover your preferred bookmarking service, and automatically route the saved url to said service
• Automatically discover your media you have stored around the web
• Dynamically find a related service in an contextually sensitive way which enhances the user experience.

In coming articles, I want to take a look at how XRDS-S is emerging as a discovery technique, and scenarios where it can be used — and how it can be a road sign in the emerging digital ecosystem.