by adam mathes
archive · subscribe

Deepleap Revisited

Ben pointed me to Kottke’s recent post on splicing feeds and distributed internet storage which got us thinking about Deepleap again. (Jason mentions this briefly at the end of his post.)

While splitting and rejoining feeds of content is nice, I think it’s really just a tiny sliver of what should really be done to route, store, organize, and distribute the bits of information we deal with.

Here’s a rough outline of what I think a Deepleap-like program architecture would look like if we redid it today. Again, the basic fundamental premise is a centralized store of information that grabs information in context and can then provide it back in context. But based on advancements in web services in the past few years, it would look a little different. Much less “screen scraping” and a lot more XML feed reading and legitimate API calls.


  1. Centralized (or local) semi-structured data store. Think something along the lines of current attempts to merge XML storage and databases. Think “XML fragments.” Think - simple object “types” - product, web page, personal contact, email message - that are recognized by the system and the ability to create more. Think controlled vocabularies for elements between objects, etc.

  2. Flexible SQL-like query system. See Jon Udell’s post for some thoughts.

  3. System of plugins to recognize and acquire content into the data store. Amazon plugin to grab books, weblog plugin to write commentary or grab it from your site, RSS/Atom plugins, a Flickr plugin. Web entry points, email entry point, IM entry points.


Various modules to grab content and send it to the backend in context. Other modules to search, format, and send data to other places.

You’d be able to “ping” the system with new content items. Web pages you want to save, products you want to buy or recommend, photos to save to an album.

Grabbing N recent entries of X object types not marked private and then reformatting to XHTML would be the personal weblog-ish face, but I imagine that the search functionality would be just as important for grabbing content back out. Exporting XML feeds, Palm conduits, iCal output, and other output format plugins would be good.

This is the sort of thing I might like to work on if I decide not to take classes in the Spring and instead write a Master’s thesis. Or it’s the kind of thing I’d like to design and work on after I finish my Master’s degree next June.

It’s also hopefully one of the directions companies like Six Apart will go. I’d rather see a company like Six Apart, or a new player, take the lead in this space rather than Microsoft or Google. A smaller company with less baggage and subsidiaries and such will be more likely to foster interoperability, (I have high hopes for 6A based on their new VP of Products.)

· · ·

If you enjoyed this post, please join my mailing list