Jamie Thomson

There’s loads of information coming out of this week’s Microsoft Professional Developers Conference in Los Angeles…Windows 7, Office Web Apps and Windows Azure seem to be the most newsworthy.

Of most interest to me though is news about the Live Framework, this is basically an API that allows developers to interact with the data that has previously only been accessible via Live Mesh and Windows Live Services. I’ve been lucky enough (thanks Alistair and Stuart) to have had access to the Live Framework for the past few months but have been gagged by a Non-Disclosure agreement – well I’ve just been told that the gag has been removed and I can now speak freely about this, something I’ve been dying to do.

Of principle interest to most of the people that read my blog is that all the data that Microsoft holds about us from its various Live Services is going to be made available via the Live Framework. So, the Live Framework now gives developers access to our contacts, calendar, photos, profile … stuff that previously has only been available at the various properties like http://contacts.live.com, http://calendar.live.com, http://<spaces-id>spaces.live.com/photos, http://<spaces-id>.spaces.live.com/recent respectively.

Let’s take a look at the Live Framework (or LiveFX as it is becoming known) overview diagram:

There’s a lot in there but I want to draw your attention to the box that I’ve outlined in red. You can see listed in there:

• Profile
• Groups (soon to be released – I’ll be talking about Live Groups much more soon as well)
• Folders (hmm…interesting)
• Contacts
• Calendars
• Photos
• Geospatial
• Search

Do some of those look familiar? You betcha! The information that Microsoft holds about you from their various Live Services will now be available to developers via LiveFX if (and I need to emphasize this strongly) you allow that data to be accessed! More on this later.

At the time of writing the only sections in that list that I know nothing about are Geospatial and Search but I am endeavouring to find out about them and when I’ve got anything to share I’ll be sure and let you know. Geospatial in particular intrigues me whilst Search might be a pointer to some of the ideas I postulated in Mesh’s hidden agenda.

That’s all for now; I’ve been dying to talk about this stuff for months and now its here I finally can. Expect more ramblings from me about LiveFX in the coming weeks but in the meantime, sit back and think about the wealth of apps that are going to get built that can leverage this data. The mind boggles quite frankly and I’ll be trying to un-boggle mine on this blog and my more developer-focused blog in the coming weeks. Stay tuned!

-Jamie

I see there is a search box on Live Mesh’s Live Desktop right now that is currently greyed out but I guess its safe to assume that at some point Live Mesh will be indexing all of the stuff in our Mesh so that we can search it.

As I said in my blog entry Evernote – Your memory downloaded I’m a big fan of Evernote because of its ability to find text within pictures and then index it for searchability. Imagine knowing that you had a photo of the Hollywood sign but you don’t know where the photo is – Evernote would be able to pick out the word "Hollywood" from inside the photo and any search on "hollywood" would return that photo.

Now imagine that that photo was somewhere in your Mesh and we had the same searchability within Mesh itself rather than going out to Evernote. That would be killer.

And why stop at text? Some day some boffin somewhere will work out a way for computers to listen to speech or watch videos and know what they are about – being able to search on the contents of all our media would be amazing.

Naturally we should be able to search the contents of our Mesh from Live Search as well as I already suggested in Mesh’s hidden agenda.

It would need some huge data centres to store all the indexes but hey – Microsoft are already building them.

-Jamie

An interesting little titbit of Mesh news here. Microspotting has interviewed Reed Sturvenant who heads up Microsoft Startup labs in Boston. Here’s what the group do (taken from the interview)

The idea is to create an internal, Boston-based development group that can build products and launch them to market the same way that start-ups do.

What is of most interest to me though is this quote:

We’ve been asked to focus on two things: products that will attract a large enthusiastic audience, and to help drive new strategic platforms. So for example we’re using the Live Mesh SDK and building a set of applications on top of that.

So Microsoft are setting up a Silicon-Valley-esque enclave in Boston with the express purposes of building applications for Mesh. Can’t wait to see what they come up with.

-Jamie

I’m sat on my train home listening to last Friday’s (25th July 2008) episode of The Gillmor Gang; for those that don’t know The Gillmor Gang is a conglomerate of technology pontificators based in Silicon Valley that are headed by Steve Gillmor. On this particular episode they’re talking about what they think Microsoft are doing and are going to do in the online space – well worth a listen if you’re interested in such things (as I am).

They talk a lot about Mesh, XBox, Ray Ozzie, Live Search and the relative relevancy (or not) of each. They didn’t come to any conclusion about any of it but it was fun nonetheless although I do think they missed an important point about Mesh and search. Its something that’s been rolling around my head for a while and the point is this. If Mesh is a success (and I believe that it will be) then there is going to be a lot of information (gigabytes worth) per person stored in the cloud and across a multitude of devices that crucially only Microsoft has access to. And what is the best way of making sense of information of that magnitude? Search, that’s what!

Hence I predict that we will see a new kind of search engine from Microsoft. One that shows information from the web alongside and interspersed with your own content and that which your friends have chosen to share with you. If you are logged into http://search.live.com then Microsoft can return to you search results of YOUR OWN STUFF and that is a game changer in search. As Joe Wilcox said earlier today “Search should be about what’s important to you.”

Let’s take an example. Imagine you remember that you had recently been reading a Word document about balaclavas; you don’t know whereabouts you have the document but you know its in your Mesh somewhere. Instead of hunting around for it you head for http://search.live.com and search your Mesh from Microsoft’s own search engine. In your results you find the document that you’re looking for and also get back helpful information from Live Search including where you can buy the cheapest balaclavas and all the information you ever wanted to know about them too. Ever wonder why in Microsoft Office 2007 all the documents are saved as XML files? Simple…it makes it easier to search them.

Want another reason? How about being able to search through all of your Messenger conversations from http://search.live.com.

There are other ways that Mesh and Live Search can be mutually compatible too. As my colleague at Conchango Paul Dawson points out in his blog entry Live Services – Social Search – Collaborative research social search is evolving:

Using the Live Messenger services, and the Live Search services to create ‘social search’ – or in my book, a nice way of being able to share online research with someone – or a group.

it’s what we’ve been talking about for many of our clients. You can see this being used for a group, or a couple, planning a holiday or a trip, sharing their early research, and testing ideas on where to go with each other.

Paul and I have spoken about this a lot lately as we try and cook up some ways that we can use Mesh in the future. Mesh presents a great opportunity for conducting your research via a search engine and “shelving” your search results so that you or someone else can view them later.

Live Search and Live Mesh– a winning combination. You heard it here first.

-Jamie

Yaron Goland doesn’t say very much on his blog but when he does its usually something well worth reading. Today he wrote one such blog What is Microsoft’s Cosmos service?.

I don’t pretend to understand all of what Yaron is talking about (his brain is on a higher plane to mine) but as a data guy by trade I’m fascinated by Cosmos which is, according to Yaron:

…Microsoft’s internal data storage/query system for analyzing enormous amounts (as in petabytes) of data

…the architecture Microsoft uses to store and query petabytes of data

…a very successful system that is growing at a breakneck pace both in terms of the number of customers we support and the size of the clusters we run

I don’t want to jump to too many assumptions but I wouldn’t mind guessing that Cosmos is the infrastructure that is underpinning the massive amounts of data that Live Mesh will need to store.

Yaron mentions that Cosmos runs upon Dryad which is a system from Microsoft Research that runs computationally heavy processes over parallel architectures. Dryad is particularly interesting to me because its first outing was as a means for speeding up the processing of massive data volumes using SQL Server Integration Services (otherwise known as SSIS, the product for which I gained my MVP award and earn my corn). I previously talked about SSIS and Dryad here: http://blogs.conchango.com/jamiethomson/archive/2007/11/13/Dryad.aspx where I pick out the following quote:

Dryad is an infrastructure which allows a programmer to use the resources of a computer cluster or a data center for running data-parallel programs. A Dryad programmer can use thousands of machines, each of them with multiple processors or cores, without knowing anything about concurrent programming

Fascinating stuff. The worlds that I inhabit (SQL Server and Windows Live) are growing closer and closer together. When you read about infrastructure of this magnitude then you start to realise why there is only a handful of companies in the world that have anything like the capacity for building a tool like Live Mesh. Just to put it into context for you a petabyte is 1,000,000,000,000,000 bytes. Or 1,000,000 gigabytes. Whichever way you cut it words can’t really describe the enormity of “petabytes of data”.

-Jamie

2 days ago the Live Mesh team released an update to Live Mesh that enables us to sync files peer-to-peer (i.e. device-to-device) without syncing them to the cloud. I’ve produced a video that shows how this manifests itself in the Live Desktop which I’ve embedded below.

 http://silverlight.services.live.com/invoke/57578/AtouraroundMesh'sLiveDesktop/iframe.html

Let me know your thoughts. Do you want more of these types of short show-and-tell videos? If so I’ll see what I can do.

-Jamie

P.S. The video is hosted on Silverlight Streaming. You will need Silverlight installed in order to view it.Windows Live Tags: Clubhouse, Live Mesh, How-to