All that glitters...

A quick word about SPARQL....

John sent me this link to an InfoWorld article that discusses the changes that will happen once the promise of the Semantic Web becomes reality.

First, congratulations to everyone who worked on SPARQL. I have gleaned some understanding over the last few years what it means to try to get agreement and drive ideas to a finished standards proposal... it's hard.

The title of this post sounds like I'm going to say bad things about SPARQL, but I'm not. SPARQL and the functionality that it will provide is very important and very valuable. I do think that it's important to put it in the context of the XDI and Higgins work that we are engaged in.

RDF and SPARQL will provide more available structured data that can be incorporated into the DataWeb. However SPARQL only addresses a small part of the problems that I talk about on this blog. For example, SPARQL doesn't have identification, authentication and authorization built into it's framework. I think this is a shame; we have seen time and again that building the capability for security into a protocol is far superior to 'bolting it on' or 'wrapping it around'.

SPARQL specifically leaves Update and Insert semantics as 'out-of-scope'. There are lots of use cases for which this is fine. However, there are also lots of use cases where you really need to push values back out.

So SPARQL is great... we will definitely build a standard plugin so that you can consume data available via SPARQL from XDI. We will probably even build a SPARQL query engine on top of our XDI engine so that any public data available from XDI can be accessed via SPARQL.

Open Source Brain

Up till now I have had exclusive access to Steven Churchill's brilliant and clear thinking as we have been working together closely for years. Now you all have limited access too... Steve is now blogging. Check out his first post on a Simple Identity Model.

I-Name news

Did you see this?

[Twitter] Joseph Smarr posted on Twitter
You can now log into Plaxo with an iName! I just attached =joseph.smarr. OpenIDDevCamp rocks, as do John Bradley and Michael Krelin! :)

It's great having John on the ooTao team... Thanks all of you!

Relationships are real

In my previous post I touched on the question of what is a Map. In our world, today, computers tend to make the distinction between the 'real world' and the representation of the world 'fuzzy'. 

If I have an interactive 'map' of my water system and I redirect the water flow from that map; which came first.... Is the physical system now simply a representation of the virtual model? Is it a physical 'memory' of the state that I changed on my computer or is it the other way round? If the 'map' and the state of the valves in the water system are 'out of synch' which is right? My intention was to redirect the flow, therefore the 'map' is right and the water is flowing wrong. I need to fix the valve so that it correctly represents the map... or do I?  

Conventionally one would assume that the map represents the physical state and that 'instructions' either successfully change the state or not. The map should be a representation of state not the authority for it. The software has the capability to 'poll' the physical networks state such that if the state changes the map can 'auto-correct' to current conditions. Depending on the completeness of the software we just have to hope that the physical network never gets into a state that it doesn't know how to represent. 

But here's the fuzziness again. If the valve has a processor and a network connection (which it would have to have to respond to instructions) it can also 'poll' the system for what it's current state should be and auto-correct. So at what point is the physical system just solid state memory?

Another example I have been thinking about is gerrymandering. Does that districting map dictate where people vote or does it represent where they vote?

This makes my head hurt!!

All of this is not JUST mental masturbation; I'm trying to work out what comes first a 'map' of the social graph that establishes our relationships and is 'portable' or is there some other manifestation of relationship that the social graph is a portable representation of.

Here's my conclusion...

Relationships are real. Directed relationship objects MUST be reified in the identity network. Maps of the social graph will show different aspects (attributes) of both types of top level entity; entities AND relationships. Like interactive maps of the physical world where you can layer utilities, streets, satellite pictures and geo-political attributes to communicate (makes portable) the state of a given physical area. The 'social graph' is a map that communicates (makes portable) the state of some entities and their relationships. 

An important quality of the 'portable social graph' is that each 'map' only represents a sub-section of reality.  I would expect different people might have access to different 'maps'. I would expose different sub-sets of my entity and relationships data to different 'mapping authorities'.

So this leads me to the conclusion that before we can really address social graph portability we need a better understanding of what relationships are.

In systems that I have built that have reified the relationship object I have found the following qualities necessary...

• Relationships are unidirectional and COMPLETELY controlled by the 'root' end of the arc. 
• Relationships are no different from any other 'claim' that I make about you, totally unsubstantiated. (good mapping authorities MIGHT only show reciprocated or verified relationship claims)
• Other people are ALWAYS interacting with one (or more) of my relationship objects NOT with my entity object.  (that's the point of reifying the relationship)
  
• Relationship objects contain several different types of data...

1. Data that I keep about you, that is mine, only mine and is never meant to be shared. Stuff like "this guy tells really bad jokes" or "it's not in his profile but I know his home phone number is XXX"
2. Pointers to the data about me that I want you to have access to. (in my world, it is the relationship object that dereferences the pointers NOT the 'other' entity).
3. Pointers to (and caches of) information that you have exposed to me about you (and sometimes others).

XFN or FOAF are ways for me to expose a sub-set of my entity and relationships to PUBLIC mapping authorities. They are but a map of something a LOT more complex that needs to be given a lot more attention. 

(of course xdi has all of this solved... if only you would all just use it :-) )

 

Social Graph Portability

There's a lot of talk these days about social graph portability so I guess it's time that I explore the xri based idea that has been running around my head for the last couple of years. This post probably isn't going to go into any more depth than I have already thought about but I hope it will inspire me to go think some more....

The basic idea is this... I am =andy, you are =steve, I can create any number of directed, typed, relationships to you by using the extensibility of the =andy namespace....

=andy*(=steve) establishes a generic relationship.

=andy*(+friend)*(=steve) establishes a friend relationship.

=andy*(+trusted)*(=steve) establishes an actionable relationship.

Let me point out the thing that I think are cool about this...

• These entries MUST have been added and/or removed by the entity that controls the =andy name space.
• Typed relationships can take advantage of the 'dictionary' space (xris that start with '+') and therefore solve a lot of the semantic mapping issues.
• By creating this entry in my name space this relationship has its own i-number, I have very literally reified the relationship. The relationship can have metadata, services and any other quality of a top level entity.
• The target of the relationship can USE this identity; =Steve can assert =andy*(+trusted)*(=steve) as their identity... the fact that xri resolution for this succeeds 'prooves' that =andy established the relationship... You can xri resolve =steve for whatever flavor of authN service that you are interested in so the 'user' can 'proove' they are =steve... I AM =steve who has a (+trusted) relationship with =andy.
• Group management IS relationship management =andy*(+family)*(=richard).
• Relationships are STRONGLY directed. An assertion about a relationship with =steve means only as much as you decide to put on it. Did you know that me and Bill Gates are best buddies and that I'm married to Angelina Jolie?
  

A couple of problems with this as it stands:

• It is ALL public. Maybe once I have finished reading the ID-WSF Service Discovery Spec I'll have a better idea how to mix and match the public and private parts of this in a more privacy protecting way.
• There is no native xri way to query the graph. Even if I wanted it to be public there's no (currently speced) way to get all of =andy's +trusted people.
  

Despite the obvious problems, I think the strengths still make this something worth exploring and the problems something worth trying to solve. Part of why I like this approach is that using some simple wildcards lets me address and permission based on the graph in the same syntax...

• When I sign up for the genealogical service it is understood that write rights are granted to =andy and =andy*(+family)*($children) and read rights are given to =andy*(+family)*($descendants)
  
• I can send a message to =andy*(+trusted)*($all)
  

I guess that what I'm trying to say is this.... I don't see the Identity Layer and the Social Graph as 2 separate things. I think it's well accepted that any meaningful abstract identity system MUST reify relationships as top level objects. It must be an Identity AND Relationship layer....

We must not get confused between the world and the map of the world. It's great that with todays technology we can create these interactive maps that let you:

• show all the houses and the roads
• now turn off the roads and show the electric grid
• now only show the sewer lines

But that isn't what exists... the connections between the house and the grid are real and solid. When we talk about the Social Graph we need to be clear if we are talking about the map of the graph or the actual reality it is meant to represent.

Option 5

I was reading this post in Phil's blog, as I do, and had to get this thought out of my head...

Phil says:

There are basically four options for deployment, as far as I can tell:

1. Sell software that gets installed on customer hardware
2. Package your code onto a hardware appliance and sell the box
3. Package your code onto a virtual appliance and sell the appliance
4. Sell a hosted solution

All of these have advantages and disadvantages and each is appropriate in different circumstances.

Phil goes on to describe some of the pros and cons of each of these options... It's a good read. There is however another solution; well it's not really another solution, it's really just a variation on 4... But it doesn't have the problems described in Phil's post....

5. Have Wingaa host it.

Now this also has pros and cons like any of the other solutions... but is an option. Wingaa is all about High Availability, High Security and Non-Intrusive Identity Service Hosting by specialists in Privacy and Trust. That is our business... We want to take the support calls, keep the geographically disperse secure data centers in hot fail over mode. We want to persist all that PII and take on that liability, that's the challenge that we have set ourselves... Don't get me wrong, we plan to charge you for it. Security and availability and lots of liability insurance doesn't come cheap... but if you have a business that needs those qualities we can probably do it as inexpensively as you can do it yourself, without you having to do the work.

There are business for whom this fundamentally isn't an option, but there are also business for whom this is the perfect solution. You have the killer business idea, THE next social app, we have the capability to not only run it for you but to run it in an environment that can be trusted to protect the privacy of each individual above ANY business imperative. You can deploy quickly and cheaply into high production value mode without the burden of the upfront cost. You can look instantly credible to enterprise partners.... they'll say... oh, you use the Wingaa network.... GREAT!

Wow... what started as a simple observation that maybe we could help, in some cases, just became a my marketing guys worst nightmare.... Andy runs wild with words. (I think maybe the fever has come back)

What WIngaa Does....

So the message on the Wingaa site still isn't simple enough. I'm going to try to fix that this week.

What Wingaa does is host services for other companies. There is more and more demand on internet companies to offer their customers more and more services. We help companies satisfy that demand with minimal cost and effort on their part.

That's a start... let's see what ends up on the site.

Looking good...

The Wingaa website has been launched. Check it out at www.wingaa.com

XRI utils

One of the things that I look for with new technologies is tools and utilities being developed that will make the technology easy to use. This week 2 such utilities have appeared...

Markus Sabadello of Parity and @freeXRI has developed an XRI Resolution Client for the iPhone... You can learn more about it and download it from here.

I also shared with a couple of people... and now with you, this Mac Dashboard Widget that lets you look into an XRDS document without having to remember the proxy resolver syntax.

If only it were that simple...

Both Kim and Paul picked up this post by Francis Shanahan about the fragmentation of our online information. The center piece of his post is his diagram representing our information spheres... here it is:


I like the diagram in as much as it STARTS to show the problem we face. I dislike it because it implies a structure and solution that WAY over simplifies the problem.

Consider these 2 questions and you'll see what I mean...

Think about the next level out beyond the blue boxes... the attributes. You'll notice that there is massive duplication of information all around circle. This diagram totally fails to represent the interconnectedness of the data. The DataWeb is NOT a set of nicely ordered hierarchies and diagrams that lock us into that way of thinking, I think, do us a disservice.

While this diagram neatly implies that the blue boxes can be canonically categorized it is simply not true. My guess is if we gave each of you the job of categorizing the blue boxes you would come up with not only different groupings but different semantics for those groupings. Don't get me wrong; Francis's diagram is as valid a projection of order onto the mess as any. My complaint is that, we, the people trying to solve these problems must not get lulled into only seeing one dimension of this problem.

I think about the problem like this... Each of you do your version of Francis's diagram but include all the lines between the blue boxes that have data duplication. Also don't limit yourself to only putting each blue box as a child to only one green box... embrace the fact that World of Warcraft is a community, a social network and a gaming site. Once I have gathered all of you diagrams I make them all semi-transparent and put them on top of each other. That diagram is a fair representation of the DataWeb.

An interesting thing to notice is that the lines that go around the outer rim of the diagram are not, on the whole, subjective. We can build a 'rule' that says if two data points have the same value and the same update rules then they should be linked. In other words the lines at the third level should juxtapose fairly well from one persons diagram to the next. Notice that the linking rule is based on values not on labels as the semantic issues in looking at the labels adds another level of complexity.

So.... to the point.... What Francis described is exactly what I have been talking about for the last 3 years. If you take Farncis's diagram, with my radial additions, and put it into a linear form instead of radial, you get exactly the graphs that I have been drawing for years. Three levels, lines going up and down the levels and lines going across the levels. This is no coincidence; there is a fundamental 'truth' about that representation that is much like, in my mind, Kim's laws. This truth is not me, or anyone else, saying 'you must do it this way' it is us trying to point out "this is the nature of the DataWeb". What we have done, and I would be happy to spend time with any of you showing you this in detail, is specified a syntax for describing, precisely, the relationships in that web. Relational databases NEEDED ERD in order to get wide adoption, if people couldn't simply communicate, capture and represent the data models they were working with how could they ever build large complex systems. We need not just an abstract data model but a clear way to graphically represent that model.

Andy in the news

who knew... Dale, of ooTao said stuff

XDI Update

What a year... I just looked back and saw that the last time posted something that was really about XDI, on this XDI blog, was in March… That’s crazy!! Now in my defense I have posted quite a bit on XRI and XRDS and these are necessary building blocks to the realization of the XDI DataWeb.

So, here’s some of the news and my current thinking…

First and foremost… we have cut the 1.0 Version of our DataWeb Server!! This is the server that we have deployed as part of the Kintera Project (that you can read about in earlier posts). This feat is doubly amazing because of the magnitude of the problem that we are trying to solve and the fact that this year Steve Churchill has been working solo on this project. Steve has performed a Herculean task in building, deploying, supporting and documenting this project… He is a one man team of 20. THANKS STEVE!!

NEXT…

We implemented a plugin framework in our DataWeb server that lets anyone build plugins to access legacy data stores. It works great BUT it is something we made up. We are looking at replacing our plugin framework with Higgins IDAS (Identity Attribute Service). IDAS provides a ‘standards based’ interface definition for ‘Context Providers’… plugins to access legacy systems.

I have started thinking about the qualities that are 'lacking' in IDAS in order for it to be able to replace our plugin framework.... not that it does’t do, what it does, well... just that there are other things that it 'could' be made to do... that it doesn't now.

With the assumption that IDAS implementations sit 'close' to the underlying systems, on the same LAN, caching should not be needed, at least for the classic network latency optimization considerations. Caching could be used for fault tolerance and system failure scenarios but that's a whole other issue. Caching can reduce IO but the problems of keeping that cache in synch far outweigh that consideration if we solve the other problems that I talk about here. In theory, the data is 'right there' so duplicating it SHOULD not be necessary.

What we do need is the ability to 'find' stuff.... Find all of the Digital Subjects whose home city is 'Oakland'. What you DON'T want to have to do is:

1) Traverse all Contexts to see which expose 'Home City' attributes about their subjects

2) Traverse all Subjects in the identified Contexts to query and test the Home City attributes

While caching would improve this problem it is far from a good solution... we don't want to be doing mass traversals, ever, at query time.

What we want to do is pre-determine which attributes are going to be 'search criteria' .... yes.... you MIGHT want to search on any criteria, in which case you have to take the hit of searching without an index... they haven't even solved this in RDBMS world... you can build SQL queries that take days to run and then add a couple of indexes and run them in minutes. Once you have determined the uses cases… add the indexes. (compound keys and simple ‘set’ math across multiple indexes can give you significant flexibility and power)

Executing a search against an index results in a list of pointers to Subjects that meet the search criteria. It should NOT result in a list of pointers to the attributes themselves… remember you are unlikely to query… get me all of the home cities for all of the people whose home city is Oakland… You probably want to query something like; get me the email addresses and names of everyone whose home city is Oakland. (We do NEED to support ‘complex’ matching logic… startsWith, endsWith, greaterThan, beforeDate, etc…)

The next problem is optimizing the data access… The ‘easy’ way to process the results is to iterate over the list dereferencing the pointers… our experience has shown that this royally pisses off the DBAs…. What I mean is, if the ‘Context’ is an RDBMS then the iteration approach results in executing “SELECT email, fullname FROM people WHERE userID = ‘XXXX’” as many times as there are results in the set. This is slow and, as I said, not popular with the DBAs. You need to be able to package your query into “SELECT email, fullname FROM people WHERE userID in ‘XXX,YYY,ZZZ,ABC’” and then parse the results back in your ‘client code’. I put ‘client code’ in quotes because I don’t mean that this is done by the application coder but it should be done at the IDAS implementation. As an application developer I want to be able to say to “IDAS…. Get me all of the emails for people that live in Oakland and get back a list of emails” and never have to care that half of the emails were in Oracle and half were in PeopleSoft. BUT, I want to know that only 2 calls were made across the network (I have had to PROVE this to our customers in order for them to accept our DataWeb Server, they really care about this).

That’s my first pass at ‘what’ we need to do…. Next we have to work out ‘how’ within the existing IDAS spec :-)

Ohhh… and robust distributed transactional management. I will add others as I think of them.

Wingaa takes flight

After months of background work Wingaa is finally born. Patrick Audley and John Bradley, recently of Cogneto and the ooTao team are glad to announce that Wingaa is finally launching.

Wingaa means 'My name is...' in the Central Alaskan Yup'ik language.

Wingaa will continue the work of ooTao in bringing together strong authentication and user-centric services, making them readily accessible to everyone.

ooTao will continue to develop core technology and provide professional services in the identity space.

OpenID and you

John Bradley posted this great post about how to use your i-name at OpenID sites that don't have i-name support yet... It's a great tech tip for those of us that like to tinker. For the rest of you, don't worry; the new OpenID specs have full i-name support and will be deployed very soon.

Adopting Evolution

In my bitchier moments I have been heard to say… “OpenID; brought to you by people who didn’t want to read the SAML spec”. I truly believe that the process of enhancing OpenID from supporting its original use cases to supporting a wide range of internet scale activities of varying values will eventually see OpenID evolve to be a fully compliant SAML definable profile.

So I have been asking myself; why has OpenID grabbed so much popularity while SAML, a much more mature, academically respected, ‘robust’ specification has been largely ignored by the cutting edge web 2.0 community…. an image came to me that I think might be profound, at least for me, and this blog seems like as good a place as any for me to try to get it out of my head.

I'm imaging a perfectly planned city… you bring together the best minds in social and urban planning and have them design and build the perfect city. Then you ask people to move to it… it’s big and empty and impersonal, its very perfection is off putting and intimidating. Meanwhile, just down the street there is a collection of mud huts with lots of people milling about, drinking beer and having fun. People are flocking to the village and it’s growing rapidly. The urban planners that built the city say; but don’t you see, you will need all the infrastructure that we have built in order to continue to thrive as a community, you’ll need police, medical and fire services, you’ll need schools and water pumping stations. But still people flock to the village to be part of growing something new and exciting. The villagers say; if we need police, someone will step up and become a police man, if there’s a fire we’ll get together and put it out. The inevitable outcome of the growth of the village seems to be a less well planned version of the planned city. It will, by inevitability, have many of the same features, some less well executed and some surprisingly better than the planned city.

To me, and maybe it’s just me, I know I would much rather be part of the village than move into the city. I might not want to re-invent the wheel but internet identity is a large complex and subtle system, like a city, it isn’t a wheel. Internet identity is going to have very organic qualities… I’m wondering if the growth, the evolution of the organic system isn’t the magic source that will actually humanize internet identity… I think it might be necessary that we start with simple organisms that can evolve, branch and each branch succeed or fail based on their efficacy in their ever changing environment. If two teams of engineers looked out over an early version earths eco-system and one designed ‘the perfect organism’ and the other designed an ameba capable of rapid reproduction and innovation which would you bet on for long time survival?

I’m not saying that the SAML community isn’t also receptive, open, innovating and evolving, they are. I repeat my original statement that I do think that SAML is more mature and ‘robust’ than OpenID… I’m simply trying to understand the juju that OpenID has (at least in my opinion).

now you're talking...

Just been sent this link... a must see if you are very serious about social networking

OpenID 2.0 discovery (with 1.0 compatability)

This is how we are doing it in our client libraries... does it look right to you?
(click on the image to see a readable version)

New RPs

Well the first of the Geffen Artist web sites have gone live as OpenID 2.0 (WD 11) relying parties. If you have a little time go login and play around… we would love to hear your feedback about the integration and user experience. The first 2 sites are:

http://www.rooney-band.com

http://www.evefans.com

More news...

I don't think I shared this with you-all...

GEFFEN RECORDS OFFERS SIMPLE SIGN ON FOR MUSIC LOVERS

ooTao, a Leader in Open Identity Management, Partners with Record Label

June 12, 2007 (Berkeley/Los Angeles) – Geffen Records, has partnered with ooTao, a leader in Open Identity (Open ID) management, to make it easier for music lovers to log on and connect with all their favorite artists, such as Snoop Dogg, Mary J. Blige, the Cure, Nelly Furtado, Trevor Hall, and Ashlee Simpson. The new “Single Sign-On” launches June 18 and will be expanded to include other Geffen artists as well as new services for fans.

Lee Hammond, Director of New Media for Geffen Records, has been working with ooTao President Andy Dale for almost a year to make it easier and more secure for fans to log on with a single user identification, or i-name. Says Hammond, “We love the promise of OpenID to reduce registration barriers for newcomers to our properties as well as make it easier for current audiences to crossover to new artists’ content.”

Artists love the new feature, too, said ooTao’s Dale, because music lovers no longer have to sign off and sign in again every time they want to see what’s happening with an artist. Using the same i-name, says Dale, fans can sign into the Geffen site and read about and listen to the music for their favorite artists without leaving the site.

More is in the works, adds Dale. “After this initial phase goes live on June 18, we expect to continue working with Geffen to leverage these identity standards and bring music lovers truly innovative services that allow them to get closer to the music and the artists.

ooTao (ooTao.com) is an engineering development company specializing in distributed data sharing and identity management infrastructures. Its founding partners are leaders in developing standards for Open Identity management.

#####

Expediting XPP

I am hoping to quickly and painlessly work up a basic 1.0 spec for XPP (XRDS Provisioning Protocol). This should be simple and easy to implement. I am going to drive the process and try to get the spec done in the next month… then people can use it, or not.

If you want to be a part of this open process add your name to the people section on the front page of the xpp wiki at http://xpp.seedwiki.com. I will organize a conference call for the middle of next week for anyone that wants to play with me.