Spying On Myself (continued)

I wrote a post about this topic back in early 2006 and have returned to it a few times since. There was a short burst of startup activity in this area back in 2006. It was dubbed the "myware" sector, a reaction to the spyware era of the early part of this decade. But it did not get any legs and most of the people working in the sector have moved on.

But I have not. I still spy on myself and publish my activities publicly where the tools exist to do that.

Here's a page that shows what music I've been listening to recently.

Here's a page that shows what I've been watching on TV lately.

And there's a blogrollr widget on the right sidebar of this web page that shows what blogs I've been reading lately.

I do this for a bunch of reasons. First and foremost, I am interested in this sector of implicit behavior data. I believe that publishing the things I do on the web will allow web services to get smarter about me and give me better experiences. And most of all, I want to control this data set.

There are so many web services out there, from Google on down, that have some or all of this kind of data about me already. The idea that we are going to surf the Internet privately is a non-starter in my mind. But we should own this data, we should be able to edit it, we should be able to determine who gets access to it and why.

Here's a small but enlightening example. Yesterday, I left my iPod playing overnight. When I sync'd my iPod this morning, all those tracks were scrobbled to last.fm. I simply went to last.fm and deleted them like this:

delete lastfm.egg on Aviarydelete lastfm.egg on Aviary.

Blogrollr, the tool I use to record all the blogs I read takes it one step further and allows me to blacklist entire domains.

Of course, there is the potential for embarrassment in doing this. When I was doing some work on Gawker for a post I wrote a few weeks back. I thought I had blacklisted the fleshbot domain, but only blacklisted the straight version of the domain. That led to an amusing email with a regular reader.

I believe there are many ways to protect users from these kinds of embarassing situations. First and foremost, most people won't want to publish what they do on the Internet like I do. Some will but most won't. But they will want to see the activity themselves, edit it, protect it, and be able to share those feeds with web services that can give them better services as a result.

Imagine if you had a single data feed, fredsactivity.xml, that was hosted on the web and you could share with web services. I'd give it to Amazon to get better recommendations. I'd give it to Google Reader to find interesting blogs to read. I'd give it to Twitter to get better recommendations for people to follow. I'd give it to Netflix and Fandango to get better movie recommendations. I've give it to Goolge to get better search results.

I still believe this is going to happen. The burst of activity a few years ago may have stalled out, but I think that's a temporary pause, driven largely by the fact that the mainstream Internet user wasn't ready for this. Maybe they still aren't. But I am. I expect that a few of you out there are as well.

Reblog this post [with Zemanta]
#VC & Technology

Comments (Archived):

  1. timraleigh

    A single (editable) data feed would be excellent! I would love it, for now I will have to resort to creating one with a blog.

  2. RacerRick

    Think of all the things that connect us to the Interweb.Fredsactivity.XML would need to run on a lot of devices, in multiple locations to be able to track everything.

    1. fredwilson

      i agree. last.fm’s scrobller started out only working on my itunes, but overtime it has become available most every way i listen to musicit will take time

  3. Smashing

    Lifestreaming will go mainstream. It’s inevitable. It’s like all other changes in social behaviour. It was weird once to see people talking into headsets as they walked down the street. It was weird once to see people wearing headphones. The notion of controlling the information out there is nice, but caching services and archives mean it’s hard to put something back in the bag once its out there. I like the idea of mass overload of personal information making it an ordeal for anyone trawling for stuff. 10,000 Twitter messages would take quite a while to go through…

    1. James

      The idea of “overloading” data so that it cannot be usefully analyzed tacitly assumes that there are people in the loop. The reality is that the data mining and analytics software for this type of thing are relatively smart and fast, no single individual could ever hope to produce so much information that it would not be worth the computer time to dig through it. In fact, more data *increases* the detail of the “you model” the data miner has at their disposal, allowing them to usefully trawl for subtle patterns that would not be possible (in a statistically significant sense) if there was a smaller corpus. The idea that you can dilute your model by overloading it with data is analogous to the idea that you can keyword spam the NSA monitoring your communications to overload their systems — the algorithms do not work that way and are not really susceptible to that kind of attack. If you cannot have privacy in any real sense, the next best thing is to have a significant measure of control over your individual profile.The critical bottleneck that exists today on the analytics side is not our ability to create detailed models of individual preferences and behaviors, but to model the environmental context in which individuals exist such that those individual models become useful. Not only is that volume of data much, much larger than data on individuals, but the kinds of analysis required to make it useful for that purpose are more difficult to do at scale. No one is doing this yet, but it is only a matter of time before someone does. It would allow software and systems to be exquisitely tailored to your lifestyle and preferences in a way that is otherwise not really possible.

  4. daveschappell

    Fred – have u checked out time management software such as RescueTime? It monitors all that u do/how u spend time – sounds similar to the self spying concept and happens passively.

    1. fredwilson

      I’ve checked it out dave but I don’t use it. Maybe I should

  5. dickcostolo

    Haven’t you just described what friendfeed is/was supposed to be? A kind of personal content network, the “feed of fred”, that could be plugged into any 3rd party web service for identity and reputation and social graph management? It seems to have instead evolved into its own meta-content destination site. Maybe gnip ends up being what you’re looking for after a couple more iterations?

    1. fredwilson

      I don’t know dick. Neither of those allow me to install software that tracks my implicit activity. The closest thing was seth and stan’s attentiontrust

  6. David Semeria

    You’re braver than me Fred. I certainly wouldn’t want to leave a trail on the web, and not only for the reason that immediately springs to mind…But in a way this already happens. It’s not hard to work out in what direction people are going by looking at their new connections in LinkedIn. Sometimes this can be beneficial: I recently added a new VC to my list of contacts and another VC (who was already a contact) must have seen that and got back in touch.I know linkedIn lets you control that flow, but most people (including me) probably don’t think too much about the implications of letting even partial bits of info seep out.

    1. ShanaC

      I find a lot of this is age and cultural background. Also how much you’ve been exposed to a variety of cameras growing up.I think you might also need to wait until Say someone like the KidMecury and I get established in Families and Mortgages ect and see how it plays out- social structure with those kinds of information is huge. We might want something very radically different when we are very established in society due to the fact that we grew up with some very radical changes. The things you are not thinking abouyt- we take for granted as normal, but otoh-normal begets revision as we grow discomforted with our place in society when we “grow up” out of this new thing known as “emerging adulthood.”Until they get in trouble- or unless they are inundated with a reason otherwise- someone my age (23)doesn’t mind putting randomness up there about themselves. The amount of photoing and tape-recording of experiences pre-internet we’ve had is huge-shoeboxes of them. And our parents shared it among communities of themselves. The Internet just changed the venue and amount. We were young enough to remember having the earliest parts of ourselves stuck on the net, with parental controls.

      1. David Semeria

        I think you’re right about younger people being more relaxed about privacy issues, but the examples you gave refer to information you yourself explicitly ‘put up’ – as you say.Fred was talking about information which is implicitly collected and made available, for example his clickstream or lastfm list.The point is that you don’t always have complete control (at least beforehand) over what gets published.

        1. ShanaC

          I like the some of the concepts set up in Nudge. I choose to think that at least some of the behaviors we are tracking as implicit should be op-in, making them explicit We set up a lot of things as implicit beforehand, and a lot as explicit; We could choose to set these choices up radically differently as a society. This included data collection and how easily we collect, what we collect, and how we inform people of what we collect. The fact that we choose to have such a system is a construct of right now- it can be changed.

    2. fredwilson

      Someone has to show the way!

  7. Cody Brown

    It’s available as a wordpress plugin now (http://codybrown.name/lifes… but I wonder how a company could scale this idea.Can it still be called Big Brother if its self-surveillance?

  8. jmarbach

    Fred, have you experimented with setting up a life stream? (All of you social media profile’s activity aggregated in one stream) http://www.sweetcron.com has been the best software that I’ve used.

    1. fredwilson

      That’s explicit date. Stuff I actually enter into the web. I’m more interested in what I do

  9. Venkat

    There are 2 larger themes here. The first is whether people WANT to maintain a single integrated persona on the Web, or whether they want to compartmentalize. The second is whether the digital you (single point in persona space-time, or a cluster) can really be the economic locus of your own data. The first question is a cultural one, and I believe in general the answer will favor deep fragmentation in persona-space-time. The second is not a technological question but a power-relations question. It is fun to imagine that we ought to own all information about ourselves and control its presentation and use absolutely. Unfortunately, others (individuals and organizations) have an economic stake in shaping our identities. Social identities are, by definition, socially constructed. We’ve seen a couple of decades worth of shift in power from organizations imposing identities on individuals (org. man era etc.), but the trend isn’t going to stabilize at “all power to the individual.” The stable equilibrium will depend on how badly individuals want things that organizations can provide, and how much control over their own identity they are willing to cede.Without these 2 broader themes addressed, I think your narrowly technical definition of the space as “myware” is premature. You cannot define the technical shape of the sector before you’ve committed to a few predictions about the social/cultural power calculus that you think will emerge.To bring abstractions down to the concrete. I think your portfolio company disqus belongs in this debate. It tries to make my comments more in my control, but the equation isn’t that simple. I have a disqus account and am using it right now, but overall, I am ambivalent about a) whether I WANT my commenting personality that integrated, and b) whether I’d rather give more control to selected single individuals (bloggers I trust) OR accept apparently higher levels of control being offered me by an organization I trust less.In general, I think the equilibrium will be a dynamic, locally negotiated one, depending on both personality variations (some people have very strong affiliation tendencies and WANT orgs to manage a good portion of their identities) while others want to be digital survivalists, running their entire lives on their own home-based server behind a firewall, with the ability to completely drop off the grid with the push of a button. Depending on economic conditions, people may be forced to negotiate more/less control.

    1. kidmercury

      absolutely fantastic comment. damn.

      1. ShanaC

        I know. I was going to comment that this is the reason why one of my alter-ego’s has a hard time writing a blog on the web- Reprisal levels are much higher in the Real life world if something goes down. Letting a computer deal with that is way too scary on a social level. This is despite knowing on a pure intellectual level that I would do just fine.

    2. ShanaC

      I’m actually tracking a vertical for its power structures,it’s use of technology, it’s who, what when, where, whys, hows. It’s frightening. Terribly frightening. And Amazing. And wonderful. It’s changed my life.The problem is I am not a totally impartial observer. I am as impartial as I can be- I don’t do huge amounts of intervention. But I am tracking my own background. I once left a comment here that I am more connected with my ancestors because of the internet- it’s true. Because of the Internet I have silent observer status, and sometimes discussion status of both archival sources and discussion lists of sources that can date back ~300 CE, some earlier, since Middle School. I’m also more turned off with what’s happened, because I see power fault-lines in what should be a very lifestyle different structure. It is still hugely important to me, which is why I still check parts of my little world’s press.Looking through the material that I have, looking through what is popular, looking through “real world” action lists- what is old is new and what is new is old. Society is radically redefining itself. Information, previously hidden, is coming forth, and is being placed in constructs previously unseen. A person under the age of 35 tends to make great change for a few years before burning out and becoming Jaded. The young are using the hippest and the best- but they are in a bright shiny bubble for a time, also unsure. it’s like a swing dance of the 1920-1930s- no one really knows how to settle the big changes. They are coming undone and free- in ways I never thought. Almost so free that as they say- their brains are coming out of their head.The old and the conservative repeat wisdom on the web, unclear about how much is still relevant or necessary. They tend to use earlier tools. They pick up on the later ones once they get angry enough, or the leadership gives them permission, or enough people are using them, whichever mixture comes first. The most moderate and left wing among them are starting to align with elements of the radicals. But not all.It’s unclear what is acceptable behavior- so your reputation can be shot big time if not careful.In between- you see how the trend works in practical space. Pays to look at an odd vertical for a very long time. Or come back to it.The trend will stabilize at the point where people feel enough in control of their information and how it is used, and can feel that they can cohesively use it to build better societies. They don’t mind sharing it- they mind its use. Between those two lines is how much and controlling the sharing. They want different levels of sharing, they want different spaces- both physical and virtual, to share as participants, sometimes as leaders, sometimes as followers, sometimes as observers.The huge push though- is controlled sharing through learning. Organizations ect, Fail Utterly if they can’t provide psychological and emotional growth through sharing.

      1. Venkat

        Not quite sure what to say here since you’ve hit a lot of themes at once, but I agree that when faced with big shifting sands of power, a lot of people hold back and try to wait for a pattern to emerge before committing. I am a selective early adopter, and in a lot of things, I join the fray pretty early. When it comes to identity management, I am generally a wave or two behind…sounds evil, but let others figure out what works and what doesn’t 🙂

    3. David Semeria

      Can you give us an example of the “social/cultural power calculus” that you think Fred thinks will emerge?

      1. ShanaC

        Yes- this was the least inflammatory example I could find.Jewschool started out as a blog of progressive Jewish voices about everything on the web. It remains that to this day. It can be over the top, but also very cool, depending on your thinking on the matter. It is often a locus point for many people and events around the country that are important to young Jewish people as they are happening and becoming. It is the site, as I would say, of first press. If you get a hit on Jewschool, you are doing pretty well in life. If you are invited to blog there, same goes.an ex-Orthodox Guy, or as ex,-Orthodox as he can be, named Shmarya Rosenberg, who blogs at http://www.FailedMessiah.com, was chasing down all the information he could about corruption in the Orthodox, primarily Chabad-Lubavitcher, community. (He’s ex-Chabad.) His anger was trying to get him to right a wrong that seems to never be fixable. One of his targets was the Rubashkin clan, out of Brooklyn. They were, up until recently, the largest providers of Kosher meat in the US. He would blog publicly about everything he could. He was reporting since 2004 about changes of US federal Law to accomidate how Rubashkin’s wanted to slaughter.There is an organization known as Hazon. It is a Jewish environmental/foodie organization. In late 2006/late 2005, they start a blog called the Jew and the Carrot. (www.jcarrot.org), which becomes an award winning source about all things Jewish and Food related, especially if you subscribe to the Michael Pollan view of how food works. Hazon though, through organizations, (Including the famous Bikkurim Grant, LimmudNY, and Kehillat Hadar) had been raising awareness about Jewish Food issues since 2000, especially with their capstone bike rides to promote commuting via bike. Between LimmudNY and Kehillat Hadar, both of which organize and promote themselves exclusively or primarily via the web via thier own websites and prominent liberal, youth oriented blogs such as JewSchool.Late 2006, Rabbi Allen Morris, a conservative Rabbi, goes to Postville, the main Rubashikin Plant, with friends, because he wants to see what’s going on in the Rubashikin’s plant. He report’s violates of OSHA regulations, and starts working towards something know as Hescher Tzedek, otherwise known as Magen Tzedek- the Conservative Movement’s OSHA?Animal Rights once in a slughater house certification stamp.The Jew in the Carrot coversthe controversy, with FailedMessiah, in the background gathering information from everywhere. As Hazon, in the background, and from its blog, realizes that the lack of understanding about what makes for traditional values of slaughter, what it looks like, what are the traditional discussions surrounding it, and how to make the issue contemporary, when they start planning their first conference, they liveblog about on JCarrot that they bought a goat that they plan on slaughtering in public and feeding to people in order to explain and have a discussion about slaughter. They then liveblog the experience of what various reactions to the slaughtering of the goat. For many people, this was am the most informative experiences they’ve had about practical kashruth in their life.Controversies about meat continue-nothing changes until May 12, 2008. Federal officials went to Postville and at the time, did the largest illegal immigration raid in the history of the US. it made the front page of major media outlets and those in the active Jewish World stopped breathing for a second.A group of liberal YU and Yeshivat Chovei Torah all orthodox students in Washigton heights Manhattan, immediately organized a boycott through a new organization named Uri L’Tzedek, over the web, which was publicized through blogs such as Jewschool, and JCarrot. They stopped eating meat and chicken until management was changed or its certification was pulled. Their actions were supported by others, primarily our friend, Shamrya, who was the source to the point of quotiation by the WSJ and the NYT about Postville, because of his continual liveblogging efforts of every single possible note of OSHA, animal, immigration, and slaughterhouse irregularity that the Rubashkin’s pulled. There were shortages of meat in some areas.The Rubashkin’s changed management and got arrested as a result One of their most important certifications got pulled. From very private conversations of people I know who’ve been in the kosher meat business (not Rubashkin), Kashurth in the US is seen as something fragile and delicate, because there are not large amounts of people keeping kosher, and keeping people in good supply is seen as an imperative mission, above others. Forcing a pull of management was seen as more ideal. The Jewish food movement got heavy coverage by the mainstream media.This in turn caused two things to happen: One: those kids felt empowered. Uri L’Tzedek, recognizing on some basic level, its place withing the Jewish world as a source of authority since it has some sense of Orthodoxy about it, has spun off another organization: Tav HaYosher. if a restaurant is Kosher, Tav Hayosher will also certify that they are OSHA compliant. The founder has been slightly harassed, despite this, he is speaking about this program in very mainstream institutions. (list available here:http://tavhayosher.wordpres…. It is run exclusively by volunteers,because they are finding that paying for certification corrupts who and what one is looking out for. This was part of the backlash against Kashuth and ethically inclined Kashruth as it stands today. They are getting a large amount of volunteers to chck in, learn languages, teach about tip lines, they are having problems finding people with financial acumen to look through restaurant’s financial books. (Ideally they should also have someone who can do analytic for them….)Hazon is still very active. Signups for thier 2009 conference have started, and as usual, it will be packed. Their annual bike ide is as usual, Labor Day weekend.Jewschool is still a major press point- right now there is a lot of convergence over a conversion crisis and somewhat over Peace Now’s database being released to Haaretz.You see a lot more people who are abstaining from meat, or who care more about organics, or who are reading about the subject. Thier kids pulled them along, they heard about Hazon and these blogs from the Papers! ect, and it’s become important to thier religious practice to be careful about food in new ways, in a much more varied community.Two: Meanwhile, there is a backlash against this system: The most conservative elements of the community are scared: They are fully aware of the changes that are happening, and don’t like them.This was in response to the raid. It was set up by some people in Brooklyn, it got about twelve thousand signers.http://www.thepetitionsite….This is from an even more right wing source, they moderate heavily, and don’t talk to anyone. They are considered a mouthpiece for some of the most extreme elements of the Orthodox community:http://www.cross-currents.c…This guy is odd- he’s orthodox, Takes some very right wing positions, and some very left wing ones. Very Iconoclastic in his own way- Very heavily followed, one of the top bloggers for what he does. Note the fact that he also is censoring because of the cross between drive for change, and drive for censorship and bowing towards more of a traditional realm:http://hirhurim.blogspot.co…Not that censorship doesn’t happen on the liberal end of this world- you just get shut out by being drowned out-not explicitly being taken out of the record.That’s the difference. And hence the desire to share and create these little organizations not defined by physical space, but by these relational spaces allowed for via the web.They tend to collapse if they get too big and not everyone is sharing/participating. At that point you need to bring in some sort of professional structure. I have examples of that- but this is getting very very long.It pays to follow a niche and what they do and how they do it. How they behave in real world terms, etc.

        1. David Semeria

          Just for the record Shana, my reply was to Venkat’s original post, which is also where the quote came from.Since I know absolutely nothing about the topics you mention in your reply, I’m afraid I have nothing to add, sorry.

          1. ShanaC

            Does it matter who can bring forth to the table of where social/power constructs can come from?It matters that it is provable- that massive powers that used to control identity are being brought down, and that one can trace a trendline based on their histories.It matters very little that you know nothing, or very little, on kashruth- it matters that this is the second, third and fourth version of the exact same story I can tell within a niche I have been tracking for 6? years.It answers the question of power, society and its upheavl on the internet- and what people are doing with it.Choose a niche, and trace its social history. Choose the tools they use and why they are using it, and what they do with it- and see what you come up with. It is a total shocker.

          2. kidmercury

            yes! fantastic point. thanks for the examples.

    4. fredwilson

      This comment gave me a headache. The good kind. Thanks for that

  10. Jan Schultink

    The killer ap might be when you add location information to the stream. A bit scary as well.

    1. fredwilson

      Right. A public latitude. I want that too

  11. Jeff Judge

    I think you’re speaking to the evolution of self syndication, whether it’s self hosted blog (ex: combing the WordPress P2 template with a syndication plugin like WP-o-Matic for grabbing your feeds), or a hosted blogging service like Tumblr/Posterous, or a aggregation service like FriendFeed. I really feel like these are all driving towards the same thing, at least in terms of the ability to personal publishing and redistribution.

    1. fredwilson

      Well I am not really focused on the publish part. I do that to evangelize this idea. What I really want to see is the data made available to each user so they can control it

      1. iamronen

        “,,, the data made available to each user…” – that phrase represents a major paradigm, and I feel that it needs to be challenged.I have a feeling that a way to move closer to fredwilson.xml is by starting over and starting right. The data should be yours to start with. Then you can decide how to make that available to other people and/or services.I think it’s very doable… but it would shake up a few web-paradigms. This is not yet mature … but is relevant to this topic. Imagine a hosted service owned by you: http://www.ontekusuto.com

        1. fredwilson

          It may be hard to start over. Why not try to change the paradigm as we move forward?

          1. iamronen

            Great question. Thank you.Ironically, I feel that current paradigms don’t really need to change. I believe they are a manifestation of the intents of current market forces. If companies are founded with primary objectives described in business metrics – then they will work towards those objectives. Most of the market players are not about you (or me, or anyone else) and your control/access of your data. They are there in the first place to trap your data (in a friendly way) and manipulate it for a profit (maybe they can profit by selling you access to what is yours in the first place!). If controlling your data happens to align with their primary objectives then you are in luck. But it’s luck, not pro-active measures or intents.I look at products/companies such as WordPress, Mozilla and at the people leading them and I get a good feeling all over. They are committed to something that goes beyond their products. I recently saw a video of Mozilla’s CEO John Lilly speaking at WordCamp SF saying he doesn’t give a shit if people use Firefox or not, what is important to him as that they make a conscious choice about what they use. WordPress has been recently flooded by great controversy about the GPL license, again – focusing on personal freedoms (and the sacrifices required by developers who work for for free in the first place, to insure those freedoms). Damn – that is some powerful introspection and commitment. To me these are the foundation of the businesses of the future.I have great faith (no proof or research to back it up) that the future of business (including successful business/financial metrics) will actually be found in commitment to higher values then what seem to be the current norm in technology. Matt Mullenweg of WordPress once said/wrote that as long as he can afford $8 a month – he has complete control of his place on the web. There is a simple & straight-forward business model. 100 million people paying half that for that same freedom while still enjoying all the benefits of social-networks – that’s an ecological and sustainable $4 billion annual market, without any advertising (and assuming of course that the free-fad blows over).Current business practices (in many sectors – not just technology) are abusive of people (sometimes as a side-effect, sometimes as a strategy). I believe that can change and not at the expense of, but to the advantage of businesses. But, like I said, that will take some major shifting of perception on what business is and where it meet’s us in life.On a personal note – Yoga can be seen, amongst other things, as a process of improving personal energy by collecting and focusing it (http://www.iamronen.com/?p=…. I subscribe to this approach. Currently technology is, on a personal level, a disturbing, disruptive, noisy and polluting experience (I am not using these words to denote a negative context – they embody a descriptive quality that goes beyond judgment) . This has a highly dispersive effect on personal energy. I believe technology has a unique potential to facilitate an opposite effect.Intent, purpose, care…

  12. John Engler

    Fred, you should check out Rescuetime. I use it to see what applications I use, and what websites I spend time on. For example, so far in July, I’ve spent 20 hours in email, and 9 hours in Excel. Insane that I’ve spent that much time in those two applications alone in two weeks.

  13. ShanaC

    Amen- I’m reading a book that’s starts parts of these arguments way back in 1994 through Eyebeam Atelier as well as Donald Norman’s books (BA research)When people want to give up control they want it to be subtle, discrete, and unconscious, and they want ways to regain that control on conscious and reflective levels quickly. The best example is that you want a fridge to maintain your shopping list and send it to your grocer while help maintaining your diet, but when you pick up your stuff from the grocer (and some of us like doing that) you want toa) be able to add some ice cream and break your diet- heavy on the organic veggies.b) re-evaluate those peaches on your list because they aren’t ripe enough, but the plums look better and are cheaper.c) think about a dinner party you are going to have next week and the sales that may come up and throw that information back to the fridge. You want to note all of this in advance of course, because you have a close friend who apparently just got diagnosed with Celiac’s and can’t have any gluten.The fridge doesn’t know better. You need to. Your IPod, and Last.FM to a point can’t tell the difference between you, your family, your friends, and random guy down the street. This muddies the water of how much control to give a device when you want it to work cohesively with you to do something.Meanwhile on the level of projective devices, people run scared. We all maintain multiple identities. And as much as I can see certain parts of you from this blog and from say your last.fm list, to gain the most from them, you would again have to take Google’s or Amazon’s perspective- discretion. With that comes a very specific price- the ability to confuse you with someone else. The deepest parts of your identity is the stuff you hide away and only partially signal to us, and therefore, those design self-spyware or other-spyware need to think about your whole spectrum of behaviors. A computer is emotion and identity blind at this point in history, so what behaviors are signaling to your interests, beyond those that you say are public?. Even if we could send out all of those signals, we could easily trick it away from the deepest levels, which is why I continually get some very funny examples of odd results from Amazon and Google in their plugged in Ad-sense type software.To truly get your music taste, I would not look just at your last.FM list- too public, too much signaling about what you want us to think about you. You mentioned records, when do you play those, which one of those do you buy, and how often, and which of those hold special meaning? Provide special software to discretely track your records (if you buy records.) The thing is, how would a software know which behaviors are a mixture between public and private signaling and get at the true you? Most humans make mistakes; personality and behavioral research of this type is really in its infancy in some serious ways.That being said- the first person to take seriously a large amount of the research about personality, culture, about emotion, design, object, body,ect, and implement on a serious level will be extronidarily successful beyond our wildest dreams.

  14. adamwexler

    I absolutely agree publishing information about yourself will lead to a richer experience on the web.I think you hit the nail on the head when you described what happened with your iPod shuffle and last.fm’s audioscrobbler. Why should you be forced to take the time to remove the data? That’s an unnecessary hassle.”And most of all, I want to control this data set.”I wrote you a note last week following up on a conversation we had last fall. We created a music website (www.gorankem.com) where you can do just that: rank your favorite songs from your favorite artists. if you’re not comfortable “publishing” your rankings for a certain artist, we’ll give you the ability to keep each individual list private. Yet, when you put the collective results together for all those that published their favorites, it becomes a pretty powerful resource for everybody else to use!

  15. álvaro ortiz

    The Attention Profile Markup Language <http: en.wikipedia.org=”” wiki=”” apml=””> was an attempt to create a standar to do just this, but it seems that it didn’t took off.Few weeks ago we thought about the concept of a CulturalGraph, an XML representation of your cultural preferences. The same way the SocialGraph is the representation of your friends online, your CulturalGraph could inform third party services about what you like, what you own, what you wish… without you having to inform each service manually.We also think that sooner or later this is going to happen.(this idea came up as we are creating a service to share with your friends what you have consumed, and we are creating an API to provide and consume this CulturalGraph… lets see where we get! 🙂

    1. fredwilson

      Cool idea

  16. cwanda

    Good point, Fred, about collecting and using one’s own personal data. I would also note collecting library items checked out would be helpful, as well as all places visited with a GPS device (where the cars have been, where one has been with a wrist mounted GPS, or cell phone GPS). It is spying, but with the consent of the spied on person(s). If one is to supply vehicles for a family, why not have a tracking capability, fully disclosed? If junior wants to drive the corolla, fine, but it gets tracked (esp. while under 18).

    1. fredwilson

      I’ve always dreamed of geotracking my kids but they simply laugh at my silly ideas

      1. ShanaC

        As a society, we already do geotag certain people: criminals.I think your children are right to be resistant. They recognize their autonomy. Smart kids.

        1. cwanda

          Criminals are all someone’ kids, right? Being someone’s kid is not inconsistent with being a law breaker.Autonomy is fine with one’s own assets. Autonomy with M&D’s assets, in ways not allowed by M&D, is a sort of stealing/misappropriation, yes? If this happens often, and grounding is impractical, why not geo-tracking?Tracking assets (cars, phones) is not quite the same as tracking a person.Tracking by GPS also allows speed checks. Once sonny gets 3-4 speeding tickets, maybe it is part of being fair and responsible to society to insist on a GPS in the car that gives a message to M&D’s cell phones when sonny’s corolla (actually M&D’s corolla) goes 85-90 down the highway.

        2. kidmercury

          yes. anyone/anything that has an RFID microchip is being tagged. some parents are giving their kids RFID bracelets. plenty of companies are embedding RFID chips in clothing so that they can track you. there are probably products in the home of everyone who is reading this that has RFID chips in it (unless you are a privacy fighter who is looking out for RFID and avoiding it accordingly), people have their pets chipped, alzheimers patients and older people are getting chipped, military soldiers are getting chipped.i’m definitely not hating on RFID or geotracking, both are immense sources of value that i encourage entrepreneurs/investors to cultivate. however we need to be cognizant of the kind of power these companies have and demand that they behave in a way that earns the trust we are giving them.

  17. MikePLewis

    This is music to my ears. I’ve been dreaming about this too. I could see it going even further:When i look at the web, i see people trying to capture experiences. They capture photos on flickr, videos on youtube, and notes with people on email. Their life is being tracked but not in a comprehensive way.I could imagine a site – call it “Lifetracker.com” which tracks all the things you do. You plug in last.fm, gmail (or other email), google voice, flickr/picassa, twitter, credit card (mint), youtube and other web services. I then matches 3 things: the data, the contacts, and the time. It creates a timeline for you and marks who you’ve been interacting with and when. There’s an API so each new web service you start using you can plug into it.There are several benefits: (1) as you mentioned, you can give this data to services for recommendations; (2) you can search your life. If google is web search, twitter is real-time search, this would be “me search”; (3) just like we don’t remember phone numbers anymore b/c we put them into our phone to retrieve any time we want, we can start throwing information into lifetracker such as meeting notes, audio recordings of phone calls, etc. so we don’t have to write stuff down and remember it. Use the cloud as a memory storage instead of your brainI see this coming and it’s really exciting to think about it.

    1. fredwilson

      I love itOnly one problem, lifetracker.com is owned by a domain squatter called pmr holdings

  18. Morgan Warstler

    The bigger issue to me is the VRM side of this… I WANT to keep a detailed record of my media consumption and my spending habits, precisely so I can obtain more freemium content because I let advertisers & publishers access my records… that I keep and own.Taken to the extreme, imagine the free hooker who comes over and moans “Cadillac” at climax – 2 weeks before your car lease comes do.

  19. symptomatic

    First: great blog and Twitter feed.Second: Whether the services you use are amassing your behavioral data or whether you’re nominally controlling it yourself, you’re still feeding various marketing databases.Personally, I don’t think marketers ever have our best interests at heart—regardless of whether they can help you find new music or new TV shows or new films. Friends can, too.But I’m very curious to know why you think it’s good to put this data out there (however done).Don’t you think that, in aggregate, it will reveal things you might not want revealed?

  20. ravisohal

    I think the concept of streaming what, when and where we do things is pretty close. Think about simple location aware services on the iPhone and web services built around it. Amazon/Netflix/other web service rec’d engines have moved towards an “awareness” model – make recommendations not just based on what I have bought, but what I have recently bought, listened to and correlated to my cohorts. Add Disqus, Fb, Twitter into the mix and were just a handful of years away from web services which nit together my web presence for me and updates my web persona for me. Part of it has been waiting for social acceptance (and I think we’re basically there) and the second part has been the technology – that piece is almost there: faster smart phones, smaller net books will help close that gap.

  21. Carl Rahn Griffith

    Serendipity driven by data may seem an oxymoron but it makes sense.

  22. pistolpete

    I really like this concept of user controlled personal data, its the next step forward for data portability. The ability for me to own my personal information and opt into a service which I trust to provide me with a better service. Mozilla has something up its sleeve called Weave (http://labs.mozilla.com/200… which looks like it could be along the same lines, however details are sketchy.

    1. fredwilson

      Thanks for the tip about weave. I’ll check it out

    2. fredwilson

      I love your handle. Pistol Pete was awesome

  23. Tony Wright

    We think the “myware” sector has plenty of legs (our free offering and our business offering have both enjoyed double digit monthly growth for a while now)!The privacy issue is definitely not a small one– we (as product developers) have the burden of providing enough value and enough privacy controls to overcome that– just as the convenience of web-based email has overcome people’s concerns for storing their private correspondence in the cloud.

  24. DaveGoulden

    Back in the spyware days that Fred mentioned (or AdWare Days as we liked to call them at Gator/Claria) we learned a lot about the value of the attention data.We mostly used it to target ads (many getting 20-30% CTR), but also in the later days of the company how to use the stream to recommend content.We built a product called PersonalWeb (think of it as iGoogle, but automated by mining the user’s click-stream data through the use of a browser plugin). One surprising finding was that content modules recommended by the system got a 3-5X lift in interaction versus content modules manually added to the page by the user.At first, we didn’t believe the stats but when we dug into it, it really made sense. People add content to their home pages or apps to FB for all kinds of reasons, curiosity, heard it was cool from a friend, but their interests shift over time and they end up with things that are no longer relevant. Most people are too lazy to manage all this stuff, but the click stream data does a great job of showing what’s relevant at this moment (therefore driving higher interaction).I agree with Fred that user control over the data is key. That’s a big part of where the AdWare space went wrong.Google has a pretty good click stream tracker. Just install the Google Toolbar and turn on “Web History”. They build a page with your history that is easy to search and is stored in the cloud. It’s saved me a few times trying to get back to something I needed.

  25. Jamie Lin

    Agree. I actually don’t care about internet privacy at all, reason being A) I’m no one important and B) for really private stuff, I never get them on online. Therefore, I’m dying for retailers/advertisers to come up with a smart way to analyze my behavior and to provide me info on their new products/services that I might be interested in. But I guess that won’t happen in another 5-10 years.PS. Another observation, from Blogrollr, is you also “spy” (albeit publicly) on your kids, in addition to yourself. I guess parents are always the same. My mom always leaves me awkward comments on my blogs that I end up having to remove them.