Gawk.it

AVC community member Kevin Marshall is the consummate hacker. I can’t count all the stuff Kevin has hacked on since I’ve known of him over the years. And I’ve tried many of them.

A few weeks ago, Kevin emailed me with his latest, gawk.it. Kevin described it this way:

I just hacked together the start of a simple system I wanted to
tell you about.  The idea is a system to let you search conversations
from around the web.

 So far I started with just the AVC comment board, and it’s
super basic right now (going to improve it over the next few days) but
should already be functional (and useful for those of us interested in
the avc community).

Being able to search the AVC comments is probably the single most common request I get when we talk about the comments here. So I knew right away that Kevin was working on something where there was a real pain point.

But I wanted more than comment search. I replied back to Kevin:

What would be great is an integrated blog and comments search

Could you build that?

In classic hacker behavior, Kevin replied that he could and less than a day later, we had integrated blog and comment search. We then iterated on the UI/UX a bit and by this weekend, less than seven days after his initial email, gawk.it was live on AVC. It is live on GothamGal.com as well.

It’s been running in the AVC search field on the upper right of this blog for a few days now. I know that a few folks have noticed it, but I want everyone to check it out.

I think the blog search part (which is the default result) works really well. I think the comment search (in the tab to the left) works OK but is getting better every day.

Give it a spin and let Kevin and me know what you think in the comments.

#Web/Tech#Weblogs

Comments (Archived):

  1. falicon

    Wow. I am humbled…thank you for the kind words. Can’t wait to see what the community thinks of the system. I know they are going to help me evolve it into something even more awesome!

    1. fredwilson

      that is the point. they will come up with feature requests you never ever thought of.customer development in action.

      1. falicon

        πŸ˜€

    2. Mark Essel

      This is solid, props Mr Marshall. I don’t blog nearly as much as I used to, but there is comment gold I’d be happy to search.Prediction: Disqus moves to acquire/replicate this feature within 3 months. But you can potentially work with any comments. Can’t wait to meet up and hear more about your master plan.

      1. whitneymcn

        Kevin’s been tossing ideas and requests over to Daniel at Disqus since I’ve known him. I’d love to see a gawk.it acquhire or consulting gig that starts putting some of Kevin’s work and thinking into Disqus.

        1. Mark Essel

          I <3 Disqus, but I’m not sure they could afford Kevin. He’s priceless.

        2. falicon

          I pester…then I build…sometimes I do it in reverse order…but almost always both are involved πŸ˜‰

          1. Techman

            You actually sound like a good person to work for Disqus.

          2. falicon

            Disqus is one of my favorite companies…but the older I get, the more I find I’m a horrible employee…I think I’m a much better partner than employee πŸ˜‰

          3. Techman

            But you can get paid! Your work is priceless.Subject: [avc] Re: Gawk.it

          4. falicon

            I can make a lot more money, and stay on the east coast, as an employee/developer for a banking firm too…but I would struggle getting up every morning…luckily, for now, I have enough opportunity and freedom to be building what I want, in the way I want, with the people I want…while still being able to pay my bills (having a successful wife helps a lot on that front)…so at the moment, life truly is *awesome* for me ;-)But I do appreciate the kind words and thoughts…it’s nice to have people like, and find value, in some of the stuff I do πŸ˜€

    3. Kasi Viswanathan Agilandam

      From Idea (14 May) to Proto is just 2-weeks. What you a superman ?I think you have left your whole business plan on the net (your blog :-)).Good luck and works great.

      1. falicon

        Thanks! It’s the old consulting adage at work…I built it in a day or so, but it took me a career to create…Advatange of being my own peersonal betaworks…lots of code/history/thoughts to cull from πŸ™‚

    4. ShanaC

      Pretty good, so far better than lijit.

      1. falicon

        Thanks! That is who this really disrupts at the moment (they have pivoted into being an ad network anyway)

        1. fredwilson

          and they sold out which usually means no more innovation

          1. Doug Kersten

            I remember emailing lijit when they first started to say they should do something like this! Pretty ironic.

          2. Todd Vernon

            Search was a great way to form a relationship with publishers over the years. We grew a substantial network through relationships which was a really fun part of the company. Ultimately, however search as a business is a very difficult entry point as the technology demands of doing it at scale are very large, and the monetization vector is difficult as the market is owned by Google. Lijit ultimately took off when we took those publisher relationships and helped with their real problem, monetizing their ad inventory. Lots of innovation ahead, but we are focusing on the pain.With that said, Gawkit is really cool. I love the comment integration. Pretty slick Kevin.

          3. falicon

            Thanks! I’m sure I’m going to hit some interesting challenges as I attempt to expand out to more and more blogs and commenting platforms. BTW – I am impressed with success of your pivot…I know a handful of publishers that are getting really good fill from your system and are very happy with your performance/service. Looking forward to watching you guys continue to grow it as well.

          4. fredwilson

            note to pivoters. please tell your users you are pivoting so they can also pivot away.

          5. fredwilson

            it feels like a bait and switch to this publisher Todd.i don’t give a shit about monetizing my inventory to be honestbut i care about search a lot

          6. falicon

            A lot of publishers probably are concerned with monetizing their inventory though…but I agree it’s a completely sep. issue from ‘search’ and the two should only loosely be tied together.I believe search is an incredibly important and useful tool for readers and engagement … but publishers can get a lot of value out of it too ( for one, the analytics behind it can be a great tool to help a blogger better understand their reader’s interests ).Is there money in it for the blogger? I don’t know…probably…but prob. not enough to be the motivating factor for including search as a feature on most blogs ( if that’s the selling point for adoption of something like gawk.it, I think its’ going to be a an uphill battle ).That being said, I do have some ideas I may eventually incorporate into gawk.it for publisher to try and monetize a little bit…to start, the DDG integration for when no results are found has a tiny revenue stream attached to it…I track which blog searches are generating that traffic/revenue, and my intention is to pass that revenue through where possible…I also think I’ll eventually let each blog customize if/what ‘ads’ or content/features appear in the sidebar navigation on results and pass that revenue through as well (again as much as possible as I figure it all out).By itself, it prob. won’t be a homerun in the monetization route for publishers…but it might be a nice added bonus for some.Still, the main reason for a blog to add search to their blog is for the readers and the engagement it helps to foster…that was my motivation behind gawk.it and that’s where I intend to keep my focus as I grow and evolve it.

          7. ShanaC

            But how do those tools make money for providing that service. I suppose you could have a cheap pay for play type model (eg: comments cost extra)

        2. ShanaC

          You’re welcome (not my fault they stopped innovating)

    5. Dave Pinsen

      Congrats, Kevin, nice work.For sports fans in the AVC community, be sure to check out another of Kevin’s recent hacks, Turfd.

      1. falicon

        Thanks!

      2. JLM

        .Damn this is………………………………………………….good!All things Texas Longhorns in one place. Well played!On Earth as it is in Texas!.

    6. Aaron Klein

      Nice job Kevin! Too cool.

    7. Fernando Gutierrez

      That is great, big congrats! now we have no excuse to just say things like ‘someone said something the other day at avc but I don’t remember exactly’. Quotes are now just a search away (btw, in additoin to the read and engage links, a button to copy permalink to the result in the clipboard would be great).

      1. falicon

        Thanks!I had a pre-existing note to include sharing searches and results out to social networks…but the permalink idea is also great. Adding that in. Thanks!

    8. andyidsinga

      Just tried it- punched in my own name because I’m always looking for a new way to egosurf ;)Nicely done Kevin!

    9. andyidsinga

      BTW – any product with the word ^g(r|a)[0-9w]{0,}k$ in its name is automatically a must look for me πŸ˜‰

      1. falicon

        sweet! I think you forgot a set of parenthesis there…but I grep and appreciate the point. πŸ˜‰

        1. andyidsinga

          doh – bragging fail πŸ™‚

          1. falicon

            heh – actually, I wasn’t completely sure after I posted my reply as well, so I tested it out real quick…and it does work (but would be more accurate/forgiving if you left off the $ at the end so that it could match the .it as well ;-)So I award you full credit…and my apologies πŸ˜‰

          2. andyidsinga

            dude – you are awesome for coming back and giving props like that ! Exactly what I love about the folks on this board! Cheers to that!

  2. jason wright

    what about a system or a feature of this system that sends a notification to a user whenever a previously saved trigger word is typed on avc, either in a daily post, or in a comment or reply to a comment or reply to a reply?

    1. falicon

      Thanks! saved searches are def. on the list…that is a core idea behind my turfd.com project and so much of that code will find it’s way into gawk.it once it’s a bit more perfected over there πŸ™‚

  3. Julien

    @falicon:disqus I tested with “gawk.it” as the search term… and well, no result. I guess it’s just a matter of indexing speed? Have you considered using PubSubHubbub over the feeds? Both Typepad and Disqus offer them and we’re trying hard to make them real time!

    1. falicon

      v1 is cronjob pulling…the job only runs once an hour so this post/comments haven’t been picked up yet. I will speed it up before too long (via push where possible)…my process is always 1. make it work, 2. make it good, then 3. make it fast/scale.gawk.it is somewhere between #1 and #2 right now πŸ™‚

      1. Julien

        Of course πŸ™‚ Just wanted to throw that idea so you have it in your radar!

        1. falicon

          yep and thanks! the more people requst a given thing, the faster it moves up my priority list…

          1. Kasi Viswanathan Agilandam

            RR πŸ™‚ …. Request Ranking

          2. falicon

            I’ve owned the domain storyrank.com for many years now…eventually it’s going to be the ‘algorithm’ behind something like this…at least that’s what I keep telling myself each year at ‘renewal’ time πŸ˜‰

          3. leigh

            this is just a reminder πŸ™‚ (i think i made almost every mistake you mentioned in that post by the way, and have bookmarked it to remind myself :)”The biggest of which, I think was reacting to what people were telling us in the early days instead of understanding/accepting what they really meant”

          4. falicon

            Thanks! It’s a really easy trap to continue to fall into…especially the developer side of me and the ‘sales’ side of me…I always want to solve stuff for everyone…but sometimes, the solution is just in revealing the proper problem πŸ˜‰

          5. awaldstein

            Agree…Great salespeople and developers listen hard then give customers not what they were asking for but what they need.

          6. panterosa,

            @awaldstein:disqusGreat salespeople and developers listen hard then give customers not what they were asking for but what they need.Hustler/Hacker/Designer synergy thru telepathy.

          7. leigh

            I just read this and think you’ll like it — greatest chef in the world says don’t listen to your customers http://hbswk.hbs.edu/item/6

          8. falicon

            great read…reminds me a lot of the stuff Andy Weissman has been talking about around taste lately ( http://blog.aweissman.com )…I think taste, story, and experiences are the real values of the future…almost everything else is rapidly becoming a commodity or at least being pushed towards one

      2. awaldstein

        Sometimes of course Kevin, making it fast is the same as making it work. Might be so in this case.

        1. fredwilson

          speed is feature #1 in my 10 golden rules for web apps

          1. jason wright

            Moses

          2. ShanaC

            Supposes.

          3. jason wright

            proposes

          4. Kasi Viswanathan Agilandam

            nugh … closes

          5. testtest

            perception of speed is more important than speed. but i get/agree with the point.google would agree, they have a real thing for speed also.with the average webpage being 1mb–and increasing in size–it makes me wonder what importance most people place on speed.

          6. laurie kalmanson

            there was something about instagram — here? — recently that said they are fast and they “seem” even faster by sending user signals that things are done on tap even if they are still processing. the user knows the action was taken and the indication expresses that. meanwhile, the hamsters keep running on their treadmills as quickly as possible.

          7. testtest

            perfect example

          8. LE

            duplicate

          9. LE

            “perception of speed is more important than speed”As anyone who has ever snapped a karate gi will confirm.

          10. testtest

            what is that?

          11. LE

            The gi is the uniform. It makes a really cool snapping noise that makes it seem as if your kicks and punches are much faster. Along those lines there are also noises that you make that create the same effect.

          12. testtest

            that makes sense now.

          13. testtest

            the uniform, right? i’ve not heard a reference to it being snapped or anything

          14. LE

            Sound is very much underestimated in terms of the effect it can have. Half of the appeal of certain reality TV shows is the soundtrack which creates the mood and keeps you entertained. Very cartoon like. You can take something ordinary and make it interesting with sound.

        2. falicon

          agree. my initial thought (and constant default) is that over time is way more interesting than real time…search has generally been about sorting history so real time hasn’t been as important, but conversations are (generally) most valuable in real time.I think if your use case is to get involved in the convo, it has to be real time…if it’s to research stuff or catch up on stuff then it’s depth and quality that matters.Figuring out the best of both worlds is the real challenge here (something Daniel and I have been discussing in email as well).

          1. awaldstein

            Not to quibble but while your logic is perfect, the result is not what I would draw.Yes…real time is what makes conversations possible. W/o real time communications is virtual and non emotive. Doesn’t work.Conversations are the new tip of the spear, the lead for discovery and that includes everything that hangs off of them, including search.Time is friction and the closer you close the gab between conversations and the info that feeds them, the greater the opportunity. The more natural the behavior.BTW..I’m really thrilled to see this post and the focus it puts on you. Congrats my friend.

          2. Mark Essel

            I have always seen avc as this hybrid between Fred’s thoughts and rich community interest, feedback, intelligence, and most of all action. Searching it as one entity this way feels natural.Rapid indexing will come, no doubt by this afternoon knowing Kevin.

          3. awaldstein

            Well [email protected]:disqus Per Mark, the clock is ticking πŸ˜‰

          4. Mark Essel

            No doubt Kevin can make it go faster, but his priorities and design criteria are unfathomable, and will no doubt delight us.

          5. Dave W Baldwin

            You are correct regarding speed second to correct. Most cannot grasp when you are talking about ‘Real Time’ that speed will take care of itself. Moving into the arena of “What are folks saying about this?” in the social graph will simply be a matter of a starting point, let’s say somebody said something 4 hours ago about a stock/app/politic/madonna and you get additional comments as the day goes on as you ask for it. Then the ability to sort the comments to your wishes drives the speed even further. It is not so much whether what someone said 1.5 seconds ago vs. the conglomeration of opinions that can be compared to the ‘flash’ thoughts of the user.All of that doesn’t even take into account the gauge on public/market opinion and what affects it in real time….Congratulations and Well Done!

          6. falicon

            Amen.Of course all this is what makes it fun for us techies. πŸ˜‰

    2. Mark Essel

      Plans for message boy to learn about my searches :)?

  4. awaldstein

    Nice!I owed Kevin feedback on this and I’ll play around today.Thanks Fred, avc as an incubator for ideas and a sandbox for creativity is one of the things that moves this from a blog to a community.

    1. fredwilson

      yes. we should build a crowsourcing platform on AVC so we can fund these projects too.

      1. jason wright

        a must.

        1. fredwilson

          once the SEC greenlights crowdsourced funding platforms, i am sure at least one of them will have a distributed model and we can implement that

      2. awaldstein

        Bingo!Would I support with a click helping Kevin whom I know, has a good idea, and could I bet use the support to make the time to work on this? You bet.

      3. William Mougayar

        You have been doing that informally already, to some extent.

        1. fredwilson

          yes, but it needs to be lighter, easier, and more efficient#rethinkVC

          1. William Mougayar

            I remember that talk vividly :)But would AVC be the platform or a gateway / stage to that platform?

          2. fredwilson

            the right crowfunding platform will be distributed so every blog community can be its own crowdfunding platformwhere is @kidmercury when we need him to explain things?

          3. Marcus

            What about something like Flattr for funding? Makes integration with commens/blogs etc supersimple. Maybe not a perfect match, but model is interesting.

          4. fredwilson

            we need to use a platform that has been cleared by the SECi don’t want to do anything illegal here at AVC

          5. Mark Essel

            Where is #Fredland now, I can’t even remember the hash πŸ™

          6. testtest

            it’s not that unusual for platforms to start off as a blogs.the seobook paid community started as a blog. and gigaom was a personal blog.it can be an effective way to start off.

          7. falicon

            that’s the hook for the theme song for everything on the web…”lighter, easier, and more efficient”

          8. Dave W Baldwin

            hmmm… you know I can’t resisthttp://www.youtube.com/watc…

          9. falicon

            OK – if we are exchanging WHO songs…this is my all time favorite (who can resist the entire rock opera actually?) -> http://www.youtube.com/watc… (though I love the one you pointed out too).

          10. Dave W Baldwin

            What’s funny is there is actually a good live recording of Sally Simpson. There are good WHO tunes for what we’re after ranging from “Anyway, Anyhow, Anywhere” to “Going Mobile”. This is Pete’s demo: http://www.youtube.com/watc

          11. Dave W Baldwin

            Have to share what I sent my treasure diver who is finally seeing clear skies coming- better version of this tunehttp://www.youtube.com/watc…

        2. awaldstein

          I wonder if it takes more than a click or two, whether it matters.Anything that stands between a click and my intentions is in essence poor design. Easy to say, really hard to do.

      4. ShanaC

        Is that legal yet?

      5. panterosa,

        Crowdsourcing within AVC would be ultra cool.

  5. William Mougayar

    I love it. The results format is clean and simple. This has potential.Β Can Disqus offer it as an Add-on feature say for x$, and split it with Kevin? Almost like a Disqus Store feature.Β What technology stack are you using?

    1. falicon

      at the moment doing it for love/interest and not money…and keeping Daniel in loop on my thought process…prob. annoying him with the email flood in his inbox :-)The stack is really just Python stuff (tornado) and Mongo…the search is currently powered by whoosh but I am actively monitoring that as it’s a ‘unique’ choice for search at this scale (wanted to do something more interesting than Solr or lucene to start)

      1. testtest

        why did you choose mongo, kevin?

        1. falicon

          I love Mongo…I first starting dabbling with it back in 2010 when I was doing a contract gig for bit.ly (btw – see their redesign today? Crazy stuff!)…I was so impressed with how well it handled the scale of bit.ly and how easy/clean it can make your DB life…I have basically moved all my stuff in that direction since (prior I was doing a heavy amount of PostgreSQL).In some of my consulting stuff, I still have to dabble with MySQL, Oracle, and MSSQL, but when I’ve got full control I’ve been going almost all Mongo these days.Also for gawk.it there is actually very little db interaction at the moment…it’s really just being used as a poor man’s queueing and log system for what blog posts I’ve indexed or not…and holds the account details (but very few people will actually need/create accounts since the core of the service is just fine in the ‘logged out’ state)

          1. testtest

            wow. guess they weren’t going for subtlety in the redesign.ah, i see. thks, interesting to know.

    2. Mark Essel

      Bonus points, had th same question, and haven’t heard of whoosh before.

      1. falicon

        There are always so many interesting tech. things I’ve never heard of…half the motivation behind each of my projects is just because I have some ‘new to me’ tech. thing I want a reason to try/play with…That being said, I had used whoosh with knowabout.it as well for a bit (my knowledge of it didn’t handle the scale of that system well enough though so I quickly switched back to raw clucene [with ZeroMQ routing the interaction])…I never liked not getting control of it though, so this is round two with me and whoosh…we shall see who’s on the mat after this one! ;-D

        1. Mark Essel

          πŸ™‚

  6. Guest

    I think what is confusing is that you search a term then it shows the search for blog posts and you have to click on tab for comments found.Logically, you would want a term searched to list blog posts first and then comments but I think the issue is Fred is talking about searching blog comments and everyone expects those results to show first.

    1. falicon

      yeah it’s a bit confusing for first times – sorry. the system defaults to comments search when go direct (and you are searching across all data in the system then)…when you use the search box directly on avc or gothamgal, the results default to blog posts and the results are limited to that blogs data set (you can expand to search across all data via the sidebar then)…Does that make more sense? thoughts?

      1. Guest

        Kevin, I figured it out the first two times I used it I was just trying to help others out…Since I tested it via AVC now I am going to go and have some fun with it (well, 14 minutes of fun at least) via gawk.it

      2. Vasudev Ram

        Who can suggest some good terms to search GothamGal’s blog for food items? :)I’m a bit of an armchair / vicarious foodie (for items I can’t get currently) as well as being a bit of a real-life one.

        1. Vasudev Ram

          In fact every two weeks or so I check out a new regional cuisine via Google and Wikipedia. Today I was reading about Sicilian and Italian-American cuisine πŸ™‚ – got to them through a chain of serendipitous links.https://en.wikipedia.org/wihttp://en.wikipedia.org/wik…Bon appetit!andLe chaim! (learnt that one from an Israeli friend. It means “To life”.)

          1. Vasudev Ram

            Thanks, will check it out πŸ™‚

          2. Vasudev Ram

            Tried it. Interesting results. Also:1. (On the quick first look), a possible issue: it seems to show some avc.com results as well – for the query you gave me.2. You are stripping out images from the results? Nothing wrong with that, and I did see the results have links to read the original posts. Just asking.

          3. falicon

            hrm…when I click the link I only get results from gothamgal…but I’ll do some extra testing to make sure (thanks).on the results…yeah I’m actually stripping out all the HTML because it’s the raw content that I wanted to index…at some point in the near future I’m going to let the blog owners decide just how much of the content is visible on results vs. having to ‘click through’ to read…

          4. Vasudev Ram

            Thanks for the replies. I will re-check that first point too over the next couple of days and let you know what I find.

    2. fredwilson

      i asked Kevin to make it default to blog posts on AVC search results.it defaults to comments on gawk.it

  7. bsoist

    Great Stuff! I had not noticed this yet, but I just checked it out a bit now. I’m sure this will prove very useful.This might go without saying, but searching by date would be great (I know I would use it). Searching a set of sites would be great too. Instead of searching one site or all of gawk.it, it would be great to specify a group of sties to search.

    1. fredwilson

      yes. in fact i think the option to show results in reverse chronological order would be very useful.

      1. falicon

        speed improvements and then this I think.

        1. Mark Essel

          I want pie with my search, and I want it yesterday :D!

          1. ShanaC

            how about pi?

          2. Mark Essel

            nah, then I’ll just end up back where I started.

          3. falicon

            It is everything and nothing all at the same time…

          4. Mark Essel

            Let’s not get irrational.

    2. falicon

      cool thanks!Actually the system can already let you narrow results to a set of blogs…I just don’t have a good UI for it yet (so for now you have to know how to properly phrase whoosh searches — same syntax as lucene — which is the same as not having the feature released yet)

      1. bsoist

        Understood. Looking forward to seeing it evolve. Python/Mongo is right in my wheelhouse. Ping me if you need a second set of eyes on something.

        1. falicon

          Great thanks! To-do list is growing rather large so will def. be looking for more people to pitch in before long…how do you feel about no pay, long hours, and difficult tech. challenges? πŸ˜‰

          1. Dale Allyn

            how do you feel about no pay, long hours, and difficult tech. challenges?I thought that was the AVC community charter. Did I read it wrong? πŸ˜‰

          2. bsoist

            also right in my wheelhouse πŸ™‚

  8. Esayas Gebremedhin

    I searched for I WISH (of course) and found this: Fred once said: “I wish there were more societies on planet earth where freedom, risk taking, and democracy define the culture”Shana once said: “I wish there were more teaching methods of math that were both proof/theory oriented and visually oriented at the same time”It’s an interesting insight tool. SInce search engines are forced to look backward, it would be very good to have the timeframe in the foreground.Being up to date is a challenge for any search engine.

    1. fredwilson

      what a great term to search on!

      1. Kasi Viswanathan Agilandam

        what a great results he got … those are really really good wishes.

  9. RichardF

    Really great Kevin, just used it to find the BBQ recommendations in Dallas that JLM gave me and promptly forgot to write down!Not sure how you are presenting the results but they don’t seem to be date ordered.I’d like to be able to send the results to instapaper or a gawk.it account.Nice to see you are supporting the Italian economy by using the .it domain πŸ˜‰

    1. falicon

      Thanks!will add the save for later options to the to-do list.I love the .it domains…turns almost any domain into an action verb (worth the extra money)

      1. Mark Essel

        oh hey, I have yousaid.it if you like.

        1. Kasi Viswanathan Agilandam

          wait until he gets the funding … he will pay you more :-).

          1. Mark Essel

            I had no intention of charging him πŸ™‚

          2. falicon

            best comment ever. πŸ˜€

          3. Kasi Viswanathan Agilandam

            @VictusFate:disqus I thought I knew that… when i put the πŸ™‚ symbol …

        2. falicon

          That makes me think of the old Bill Engvall comedy routines…the ones where he would end with “here’s your sign”…good stuff πŸ˜‰

  10. gorbachev

    Great idea.One serious issue though. The search results are sorted pretty badly. It appears to use an algorithm that ranks search results based on frequency of the search term compared to non-matching words in the commentsSo if I search for foo, the first comment that would show up would be:Foo, foo, but foo. Foo.A thoughtful multi-paragraph post that mentions foo once would be at the bottom of the search results.This problem keeps gawk.it in the “interesting, but flawed” bucket for me.Kevin, I would think this should be your highest priority rather than adding indexes. The first obvious improvement would be to sort using likes. Even in that case the sorting wouldn’t really be relevant to the search terms the way “real search engines” do it, but it should be better than how it works now.

    1. fredwilson

      i agree with this critique on commentsblog posts are sorted better in my opinion

    2. falicon

      Thanks for mentioning this! My org. version sorted by date rather than search term relevance…there were many cases were it made ‘finding the comment I was actually looking for’ not good.Ultimately, I think I will have to offer sorting options to support all the different use cases.Moved up the priority list. Thanks!

    3. Mark Essel

      Excellent feedback, how about an advanced search dashboard like friendfeed’s (if you’re familiar with it). Allowed time & qualifying (5 likes) filters.

    4. falicon

      I’ve updated/upgraded some of the sorting features now (especially on the comments section)…if you get a chance I would love to have you give it another spin and let me know if it’s better for your use-case now. Thanks!

      1. gorbachev

        The newest first default is definitely better.The relevance sort looks to still give me short single-liners first, longer posts after a pageful or so.

        1. falicon

          Cool thanks for giving it a go…the ‘relevance’ section is in fact the same as what the old default was (I haven’t started to muck with that calculation yet — but it’s next up once I get some of these other ‘easier’ things off the to-do list).Thanks again for the feedback and your patience as I attempt to implement it all…super helpful!

  11. LIAD

    Nice one dude. Sweet MVP.When you get time, sortable/filterable results would be great.What got me juiced was the sentence “Expand to search all of Gawk.it” – aka ‘Expand to search of all Disqus’, aka ‘I’m building a Disqus Search Engine’.#ThinEdgeOfTheWedge

    1. falicon

      Thanks and good reading between the lines…that is the immediate direction, but I hope to eventually expand beyond Disqus as well to all online conversation systems (and indexing the blog posts is already outside the disqus data set I think — though I really think of blog posts as the initial bit of a conversation).For the short term though, Disqus has the best data set (and API) to keep me busy hacking away on interesting search realted stuff πŸ™‚

      1. William Mougayar

        It’s a separate conversation you and I will have, but don’t make the mistake BackType did when they started indexing the whole commentsphere & went under, doing it.

        1. Mark Essel

          The intent here is the blog & comments there. Seems focused enough, and differentiated. I know Engagio has a solid search utility, does it include the original blog post?

          1. William Mougayar

            We have the original post (via the API of course), but we’re not currently displaying it in the search results (but we link to it). We could display it, easily. Also, we already have a global search but that hasn’t been revealed yet :), across all commenting/conversations and social networks. But what Kevin has done is brilliant in its simplicity and immediate usefulness. He has to decide carefully how to take it forward. He is sandwiched between Disqus and Engagio, but if he plays it right, it will all work out well for everybody.

          2. Kasi Viswanathan Agilandam

            @wmoug:disqus bidding?

          3. falicon

            Yep – I think disqus also has search internal but not released. I have a few ideas that I think fit nicely in the middle…I also have the advantage of it not being a business (so no real overhead to cover, or models/decisions to have to force)Btw – there is aslo nice duckduckgo synergy to be found here too…

          4. William Mougayar

            Whoever ships first starts to win. You shipped, you win. Big Congrats!!

          5. falicon

            I envision a world where we all win (because we are all playing slightly different games)…and I’m going to continue to build towards that world.As long as the users are getting unique value from what we each are doing – it’s all good πŸ™‚

          6. leapy

            Really great tool Kevin. You have a wonderful attitude towards user input too. I would love to add one more point:About ten years ago I ran a start up building a search tool that presented search results as trails of linked data-points as opposed to a google-type “results list”. The premise was that users seek information that hovers around/across multiple pages (or comments, I guess) rather than being contained within a single page instance(s).We didn’t have the scalable compute power that the cloud provides nowadays but it did work effectively on enterprise-scale document repositories.We ran out of funding at the time. I believe that gawk.it could benefit from reconsidering the UX of search results – especially as the content of a comment thread is usually more powerful that its constituent parts.Feel free to contact me if you are interested in exploring further.

          7. falicon

            Thanks so much!I would love to learn more about what you were doing and your ideas around presentation of this data…do you have any parts of it live or anything I could maybe find in the wayback machine? Any old screen shots or docs or anything you could send me? Would love to digest it if you don’t mind sharing ;-)One thing I can say that I’m def. thinking about/working on related to the display of results of comments is around providing more information around the thread/person (ie. how many responses are in the thread, maybe some general details on who is/was active in the thread, etc.)…I think that type of context could really help make the conversation results a lot more ‘useful’…

          8. leapy

            Let me see what I can dig up overnight….

    2. William Mougayar

      If this isn’t a Disqus Search Engine in the making, I don’t know what is.

      1. LIAD

        a lot of Disqus related innovation emanating from AVC. You & Engagio being at the fore.#AVCMafia

        1. fredwilson

          when one of our portfolio companies becomes a platform, i relax a bit more

          1. LIAD

            #Chess #GrandMaster

          2. falicon

            BTW – duckduckgo integration is in the works as well…two USV platforms to help power the awesome πŸ˜‰

          3. Techman

            Well Disqus has always been a platform, but now it has matured a lot.

        2. William Mougayar

          Lol.AVC is not just a community, it’s a startup ecosystem, a mini online Silicon Valley of its own.

          1. LIAD

            true that.Just waiting for Fred to announce ‘AVC Incubator’.It will include all standard incubator bells and whistles as well as a $250k *grant* for each startup from the AVC Foundation.

        3. ShanaC

          So what is your mafia name?

          1. LIAD

            the first rule of #AVCMafiaClub is not to talk about #AVCMafiaClub πŸ™‚

  12. Rohan

    Nicely done, Kevin.Iterate until awesome.The Grimster will be proud.

    1. fredwilson

      we will see about thati hope he has his coffee before reading this

      1. ShanaC

        I wonder how much coffee he needs before not needing to eat someone

  13. Ciaran

    Looks like a link from AVC is the new digg/or reddit. 504 Timeout.I’ll check back later!

    1. falicon

      yeah – sorry about that. it’s tougher on a server than being techcrunched ever was!

  14. rfreeborn

    tried searching for Lijit on avc.com (seemed appropriate given the topic) and gave up after “waiting for gawk.it” lasted for over a minute. Not sure if it meets goal #1 – works…oopss….Gateway 504 Timeout error

    1. falicon

      Sorry about that…it should be stable for you now, but please let me know if you continue to see issues (not sure how I didn’t see this comment prior to just now — so also sorry for the delay in response).

      1. rfreeborn

        Boom – snappy response! Was wondering if you *were* thinking about the old Lijit product when you created this?

        1. falicon

          Cool thanks. About to push a number of updates based on all the feedback given here (hopefully later today) so it *should* just keep getting better ;-)I actually hadn’t been thinking about lijit (or backtype which also dabbled in a sim. space) at all when I put this together…I was thinking more about Disqus and how to find interesting stuff in their dataset (specifically within avc, but with the thought of eventually going beyond just the one comment board)…the blog search bit (which is likely going to be the ‘real hook’ to helping this system gain traction) is entirely thanks to Fred asking for it.

  15. reece

    nice work @falicon:disqusalways hacking on something πŸ˜‰

    1. falicon

      living the dream and dreaming to live…

      1. giffc

        +1 Kevin πŸ™‚

  16. Richard

    Nice hack kevin, Boolean logic and proximity searches! (With a great ui/ux.) This is google Achilles heal. Feel free to reach out to me for ideas.

    1. falicon

      Awesome thanks!

  17. willcole

    Awesome work @falicon:disqus . Good luck with the servers today!Also – Results for blog posts should have dates like comments.

    1. falicon

      Thanks! Missing dates bug me too…harder to grep from the page content, but will get it figured out before too long.

  18. kidmercury

    this is awesome and on the right track. by bubble 2.0 standards, worth at least $4 billion. hire a couple engineers and we can double that number. cash it before it pops and retire!lol seriously though i do think this is a viable way of getting a foothold in the search market which as we know is a huge opportunity. looking forward to seeing this product develop!

    1. falicon

      I hate money…but yes, I would begrudgingly accept $4 billion today. ;-)It is def. interesting that search has been such a big, known, powerhouse for so long and yet there is still so many interesting/different things yet to be done with it…

      1. testtest

        you hate money. you hate the promise of a service or a good.

        1. falicon

          I guess I mean I just hate the focus on money…I care much more about the people and the experiences…money is a necessary evil in my worldBut don’t get me wrong, having been super poor and having experienced a bit of wealth, I def. appreciate having money and the lifestyle is allows me (and my family) to have…

          1. testtest

            i can appreciate that.some areas around money are vital though. at least from my point of view. for example business models can be used as effective weapons. and who ever has the most resources can cripple their competition.

          2. panterosa,

            @falicon:disqus I am so interested in your comments on money. I share my passion for work as a more interesting and nourishable goal than spreadsheet and b plan, which I find very dull, and stressful. The content and design for my ‘user’ are really the focus.

  19. ShanaC

    I finally found that missing post college advice!

    1. Kasi Viswanathan Agilandam

      @ShanaC:disqus I am just using this thread to pass on message and nothing to do with your find … i found something you may be interested in …You were doing something on healthcare…. does the following website has any relevance to what you were trying to do? I remember you were talking about genetic ….http://www.geni.com/

  20. Cam MacRae

    Site search seems to be a hot area at the moment, with elicit and someone else whose name I forget both securing funds over the last couple of months. @falicon:disqus couldn’t ask for a better stage to launch on.Not sure that whoosh is going to cut the mustard though, because I can see this blowing up fairly quickly — and you don’t want to “blow up”, while blowing up, eh?

    1. falicon

      Yep – data set actually isn’t that huge right now so good for whoosh so far…at the moment volume of traffic on my puny (ie cheap) server is the real issue πŸ™‚

      1. Cam MacRae

        Be worse if you whack a solr instance on it. Funds holding you back? If ever there was a day to pass the hat around it’s today πŸ™‚

        1. falicon

          Not really a funds issue (yet)…I just always try to push servers to the limit before I upgrade, cluster, or add to them…that’s the inner geek in me being seduced by the technical challenges over the user’s actual needs (bad bad bad I know)…never said I was perfect ;-)I’m also horrible when it comes to money…I love everything about startups and hacking, except money (a trait my wife does not appreciate)…if it was up to me, I would probably attempt to survive on trade and goodwill (wife and kids don’t work well for that though)…still, my plan when I finally killed knowabout.it was to never really ‘go looking for funding’ again so it’s unlikely I’ll be putting my hand out any time soon (actually – pubgears is my ‘real’ startup right now [in the adtech space], luckily we have always been profitable from day one and continue to grow at a nice pace…so I’m lucky to not be in a position to need to raise at the moment).

  21. John Best

    Possibly browser related (Opera) – I searched on “hats” and got two results. I noticed that the scroll bar indicated there was more, so I scrolled down. I got a spinning loading/working timer, then the page jumped back up (but with the scroll bar still indicating more). I scrolled down again, with the same result.It was as if the search (or some other process) was still running even after the complete set of results was returned.Other than that, though its very impressive. More so given the time frame!Will we see a rename to successcon?

    1. falicon

      Thanks will look into the issues asap

      1. John Best

        Cool, not a deal breaker by any means, though. I like the possibility of AVC turning into an indexable business library via Gawk.itPS, I’ve only just noticed it’s falicon, not failcon! I blame too many .gifs.

        1. falicon

          heh nice! Screen name comes from combining two of my passions (the Atlanta Falcons and computers [ie. icon])…just so I could come up with something unique that was usually available when creating accounts with various systems…evolved into my online identy over the past 19 or so years (I think the first time I used it was in ’92 or ’93)…

  22. James Ferguson @kWIQly

    If you want to emulate Fred use the word hypothesis sparingly !I searched on word “hypothesis” , hit one posting from Fred and 119 comments. My hypothesis – that in days of lean start-ups and fail fast ; Fred would have used the word frequently.This struck me – What an interesting idea to create a search engine that returns “phrases Fred uses on his blog but that I don’t on mine” but those that have similar context. This blog comparison engine would drive education and nausea alike and drive SEOptimists crazy, where worlds of fawning and learning rhetoric from our lucid heroes collide into a bespoke melting pot of data.An example would be – “I could care less” versus “I couldn’t care less”Perhaps this is a great idea – and perhaps you give a *^%$” when I don’t. πŸ™‚

    1. James Ferguson @kWIQly

      I should add – GREAT Iniitative – well done ! and good of Fred to cover it

      1. falicon

        Thanks! From the first bit that I got working, I started thinking that it’s a really useful tool to determine what someone has or has not actually covered in prior blog posts…and also to see how involved the pre-existing conversation is around a given topic (which I think can help shape/drive what to blog about quite a bit).Hadn’t really thought about the direct comparison of blog or comments content yet though…def. some interesting learnings that could be pulled out of that (I know in getting gothamgal up and live, I found it fun to search her content for the word ‘tasty’ vs. searching avc for the word ‘tasty’…same word, completely different results…both super relevant if you know the community).

  23. Kasi Viswanathan Agilandam

    @VictusFate:disqus @awaldstein:disqusI searched for ‘social gravity’ of yesterday’s discussion … interesting to note it came up in 2010 and @leigh saying once in 2011 ‘now you are speaking my language’….This is getting real interesting [email protected]:disqus it goes for a toss if there is a spelling mistake in the search term. ( am i asking for a google of disqus?).

    1. Mark Essel

      Hey I remember that term. I had used it to try and describe a feature I wanted to build for a couple of apps in late 2009/early 2010. It would draw attention (and engagement) to topics of shared interest, as opposed to being url/blog or people centric.Arnold brings the term to life with a vibrancy and eloquence that I lack :D.

    2. falicon

      would love to build in spelling and auto-correct/complete features…I’m near the bottom of the list when it comes to the ability to spell so it’s something I badly need as well…I’ll have to look into how to pull that off well (or at least look into finding someone that knows how to pull it off well and then getting them to do it for me)

  24. laurie kalmanson

    #becauseawesome1. this is **exactly** perfect in the idea; the execution will come. disqus shows where a user comments but doesn’t pull the threads. this does.2. let me set blogs/convos to follow, in columns, like tweetdeck/hootsuite3. sort with options: most recent / most commented / oldest4. option to ping when something new is postedthreaded convos from posters i want to follow, updated, timely: a tap away.#livinginthefuturealso, related but OT: i had a poem from the nyer mag hanging on my fridge after grad school –half remembered it the other day, found it in the archives with one keyword phrase. the paper in the older issues is nicely aged in the scanned image.

    1. falicon

      Super awesome! Added all this to the to-do list…updates to follow πŸ˜‰

      1. laurie kalmanson

        Oooh wait there’s moreEnterprise manages many accounts and wants to see those accounts and the streams there Do a pinterest style mosaic view and each box can be a person it a keyword depending in the search More info at a glance than columnsColumn view is more personal more focused on longterm#livinginthefuture

        1. falicon

          Additional views is a great concept…also I think UI/UX is my weakest skillset (well related to this stuff I mean)…so will def. put this on the list, and get someone better than me to tackle it (once I pull them into the project). Thanks!

  25. Kirsten Lambertsen

    Really nice, and fast! My first thought was, I wonder if it will search on userid as well as comment content, and it did :)I did a search on my username and just have a few things to report for you:(using Chrome on Mac) – after running my search, all tabs reported 0 results, but there actually were several results in the Comments tab – since I searched my username, I was looking for a particular comment (a large one) that I made yesterday, and it was not there – I’m sure you’ve thought of this, but sorting by date will be great Are you going to integrate it with Engagio? ; -)

    1. falicon

      So awesome thanks! This is the feedback I was really looking forward to. Feverishly making notes and looking forward to sitting down and hacking on some of these later tonight πŸ˜‰

  26. terrycojones

    Good one Kevin! I always tell people you’re a human rapid-prototyping machine.

    1. falicon

      Hey thanks! I should make a note to get all the public comments indexed/tagged into FluidDB as well…then some *really* interesting stuff could start to be built!

  27. Ricardo Diz

    Very cool and helpful! I had wondered about something like this to help navigate the site…

    1. falicon

      Thanks! My initial motivation was just because I wanted to search all these comments myself…I really had no idea if anyone else would want to use it or find value in it…and I wasn’t sure if/where it would take me…but I knew it would be worth the few nights of work to hack together πŸ˜‰

      1. Ricardo Diz

        Thank you for the hacker spirit falicon! Feedback in the next few days, after I have time to use it a bit πŸ˜›

  28. DanielHorowitz

    Looks good. How do we bring together similar conversations that are happening across the internet in different places/communities via different communication mediums? Can we add a touch of reputation? What do you think of engagio?

    1. falicon

      Thanks! Bringing together conversations is a big part of the concept (system doesn’t really do that yet though)…I love engagio and I think they are evolving rapidly into something awesome (discovery of conversations is a super sweet spot in my interest graph)…I’m chatting with William offline on this a bit, and I’m sure we’ll find a very nice synergy over the next few weeks (in my head at the moment, engagio is great for staying engaged in conversations and disqus is great for having/tracking conversations…neither is currently focused on searching conversations, especially from the blog or commenters point of view while on a blog…so I think gawk.it can really help across the board with that specific thing).Since this is just a project of passion for me, there actually is a very good chance that gawk.it can/will be displaced by either Disqus or Engagio before too long though…just depends on how much they want to focus/invest in search and which problem sets they really want to focus on for their users and customers (and how much I can pull off as an independent).

      1. DanielHorowitz

        Definitely valuable…I just searched “term sheet” and the results are good. I’m wondering how to better organize this for the user? What’s the important data to display? How to better rank the results? How to dynamically alter the results based on what people are searching for (This may be dangerous, but it seems like valuable data) Can you help me figure out what I want to search for (i.e. closer to discovery, autocomplete my search term, show more options e.g. google?)On Turfd, I searched for Avery Bradley, and I see that every story is about his shoulder surgery (I had the same one ;)) Rather than display 20 places having the same conversation, can you aggregate this and give me one result that says Avery Bradley injury/surgery, with all the sources (perhaps e.g. techmeme) This way you can better display all the different stories discussions happening regarding Avery Bradley. Perhaps the shoulder surgery injury is the most important and current one and should be prominent and first, but surely there is more. No sense in giving me 10 results that are roughly the same?

        1. falicon

          So awesome. My main thinking is in the same realm right now…For turfd, it’s ‘supposed’ to be doing auto-association of similar stories…but clearly it doesn’t work great yet…I actually have another awesome hacker pitching in on that project right now and we are focusing on player tagging at the moment (it’s a big push we are going to be putting out in about a week I think)…once we get that to an acceptable level (based on our opinions)…we are going to dig deeper into auto-association and de-duplication…it’s a major work in process. So happy you checked it out though and I hope you’ll stick around to experience it’s evolution as well!

  29. Tom Labus

    I like the way Stocktwits segregates comments for each stock. If I comment on Apple it goes into that steam.

    1. falicon

      That’s a tagging thing…I like that a lot too (doing a lot of that within sports for turfd)…will have to think about the best way to auto-tag on conversations. Could be awesome. Thanks!

  30. mikenolan99

    Very cool….

  31. Dale Allyn

    Great work, Kevin (and Fred). Like others, I noticed sort order right away, so I’ll add my voice to those expressing interest in sort control and date range.I know that you’re dabbling with content, but I got several results showing “Is this your domain?” from GoDaddy. Obviously you’re including storyrank.com in your set, but it may not be necessary with all the AVC and GG content. Presumably you’re just wanting to confirm indexing across multiple sources. Just wasn’t sure if you knew it was coming up quite often (like 12 times in sequence, flanked by Fred’s posts prior and after).

    1. falicon

      ah – thanks for pointing this out…the storyrank stuff should have been removed from the master collection…I’ll run a cleanup script on that.The only blogs I have ‘officially’ included into the system right now are avc, gothamgal, and cdixon…I put mine in for some testing but meant to remove it from the master index (flub on my part – sorry)…will be adding others to the system via request (until I can release tools to let users do it on their own without messing with the quality of the system as a whole).

  32. JLM

    .Game changer. This makes AVC.com into an archive of incredibly useful information.It transforms the usefulness of AVC.com from participating in an interesting chat to being able to explore an entire library of truly useful information.This is huge.Well done. Well played!.

    1. falicon

      Wow. Thanks. You just made my day of smiles turn into a wide mouthed grin now…

      1. Techman

        It is always good to see a developers work become a success. I’m proud.

        1. falicon

          Hey thanks! πŸ˜‰

    2. Mark Essel

      You can look up your old pearls of wisdom, and so can I!

  33. Elia Freedman

    Nice, Kevin! Job well done. One comment: the search box is having issues on my iPad. Click into the search field and it loses focus immediately (keyboard pops up and then dismissed). It does this a few times in a row until it sticks for some reason.

    1. falicon

      Thanks…now that you mention it, I have experienced that on my ipad as well…I think it’s related to the javascript used for ‘endless scrolling’ but not sure yet…will debug asap.

  34. gbattle

    The best hack Kevin ever did was the photoshop work in his avatar. C’mon my follicular challenged brother, you KNOW you don’t have any hair up top. Be bald and proud. ;-)Digging on gawk.it. Disqus should absolutely steal/incorporate this into their premium products, both vertically for comment domains, and horizontally across subjects. I want to search for related content as it relates lexicographically and socially. They are in the driver’s seat to get this done properly … and explore many a revenue opportunity as a result of intent-meets-social-graph at scale.

    1. falicon

      Nice! You know my avatar was actually drawn by an old friend of mine (who has since passed away)…he did it based on a photo I provided him (back when I had hair)…and it’s original use was for the back cover a comic book by Buddy Scalera ( http://www.amazon.com/Days-… )…of course promoting an old-school hack I did back in the day ;-)Also I will totally build out search for related content as it relates to lexicographically…but first I gotta get out my dictionary and figure out what most of that means (you are too smart my friend, too smart!) ;-D

  35. Kirsten Lambertsen

    Social media managers would look at this and see a great monitoring tool πŸ™‚

    1. fredwilson

      yes. that is one of the main reasons i have encouraged disqus to implement search

    2. falicon

      For the blog owners that install gawk.it, they already can get some detailed graphs on how much their content is being searched and when (and soon they’ll even get a keyword breakdown to see what terms were searched)…I think it’s an important bit of knowledge for the blog owners to have access to (helps them understand their audience and potential audience much better)…so I plan to improve this over time for sure.Not focused on any group outside of the commenters and bloggers at the moment (or even thinking about subsets of them yet)…but I will spend some time thinking about it now that you bring it up. Thanks. πŸ˜€

  36. Ryan

    It’d be nice if it highlighted the search term like when you do a Ctrl + F search…

    1. falicon

      Thanks…will work on adding this (whoosh actually has the feature built in, but it didn’t operate well enough at scale out of the box so I think I’ll try to write something custom to achieve a v1 of this)

  37. Guest

    I like how people are spelling out phrases with their domain names. “Gawk It” that’s cool.

  38. rjjacobson

    How are you pulling the data? Just curling the URLs?

    1. falicon

      Yep, nothing too fancy at the moment.The blog posts are screen scrapped (and some custom clean up code is applied to determine what is the blog content vs. everything else on the page — ie. a very basic version of readitlater or instapaper for this projects specific needs)…the comments are pulled in via the Disqus API.

  39. Aaron Klein

    A few little suggested tweaks.I started by searching my own name. It defaulted to a search for posts, of which there were 0, but there were hundreds of comments. Suggest defaulting to the tab where there are results.Also, in the results, I couldn’t figure out the sort order: http://d.pr/i/YQJIAnd if I click the headline above the comment, I get the blog post, but if I click “read the blog post” I get the comment…seemed like it should be in reverse.What an awesome product!

    1. falicon

      Hey thanks! I will look into the things you point out asap…I like the idea of defaulting to where there are results (I really love when software “just does the right thing”) but I am worried that it could add confusion for users…will think about it a bit more. Thanks!

      1. Dale Allyn

        Switching default output can be jarring or confusing to the user, as you suggest, so such things can be addressed with the content of the message as well. One step would be to make sure the user notices the tab which does have results by means of design (color, prominence, etc), but also the message can include a pointer to the other tab (including a graphic) so the user immediately knows to view the other tab. e.g. “Ahh, man, your search didn’t return any results under blog posts, but there are 213 items in the comments tab. Click here to view”.

        1. falicon

          great points/thoughts…going to use this. Thanks!

  40. Jay Janney

    Hi Fred, Kevin:It is useful. I typed in my own name, and found I had posted 21 times (I would have guessed 4 or 5). I went back and typed in four or five search terms, and found they brought up fairly quickly the blogs I remembered. Customer financing, Littlebits, woman entrepreneurs, the search returns were fast.I found it less helpful if I wasn’t sure of a good keyword or not. I remembered the “Darwinian hub of evolution”, and searched for that. Got tons of hits on evolution, including some tat seemed bizarre (a post really critical of Sarah palin: I must have missed that day)!. My sense of when i don’t know what search term to use, is that I’d either get no hits, or too many hits. Not sure there is much that can be done about that, but for what it is worth, thought I’d pass along that info. Also, sometimes the post listed a date, sometimes not.I hope you find this feedback helpful.

    1. falicon

      Very helpful thanks.I have been playing with various searches for the past week as well…trying to figure out the use cases that work well vs. the ones that just plain suck (unfort. I have found a few that is really does suck for as well). I think some of the sorting and ranking suggestions others have mentioned throughout the comments will help address some of this (at least I hope) πŸ˜‰

    2. William Mougayar

      AVC Vanity searches! ^10

      1. falicon

        This is a big use case that I wanted to exploit from the start…I find it really interesting to see how many times I’ve commented somewhere over history (and to look at others that way too)…I think I’ve probably doubled my historical comment count around avc today though. πŸ˜‰

  41. Otto

    If only I could make a living commenting…That said, now that comment search has arrived maybe I can use that and Disqus to organize my comments into a comprehensive and readable blog… assuming anyone would want to read it. Nonetheless, it will be good just for my own personal reference.

    1. falicon

      Not sure if you could make a living doing it…but your comment does give me an idea for a ‘publish to blog’ feature (ala Tumblr’s retumble feature)…I think engagio might have something like this already (if not I think I’m going to push them to do it). ;-)Also – an aside that your comment reminded me of, I find Tumblr search mostly broken as well (I *think* they just do tag search, but tons of people don’t really do tagging when posting — myself included)…my hope/thinking is that gawk.it can help fix search for many Tumblr users as well…

      1. Otto

        It seems like only a relative few tags get searched on Tumblr and many of those tags seem too broad. For example, #photography. Well, okay, but that actually tells me very little. One of the problems with tagging or SEO is that people start writing for search bots instead of readers. Tumblr is better in this regard but as a reader I found their search tiresome, not to mention their mobile app doesn’t include search at all.

        1. falicon

          100% agree. This is a big problem that I believe gawk.it can/will also help with.

  42. Dusty Bottoms

    Seems very cool

  43. Vasudev Ram

    Great idea, Gawk.it. I tried it out a bit. As others have said, there are some issues. Saw your replies on that.The defaulting to blog posts rather than comments in the results is a bit of an issue. But it would be the same if the other way around. So a suggestion. next to the search box, put a drop down or other control to allow user to select one of Blog Post / Comments / Both and search and show results accordingly. I blogged about Gawk.it here: http://jugad2.blogspot.in/2…Good luck with Gawk.it πŸ™‚

    1. falicon

      Thanks – I am def. thinking about ways to let users do more powerful search without making the UI/UX more complicated (I love the simplicity and power of google’s single search box).The most likely answer for now is an ‘advanced search’ screen…even though that feels like a very old-school approach to me…I have to think about it more I guess.

  44. FAKE GRIMLOCK

    THIS HOW ME, GRIMLOCK TEST ENTREPRENEUR.1. THEM HAVE IDEA.2. NEXT DAY SHOW MVP.

  45. Prokofy

    I think it’s a great idea, but I’m puzzled at what the principle is that puts things on top. Is that a secret algorithm or can you explain?Example: I wanted to find Fred’s remarks/posts/links on “Instagram”. I put that term into the search engine. The first return says “I am Fred Wilson” and has something on liking Instagram down the page. The second return is something apparently he said about Instagram. The third return has an icon of Fred’s face, the tag “blog post by Fred Wilson,” but then the text is a blog post by Scoble (I started reading it thinking it was Fred, until I came to the line “I work at Rackspace”.)So how did that get there? Possibly Fred linked to Scoble in a blog post? Or? What I need in the returns is a clearly marked headline of a blog post, its author, and a highlight of the word search term.

    1. falicon

      results are currently sorted be term/phrase frequency but I am going to tweak this based on everyones feedback here (working on it already)…one thing to note is that the content it ‘searches’ is more than what is displayed (because I add a bunch of meta data to what is indexed)Not sureabout the scoble thing…will look into that one and let you know.will also try to do something better to denote stuff as you mention. Thanks!

  46. Lisa Mogull

    I LOVE this! I remembered an AVC post from a while ago that I wanted to share with my partners (the one about Airbnb and ObamaO’s). The old search didn’t find it but the new one did. Awesome!

    1. fredwilson

      Yessssssss

    2. falicon

      Awesome! So glad people are getting value out of it and that it’s working for their use cases…as a developer, that is the ultimate compliment I can get…and it really motivates me to work even harder to make it better. Thanks!

      1. Lisa Mogull

        No. Thank you!

  47. Donna Brewington White

    This is fantastic, Kevin.You remind me of my grandfather — at any given time he had a dozen or so watches and clocks — all broken — that he would tinker with in his spare time to get them working. He was a hacker in his own right. You are a true tinkerer — you always seem to have something really cool in the works.One of my favorite quotes of late was from a comment by @daveinhackensack:disqus : “I’m bullish on Kevin Marshall.”BTW, I just searched for the word tenacity and had a brief nostalgic stroll down memory lane looking at some old posts. I also looked up Donna, vain as I am.

    1. falicon

      Thanks! I love the story about your grandfather…if I didn’t have computers and software to tinker with, I would totally have been into something like that as well.I love that quote from Dave as well…so great to have friends like that (even if they are Giants fans!) ;-D

  48. Techman

    Is there any thoughts of being this to Blogger really well? I’m looking for something that can replace the default search widget, because it does not work at all on mobile. Can Blogger mobile integration be coming? @falicon:disqus @fredwilson:disqus

    1. falicon

      I am actively working on expanding the platforms that gawk.it fixes…at the moment, it actually can be installed on a blogger blog (or any blog really), it’s just not as clean and simple of an install process as I would like it…Email me ( info at falicon.com ) if you get a chance and let me know your blog url…I’ll help you get it set up. Thanks!

      1. Techman

        Can you use Gawk.it as just a search engine and disable the comment indexing? I’m not all for indexing of comments, but searching would be good. The search box would have to point you to the mobile version of my posts when searching from the mobile site, for example. Subject: [avc] Re: Gawk.it

        1. falicon

          yeah – it doesn’t actually have to index comments (there are a few blogs in gawk.it already that are just blog posts)…Though, I do feel compelled to say that the core thesis is to index the whole deep conversation…this way people can actually find the quality bits of the conversation they want (and also filter out the bits they don’t)…regardless of if it’s in a blog post or a comment…But it’s entirely up to the blogger as to what they want in the system or not…they are the center of the content universe within gawk.itEmail me and I’ll make sure to get it set up perfect for your situation and needs…

          1. Techman

            I’ll look into emailing you later on. I’ll try to see if I can get a to get existing search working again, but if I can’t then I’ll email you.

  49. Techman

    Ok, I just started using it at my site about a day or two ago. It works pretty well. I have already submitted lots of feedback and bug reports, so all it can do now is get better. Thank you @falicon:disqus for putting together such an awesome product.

  50. RichardF

    lmao

  51. falicon

    shh…don’t tell kid! πŸ˜‰

  52. Kasi Viswanathan Agilandam

    two towers went down is it? … sorry two servers mis communicated :-).

  53. kidmercury

    try now — it’s working. and i like the results — pretty good! i only see two comments on the front page that are not related to politics. they come from a variety of users (usually from me or someone referring to me) although there are a bunch referencing the movie “inside job” which is about the 2008 economic crisis rather than 9/11. google’s non-personalized search results are mainly about the movie and contain no references to 9/11. looks like gawk.it is better than google!

  54. ShanaC

    oh god…. πŸ™‚