General News

Google promises to fix Chrome bug on Windows, causing laptop batteries to drain

Microsoft News

Search for the perfect US school for your children with Bing

Microsoft News

Microsoft now offers 'right to be forgotten' form for removal from Bing

Windows Phone News

Microsoft preventing new Lumia owners from changing the default search engine to Google [update]

Microsoft News

Project Adam's visual search could make Bing and Cortana even smarter

General News

Microsoft takes Cortana to college to help with academic research

Apps

Bing gets every prediction in knockout stages right, sides with Germany for the final

Apps

Bing has predicted the World Cup 2014 match winners ... so far

General News

Songza has just been acquired by Google

Apps

Microsoft adds new education and book features to Bing

Apps

Cortana gets new service update that adds Bing predictions

Microsoft News

Bing now includes tweets and celebrity Twitter handles in search results (US)

General News

Android tries to find visual identity through 'Material Design', borrowing from Microsoft

Apps

'Bing is embarking on the most ambitious geographic and product expansion in its history'

General News

Cash in your Bing Rewards Points and go ad-free on Outlook.com

Microsoft News

Microsoft supercharges Bing servers with new programmable processors

Microsoft News

Microsoft still trying to figure out its 'right to be forgotten' setup for Bing

General News

Songza curating multi-million dollar bid from Google

Microsoft News

Bing turns five today, Microsoft celebrates with a look back at its best homepages

Microsoft News

Satya Nadella: Bing and Xbox are not for sale

214

Is Microsoft’s “Bing It On” campaign a sham? The evidence doesn't look good.

UPDATE: Please see our interview with Bing scientist, Matt Wallaert, here.

Recently, a law professor from Yale claimed that Microsoft’s famous “Bing It On” campaign is no more than a collection of lies. Ian Ayres, stated in Freakonomics that the company’s Bing ads are misleading and deceptive. To prove his point, Ayres set out on a “Bing It On” challenge using Amazon’s Mechanical Turk marketplace.

Amazon’s Mechanical Turk website allows anyone to offload tasks to a marketplace where anyone can work from home and get paid to fulfill those tasks. In this specific instance, Ayres recruited 1,000 people to take the “Bing It On” Challenge; his results, were quiet startling.

“We found that, to the contrary of Microsoft’s claim, 53 percent of subjects preferred Google and 41 percent Bing (6 percent of the results were ties). This is not even close to the advertised claim that people prefer Bing ‘nearly two-to-one’.”

The results were so far off from what Microsoft had claimed, that Ayres (let’s remember he has a PhD in law) stated that Google could file a deceptive advertising suit against Microsoft.

Today, Microsoft responded to the study with words from behavioral psychologist at Bing, Matt Wallaert. Microsoft refused to release a collection of their data from their own studying citing “reasons of science and privacy.”

Wallaert stated that the results from the original “Bing It On” challenge aren’t based on the claims of “a wildly uncontrolled website, because that would also be incredibly unethical (and entirely unscientific).”

The problem is that Microsoft never attempts to answer exactly why Bing was trumped so badly. Even if the environment was less controlled, results would not expect to be so widely different.

Ayres stated that Microsoft is most likely choosing queries for their users that they feel will deliver better experiences on Bing. Wallaert fought back and stated Microsoft collected the top queries from popular search terms that they found in Google’s 2012 Zeitgeist reporting.

According to reports from NetMarketShare, Bing’s market share on Desktop devices are up to 5.70% this month, that is a far way behind Google’s market share of 70.53%. That being said, Google had suffered a 13% desktop market share drop in July of 2013.

It is no secret that Microsoft is struggling to fight itself to the top, but the question is – would Microsoft lie to get ahead? As of now, Microsoft simply has excuses and is refusing to back up their claims with hard data and evidence.

I myself am a Bing user and love it, but even I am questioning the legitimacy of the company’s “2 to 1” claims. What do you think – is Microsoft telling the truth or playing a few lies to get ahead and get noticed?

Bonus Question: Have you taken the “Bing It On” challenge? What were your results and did they contradict what search engine you currently use on a day to day basis?

Sources: Freakonomics and Bing; via The Verge, NetMarketShare

1
loading...
0
loading...
28
loading...
0
loading...

Comments

There are 214 comments. Sign in to comment

So 1000 people are able to accurately depict the overall opinion of millions? All his survey does is prove that he used a different sample population than Microsoft did, anyone can fabricate any stat they want be selecting a sample population that is biased,

That's called statistical analysis through "random sampling." its a tried and true method, albeit deeply flawed.

Nick Russo2 says:

We can all agree that the experitment performed did not use random sampleing but rather a select group of people who chose to use the program chosen by Aryes.  

Josh Harman says:

LOL - "tried and true" ... "albeit deeply flawed".
Anyway, this guy doesn't have the best credibility, he's a plagiarist.
...
http://wmpoweruser.com/bing-responds-to-criticism-of-the-bing-it-on-chal...

4look4rd says:

If done correctly then no its not deeply flawed. You are basically calling science itself deeply flawed.
If you don't know what the central limit theorem, and the law of large numbers are then gtfo because you don't understand a thing about statistical analyses and therefore you are unqualified to make such criticisms.

RedSamurai says:

Science itself is often deeply flawed -as it has been evidenced times and times again-

+1111111000000000000000 :)

kayb27 says:

This is the best rebuttal I've ever read. Thank you for the post.

Edwin Checo says:

The point is that his research can be used to argue with Microsoft's own research including the numbers they came up with which btw are pretty large

JasonH81 says:

Thanks for the link, hopefully WPCentral posts it too. Great read, and they completely destroy Ayres with their response.

So 1000 people from an online marketplace is random? What is mechanical turk? I would suspect there is some correlation in the population who even know what it is.

xratola says:

I've taken stadistics I & II and with the methods there are to classify people into groups in order to make a market research or a survey there's like 1% or even 0.5% of getting inaccurate results. For me there are two possibilities
or they are lying or they're stupid :)

Intrepid00 says:

Going to Amazon's Mechturnkey isn't a random sample. It is literally going to a sweatshop to for a focus group.

PolishHitta says:

Is this statistical sampling include only from US?

cspkcats says:

My guess is this statement in the paper:
 
"Some savvy users even claim to be able to distinguish the two search results solely on the basis of page formatting."
 
A savvy user that already dislikes Microsoft or is familiar with Google will tend to pick Google more often and vice versa. (Considering that 66% of the people use Google, I would assume the bias would be more towards Google unlike what the author of the paper hints at.) While this does bring into question the legitimacy of the results, I think the differences in results could be a result of the difference in populations and the sampling methodologies. For instance, two pollsters, one commissioned by GOP and one by the Dems will generally come up with differing results.

cspkcats says:

http://islandia.law.yale.edu/ayres/BingItOn_Draft%209.pdf
 
Here's the paper. A few things that might explain differences:
 
1. Do Microsoft's studies restrict the population to adults only?
2. The study has 40% females, while Gosling's sample is at 57%. Is that difference significant, especially when the authors later conclude that females are more likely to pick Bing?
3. I think it would have been interesting to ask the users about their preferred search engine (Bing, Google, Yahoo, etc.) and how internet savvy they considered themselves. I have a feeling there will be a correlation between this and the results.

willied says:

I didn't even read the paper but I thought to myself that the formatting of the pages could definitely put things in Google's favor because of bias for Google.

Bartdog says:

definitely, when I took the "challenge" i had a hard time ignoring the obvious formatting that let me know which one was which.  I know some people would do as you say and pick goog (or even bing) based on the fact they knew which it was.  The results were close enough that it was tough to really choose one otherwise.

cspkcats says:

Yes, and the paper also cites another study, which makes an even more perplexing claim: "A study by the Catalyst Group found that users preferred Bing‘s visual design over Google‘s, but also found that most thought the two search engines produced equally relevant results, and overall, indicated a desire to continue to use Google as their primary search engine."
 
I think that means that a lot of people are biased coming into the study and if they're able to figure out which one is Google, will tend to pick it.

WiNGSPANTT says:

Yes, 1000 people is actually a fairly valid sample size. Even if it weren't, how would anyone know they're worse than what MS used if MS won't release ita data?

NIST says:

The stats are a reflection in time. Theoretically you could keep running the test until you got the desired results. The Yale test results are only good for the time and population participating at that particular time. To extrapolate that result to make accusatory claims against Bing is ridiculous.

Esseret says:

Statistics is a joke of a field.  Anyone can get whatever numbers they want to show and make it look legit.  Just as an example, let's say we weant to do a survey to show that more americans prefer the word 'pop' over any other term for a carbonated beverage.  All that is needed is to take your random sample where that actually is the case.  You can then show that all ages, colors, and other diversities were sampled, thus making your findings acurate.  But now let's move our survey to another region where perhaps all carbonated beverages are called 'coke', regardless of who makes it or what flavor it has.  Now your survey shows that 'pop' is no longer the preferred term.
 
And this is just one small sample of how easy it is to skew your results. And since it is IMPOSSIBLE to be an adult human and NOT have any opinions whatsoever, these sorts of things tend to get skewed simply because we are not perfect.  You can find this with even the great scintific minds of history.  They had an idea of what the answer was they wanted and they skewed their tests until they proved they were right (of course they WERE right, but still...).  So if Microsoft says they got a 2 to 1 ratio, great.  I'm sure they did.  And if some law professor says it's much closer to a 1 to 1 ratio, also great.  I'm sure his results are just as valid as Microsoft's.  It's why I don't tend to listen to numbers but judge things based off of recommendations by friends and family and personal experience.  (If you want another easy to prove skewing of stats then look at mpg/kpl - what's on the sticker is NOT likely to be what you get driving under varied road conditions.)

wizll says:

word, whenever there's any "statistical" data in advertisements, i'd take it with a grain of salt. http://en.wikipedia.org/wiki/How_to_Lie_with_Statistics this was the book that I was handed in my first statistics class. still holds true.

Jim_MAY says:

As Micheal Jackson would say, "Just Bing It"!

karelj says:

If this is true and MSFT gets busted, then they deserve whatever negative press and reputation comes their way.
Too much time spent on these negative ads. Work on your products and services, make them better and the results will speak for themselves.

bcasteel says:

I agree. Hope this ends up being unsubstantiated. Either way I think "Bing it on" is a flawed idea

AmFuzzy says:

Funny you say that... did you say the same thing when Apple had years of the Mac vs PC ads which were EXTREMELY misleading?

karelj says:

Yes, I did. But nice try...

I think deep down even Microsoft knows Bing is just not up to par with Google. They keep trying to throw pretty stuff in like the useless images on the homepage but the search results are still quite a bit off from what Google offers.

IamDefiler says:

I think it may be personal choice as to what you want/expect to see from a given search. I know a lot of people who prefer Google over Bing but I prefer Bing over google. In the end, who cares. Use what you think gives you the best results.

Yes...yes they are. And by that, I mean Bing's results aren't biased to just companies that are owned by Microsoft or are closely affiliated with them.

You nailed it for me. For Google searches, I always get advertising or paid advertising as primary search results. for bing, I get answers based on my search without necessarily being sold something. So for me, I prefer bing 20 to 1.

sunbee says:

I agree with that. I hate the annoying adverts on the screen or results with Google searches but Google name was so much widely popular most of them tend to hit Google for any search including me but from the time I started using Windows phone I prefer Bing with quick search touch button.

Zapf says:

Lets be serious, bing has paid ads too. I get them as the first result in my bing searches on my phone. They get labeled, but they still are there, filling up the first screen of results, looking almost identical to normal search results, every time.

I don't generally see the ads fill up the ENTIRE first page, but they are there on the phone. We were discussing the desktop version of Bing, so I hadn't considered that really, and as far as I can see, there really isn't obvious ads or biased results on the desktop version of Bing.

coip says:

I actually find Bing to be better than Google: it almost always finds me more accurate results, or they're basically the same results (but Bing has a better UI). I also love the new Page Zero feature and the social search integration (which often leads me to exactly what I want).
Anyway, here's Microsoft's full response. There is no way they get sued for these claims, even if they're misleading. Companies have been doing the "taste test" marketing strategy for decades: Pepsi vs. Coke, Miller Lite vs. Bud Light, Bing vs. Google. I don't know how effective the marketing actually is, but when I took the Bing It On Challenge, Bing won. And when I had my friends take it, most of were surprised to find that Bing won for them too.

AlwaysHedged says:

Agreed.  I've been using Bing for a couple of years.  Don't miss Google.

kurotsuki says:

Agreed. I try it myself and clearly chose bing. But it might be biased (I already use bing as default search engine on my firefox before bing it on is even started). But when I ask five of my friends, who never know about bing (they only know google), to try bing it on, the result is surprising that all of them are picking bing as the winner.
As a side note, Me and my friends are from Jakarta, Indonesia

Its not useless if it makes your experience better. Also, Bings search results may lack in" select" areas, but otherwise its no different for me.

The useless thing you mentioned, along with many other things, make Bing a better choice for me.

Joel Back says:

Regardless of what they think they have to keep fighting Google and putting money into it or else get out of the internet business altogether. 

sqlchicken says:

One of the biggest things that annoyed me about google is if you do a search, and copy the link from results page the URL copied is a Google one (for ads\tracking) whilst Bing gives you actual URL. Little things but drove me to Bing a few years back, been happy so far.

kurotsuki says:

When I search for "arsenal" search term, this is what I've got from copy-paste the search result's link on bing:

http://www.arsenal.com/

and this is from google:

http://www.google.com/url?q=http://www.arsenal.com/home&sa=U&ei=GXIOUvTC...

So ... yeah ... bing's way of giving the actual result is clearer.

Raesu says:

That "useless image" makes my day, every day. ;)

xankazo says:

Likewise! I save all of them that can be saved.

SmoothDog says:

I concur. Love the image and the links to explore on it.

its all in context. Bing may give better results on some keywords, with Google better on others. For me, Bing is more accurate. For you, google may be. it all depends on what you search.
 
honestly, bing does compare to google. these studdies are always finicky, since the search criteria may be completely different from Microsoft's original study, and Google could of changed their algorithm and search ranking since then (not to mention the internet in a constant  state of change), causing different results.
 
i use bing, since the results for me are better.
 
funny thiught: its possible in this study attempting to dissprove bing it on, the users didnt pay attention to the "searching for *. search for [original term] instead?" thing Bing does when it thinks you misstyped a search, since i will state Bing does that way more often than Google.

Josh Harman says:

Nah, Bing has always been better than Google for me. Google has too many ads and ad based results to give accurate search results.

jlzimmerman says:

I like the "pretty stuff" and I have always been able to find what I need when I search with Bing.  Google has too much nonsense ad crap;

lubbalots says:

Mindshare or more or less mindwashed. There's this innate or imbedded belief that Googling, Google it, is cool, and that all your prayers will be answered by google. If Bing can't do it, I hope Yahoo obliterates Google off the face of this planet. Go Marissa Meyers! Wipe that do no evil clean of its evil.

ymcpa says:

I disagree. For the vast majority of stuff I've searched, I've gotten pretty similar results from both. I know people in other countries complain about Bing results, but in the U.S., it's pretty close.

Well, firstly, I love the Bing image of the day light up the lock screen of my phone. Secondly, I love the fact that my search results aren't ad's oriented. Thirdly, Bing DOES NOT repeat your search results 6x times like Google does. I am more than happy to use Bing over Google any given day.

thesorehead says:

Agreed. For most general searches, Bing is on-par with Google (but not necessarily better). For specific things, Google is better - e.g. picture search, translation and finding venues or businesses on a map.

edjr07 says:

I took it and BING did win. I tried it with my fiance too and now she is a Bing user. So...theres that. Also I have a small plastic camel that I walk around with on hump day.

Jazmac says:

Who gives a crap? I use Bing and will continue to use it. Those that like the other engine, use it. Who cares?

BCH says:

I don't feel either side has really established one or the other is wrong. In my experience, both engines deliver pretty comparable results. Having taken the Challenge three times over a several month period using searches I made up on the fly, I have chosen Bing twice and Google once

Micah Dawson says:

I usually got better results with google sadly.

VanillaFrog says:

Were they all from the US? Bing seems to be sub par in a lot of countries.

jcutting says:

Excellent question.

ymcpa says:

I believe those ads were only shown in the US. So, it doesn't matter how Bings does in other countries as far as the claim that it mislead people. It does matter if Microsoft wants to build up its market share.

VanillaFrog says:

It matters where the test results came from on both accounts. If that differs then the results will also.

I use Bing sometimes, but I get better and direct search responses with Google. So I could find this possibly true. Unless I'm not understanding this correctly. If you sham the public, that's not a good look.

bcasteel says:

I took the Bing it On challenge several times, and almost always it's 5-0 Bing or 4-1 Bing. For me, Bing usually beats out Google

I've had the opposite experience. I used Live Search before there was Bing, and have used Bing for as long as it's been around. I avoid Google if at all possible, partly because I can't stand them, partly because Bing gives me everything I need. Yet, both times I took the challenge, it told me Google won (3-1 most recently, don't recall the exact result of the first one). Oh, well.

iZangetsu says:

I have taken the challenge and it's because of that, that I now use Bing. The results for me were 4 to 1 , Bing to Google. I don't see what the problem is. -______-

tallgeese says:

MS is not telling the truth. I took the Bing Challenge and Google always won with my queries.

But, I still prefer to use Bing.

you =/= everyone else

tgr42 says:

100% pure marketing BS.  Ultra-transparent.  Who ever thought for a second that these ridiculous claims could be true, when it is so easy to see for yourself?  The lovely thing about the web is that the competition is always just a click away.  All it takes are a handful of REAL WORLD queries and Bing's inferiority is immediately apparent.  Although I did have Bing Maps correctly locate a business once when Google Maps failed.  That was a real shocker.

pr0phecy says:

I do have to admit, Bing is a lesser experience here in Netherlands than Bing in the US for instance... but hey, it's Netherlands we're talkin about here... or should I say iNetherlands.

Fade_z says:

What makes It lesser of an experience in your country compared to the US?

lpforthewin says:

I presume it's the same issues as in the uk so more ads, no snapshot card thing, no bing rewards, no social sidebar, no local result cards or quick links, no news and I'm not sure if the maths functions work.
The results outside the US look awful, they fill a tiny amount of the screen leaving loads of white space for no reason. I have my region set to us and make do with putting uk on the end when i need a uk site.

Zeroplanetz says:

I've taken the challenge 5 times now. (each on a different device) 4 times was a winner with Bing and 1 was a tie. Google is just to loaded with ad ridden crap. So I say Bing for the win! Screw this so called lawyer. Titles mean jack sh** to me.

regi7 says:

I took the Bing it On challenge and I actually preferred Bing.

BK-one says:

To quote the great one.. ""it doesn't matter... what your search engine choice""

aitt says:

Which is why bashing the competition to much can come back and bite you on the jackass. Make your products better, focus on the real issues at hand and stop focusing on trying to scroogle people out of Google.

This is very OBVIOUSLY not true. Why else would this type of marketing exist? Remember the Mac vs PC ads... It is the exact same thing. It is effective, plain and simple.

I took the Bing It On challenge 5 times, and once Google won, once was a tie, and 3 times Bing won. Before that, I had already switched to Bing because I dislike the all-white look of Google, their biased results, and like to look at beautiful things, so Bing appeals to me.
 
Just as some others say, it's quite possible that their results are another case of "taste test" strategy. I could understand if you did random tests and found a 90% Google/10% Bing result, but Microsoft was bragging about them being the 90%, as the numbers are so vastly different. 2/3 = 66%, supposed random result was 41%. 25% difference is possible in a random "taste test".

freshfelicio says:

I like bing more, especially after the redesign. The image search is much more comfortable. But they have to improve the UI of Bing maps.

sicnus says:

MSFT lied!? THE HELL YOU SAY! I took it and picked google. However, I'm liking the newer Bing implementation on my WP 8.

I would be interested to hear from Bing users outside the US. I live in New Zealand and use Bing a bit - mostly on my Windows Phone and find the results to be really US centric wheras Google provides local results. As an example I searched for a local Motorcycle dealer by name and location on Bing to find their website. Results were about 2 pages of dealershipswith the same or similar name all over the US. With Google the first listing was the one I was after. I have had similar experiences with other searches too where I have started out on Bing but had to change and search it on Google

Dazzi says:

I'm in the UK and I have the same problem, in terms of US centric results. I hate to say it, but in my opinion, Google is way better for UK results -- I wonder why Google is better though?

ysureshot says:

Did the challenge months ago and Google swept twice. Now hearing this...

Micah Dawson says:

I can't wait to read the verge's spin on this.

Toxid_Fox says:

The Bingiton.com website has now been changed so it redirects you to the Bing homepage. I don't know if this is because if this information or simply because they don't run this promotion any longer.

Actually, I just went there, took the challenge, and Bing won (again). There is an explanation of the results there too.go read it. Makes sense to me.

I imagine that at one point the statistic was at least close to accurate. By now Google has had the chance to modify the presentation their results by now so they fare better at the Bing it on challenge.

Hagenlund says:

It's conceivable that the Amazon users were international and the MSFT users were from the US. Bing is unreleased and horrible in many countries, but apparently quite good in the US. Much like Xbox Gold, WP8 podcast support, Local Scout, etc, etc. MSFT is really lackluster at regionalization.

schandlich says:

Oh it's more than conceivable.

cspkcats says:

No, if you read the paper, they restricted it to US folks only. That said, I think Microsoft's results were biased towards Microsoft (obviously) -- kind of like GOP/Dem sponsored polls tend to be partial towards them -- and the author of the paper seemed to be biased towards Google.

AndyGriff says:

Eventually, Google will shoot themselves in the foot by becoming complacent and taking too many liberties. If Microsoft concentrate their efforts on their own products, they'll be fine in the long run. There's no need for this sort of sniping.

RockmanNeo says:

I use Bing exclusively. I took that test and Google won :\

kbilly70 says:

I've taken the challenge 5 or 6 times in the past and Bing came out on top every time. It was usually close though with 3-2 results.
 

pwachleman says:

Wow, shocking, ads fudge the truth...

RussEby says:

I use Bing and have for quite a while. I'm used to it. I think that makes a big difference. As far as these studies go, my dad used to say if you don't like the results of a study, just keep asking sooner or later you'll hit the numbers you want. Paying people to take a survey is a big difference then having them take a survey because they want to and are curious as to the difference.
Think about the person being paid, The faster they go through the survey, the more they get paid. I doubt they even read the search results.They just clicked an answer. Sorry, but the PHD is not impressing me.
Did MicroSoft lie? Stretch the truth? Not anymore then any other ad campaign from any other company.
I will continue to use and recommend Bing. If you want to use Google, go for it. Let the two power houses compete. Both sets of users will win!

Blurbia says:

Google is better than Bing, IMO. I Bing-ed it on, Google won, and it wasn't close.

TheDarKnight says:

Someone is looking for attention

cae758 says:

I use Google on everything, but my NL920. Not because it's necessary better, just what I'm used to and it works.

EchoOne30 says:

Listen, I love windows phone and will be on this platform for life. However Bing sucks terribly and is never anywhere near as good as Google search results. Google the company may be terrible, but their search is still the best.

I don't buy it for a second. Every single person I've introduced to Bing, some hardcore Google users, always end up preferring it. Some guy just digging for controversy.

I'm glad this clown has this much time on his hands. I just did a survey with me and my brother. We both prefer Bing. So I guess Microsoft understated their facts....its a lot closer to 100%

kbilly70 says:

I find it amusing that a law professor actually thinks there is a lawsuit in this.

aitt says:

3 - 2 Google

Josh Harman says:

Left out of the article is that this guy has 0 credibility after being sued for plagiarism.
...
http://microsoft-news.com/bing-responds-to-criticism-of-the-bing-it-on-c...
...
I took the "challenge" and Bing won overwhelmingly.

allos autos says:

That is tremendously important. Plagiarism is the deadliest academic sin.

shalgrim says:

You guys need a proofreader. Is painful in to reading yr site.

jfmanzo says:

There is no such degree as a "PhD in Law." In the US, a law professor might have a JD and perhaps a PhD in some relevant cognate discipline such as history, or s/he might have an MLL (Master of Law) which is bizarrely more advanced, in US academic nomenclature, than a JD (which is a practitioner degree, not a scholarly degree- a JD is like a PsyD whereas an MLL is like a PhD in clinical psych). But no, you cannot get a "PhD" in "Law."

allos autos says:

Sorry, but http://www.bing.com/search?q=phd+in+law&go
 
EDIT: Yes, there are PhDs in Law, even in the US. However, I found this guy's CV and his PhD is in Economics.

MaulerX says:

I've taken the challenge several times and Bing won for me in all instances. Not sure what it is, but I think it's because Google results are so full of ads that it has become a major distraction. I really don't think anyone can really dispute this. In fact, you can easily tell which is which just by looking at all the ads. This guy "recruited" 1,000 people... How do we know that he, armed with this knowledge, didn't tell them to just pick the ones with the ads? Conversely, because of the ads (or lack thereof) , I can see why real people would pick Bing over Google.

I took the challenge when it first popped up, I was told I'm a google user, but I use Bing. I don't have any issues finding what I need, haven't touched google for well over 2years

nrln says:

The way ive convinced people to use bing was not through that challenge. The challenge is dumb. Bing rewards is awesome. Thats how ive converted people. Just use bing as the search engine on chrome. Ive gotten so many gift cards from it so far.

schlubadub says:

That's great if you're in the US, otherwise you get nothing

I got two other guys in my office and we just took the challenge, two of us picked Bing and one picked Google. 2 to 1 ... Done!

Blu3V3nom07 says:

I use bing because I get free albums out of it. And soon a Battlefield 4 Premium Pass. :D

dkp23 says:

I took the challenge, google won out for me. 
 
Bing to me isn't really that good at all unless you do basic searches such as French Fries.  Once you get to more complex searches with multiple words, it isn't even close.  Only time i use bing is on my windows phone, but if i were to search something on my pc or tablet, its google every time. 
Bing is not even close to Google. 

mattdistro says:

I took the challenge a few months ago and Google won, primarily because it gave me a Wikipedia result on the first page, Bing stuck it on pg. 2 on numerous occasions.
 
That being said, I prefer and use Bing as my default. On occassion I do have to use Google for more complex searches, but it's not an every day occurence.

MethodGT says:

I did the Bing it on challenge at my local Microsoft store. Google actually won, which I was disappointed about. But the employee still gave me a 10% off card.

k00ksta says:

Google does not give me points for searching the web which I can then use to buy stuff on amazon, wp8 and xbox. Nuff Said.

Dazzi says:

Unfortunately, Bing doesn't give me points for searching either, here in the UK. Nuff said lol.

allos autos says:

Rhetoric is rhetoric. All that is needed for Microsoft's statement to be true is that there is a pair or more of studies which when aggregated have users preferring Bing to Google to the tune of 2 to 1.  Anyone who believes a marketing tool at face value is not going to make it very far. I'm reminded of the only time I've done the Pepsi taste challenge. It was 35C out and the Pepsi was served cold while the Coke was warm.  Guess which one won.

Arsenic17 says:

I would like to see the method he employed to these people. He paid them a dollar to go to a website and do the challenge. Then report back with their resulting search engine. Maybe people lied who knows. Two to one is 60/30%. Meaning only 20% of people needed to lie to get these drastic result changes.

WiNGSPANTT says:

Bing It On is a great idea but maybe they should've claimed people like both equally instead of claiming a 66/33 split.

Grizlyadams says:

Seriously? It's an advert; my assumption with ANY advert is that the advertiser is "selling" (ie blurring the lines, bullshitting, obfuscating and generally acting by vd

erzhik says:

Its an ad that targets the competition. Of course it's a sham to an extent.

marantaz says:

IF Google had substantive reason to believe that they could bring suit with reasonable hope of victory, I doubt they would need this fools chaotic study as evidence ... they'd have far better to present to the court, they've got more lawyers than Hell. I suspect, much like many studies we hear and read about done by 'professors', this was another waste of some federal funding by some waste of human flesh seeking notoriety and more funding. Only idiots believe everything they see in advertising, and only idiots waste their time trying to prove the obvious...that advertising is not truth. DUUUUUUUUUUUH. Professor, you fail.

AndreHBuss says:

I use Bing every day, works much better than Google.

cwc2311 says:

It's funny that a quick Bing search for Ian Ayres suggests "Ian Ayres plagiarism". Turns out he's actually been caught plagiarizing, so I wouldn't really call him a credible source.

jlzimmerman says:

Yeah, because Google, Apple, and other corporations have never lied or misled folks with their ads.  Riiiiiiight.

Pappa Ed says:

Everyone knows that 87.32% of all statistics are made up in the spot.

Doesn't matter...to hell with everything google.

n3rfh3rd3r says:

People just need to chill out.  Does it really matter which search engine is "better" or "most preferred"?  Why the need for validation amongst the masses when at the end of the day it does not do anything to make the search results more accurate?
For the record, I use Bing.  I still use Google when I'm at work or on a machine where I do not want to adjust another user's default settings.  Apparently which search engine you use is a serious problem. 

gibbage says:

Marketing 101. If you make a claim, be prepared to back it up. Microsoft's marketing team seems to have forgotten about that. Reminds me of the iPhone "2x faster" marketing that turned out to be total Bs, but Apple got away with it by simply stating "We never said what it was 2x fast than"

gevabar says:

Bing is better

Joe Hall says:

I use Bing 90% ish of the time, but if I cant find what I want then I do use Google, but most of the time I find what I want on Bing.
 

cspkcats says:

You can't seriously compare the 2. The 2x claim is either true or not true and it's easy to prove. A lot of factors determine Microsoft's claim because it's all sampling. All Microsoft has to show is that it ran a test and the results from the test. For instance, a drug trial repeated 5 times over might produce different results each time.

Scooter923 says:

I took the Bing it on challenge and four of the five rounds wee essentially a tie, but one of them, a search based on some esoteric preferences of mine revealed why google is generally my default: for now it continues to give me the information I need for those off beat searches. It goes back to the old saying that the devil is in the details. Most people could use Bing for most searches and get a long fine but they still don't have the refinements that Google has and always seem to be a step behind.

aemehran says:

Well, being a scientist, I wouldn't accept either or until I see the results and the protocol. However, I have been using Bing for the last year or so, and I don't think I'll go back to Google. I like Bing better.

So your scientific conclusion is that they binged it on.

Intrepid00 says:

His sample pool is a farce and not random. Using Amazon's turnkey is literally going to a sweatshop in Asia for your sample. Also I'm not using the Google defenintion of literally. I am using literally as literally.

Manny C says:

I love Bing, I've begun to use it all the time as my default search engine on all my devices, it's beautiful & great!

marantaz says:

You know, I've seen these iPhone ads with people talking on iPhones with colorful bodies...makes you think everyone has a colored iPhone...that can't be! I've seen white ones in the wild! They must be lying! Lets sue! Who's with me?

the article title is hugely misleading- there is NO EVIDENCE being presented, only one persons study versus another's.

Reflexx says:

It's a poorly researched article too.

xankazo says:

Well, I use Bing, I like it and it's my only search engine. I'm happy with it.

1000 people willting to do random tasks from people on the internet from Amazon Turk...  prefer google.   Good to know...  but it's still better to ask a random person on the street than to pay someone in a website.   The internet proficiency of someone willing to find Amaxon Turk, is different than that of a layman.    Still interesting info...  but he made a false jump from that data to Microsoft's data.     Microsoft is making it worse by not releasing their data.

Reflexx says:

What data? The guy wants web data, but MS doesn't use the web data foe their stats. They don't collect any web data as part of their privacy policy.

Groover1971 says:

I have taken the Bing it On challenge several times and Bing always won. Some of my friends and relatives have taken it (some clearly stating that they were Google fans) and Bing always won when I watched them take it. I have it as my home website whenever I go into IE.

I looked at Google the other day and started wondering "What year is it anyway, 1999?"

schlubadub says:

The fact is that Bing is pretty hopeless outside of the US. Microsoft knew this, and restricted the Bing It On challenge to US-only. I tried to do it, but wasn't permitted.

You misspelled 'Bring'

schlubadub says:

Actually, it's called "Bing It On", not "Bring It On" as it's a play on words...

mjrtoo says:

Dint these people have better things to do than quibble about freaking search engines?

SwimSwim says:

Who cares if the results are real or fudged? Just use the browser that works best and we can all get on with our lives. I used to use Google and hate on Bing, but I gave Bing a go, found out the results were exactly the same (just displayed differently), and I switched just for the hell of it (I love the looks people give me when they see I'm using Bing, I've always gone against the crowd).

But seriously, just use what worms best for you. I personally love the Bing Photo of the Day, and Bing Rewards are nice. However, Bing is also infamous for limiting many features to the US, so I wouldn't be surprised if people outside the US found Google more usable.

So let's drop this. I'm a form believer in using what technology works best for you. Yes, I hate on Apple, but mostly because many buy it unwaveringly and without question, without even knowing how it stacks up against the competition (anyone seen Jimmy Kimmel exploit this trait in people? Always funny). That's the only reason I get mad at people who just hate on Bing without trying it (like I used to). Give it a go, and if you don't like it, just keep on Google-ing.

Sorry for this long winded banter and babble, I got a little carried away there. In case I didn't make it clear: the results may or may not be real, but who cares? You have a mind of your own, so you decide which one's best.

hopmedic says:

I'm not going to discuss the merits of the claim, as I would be arguing as an unarmed man, but I can speak to my own experience. I've taken the challenge several times myself, and each time, I ended up choosing Bing three times out of five. Odd that it was the same each time, but I'm not going to say it's wrong.

The title of this post is completely misleading, it's sounds as if someone has some concrete evidence that this campaign is a sham, while all the person has done is just conduct another survey of his own in an attempt to prove them wrong. In my opinion, just another survey is hardly any evidence, and his attempts in proving them wrong with just a 1000 people taking his survey is an epic fail.

Reflexx says:

It wasn't even 1000 people. It was three experiments with a grand total of 1000 people if you added them together.

And at the same time, the guy says that MS used too small a sample size. LoL

eortizr says:

Bing sucks, never give me the result I'm looking for, I do the same search in google and the first link is always what I'm lookin for., don't beleive it? try it for your self.  

"don't believe it" try it for your self"---- is that a bing it on challenge?

 

I did. Bing gave me what I was after. Google didn't. Now what?

Reflexx says:

Wow. I think that this is the first time that WMPowerUser everhaf a better article (about the same subject matter) than WPCentral.

MS's response did cover things that aren't pointed out in this article. It makes me wonder if Michael even really read the MS response.

joltcoder says:

There are lies, damn lies, and then there are statistics.

ehalo21 says:

I'm a huge fan of Microsoft but I prefer to use google. If I'm on my windows phone I don't mind being but other than that I just rather use google.

jlynnm350z says:

Lol, should have known, let me guess the lawyer likes Google better than Windows. . . Hmm I wonder

garysvb says:

This is an example of sample bias by an attorney attempting to drum up some business in frivolous lawsuits, that's all.  You may as well apply the BIO challenge to users of this website and give Bing the win. Bottom line, the biggest problem with the Bing It On challenge IMHO was that any knowledgeable user can pick out Bing results vs. Google results just by looking at the images that Bing provides in the search results that Google does not. 

ltjordan24 says:

   I took the challenge way back when. I think they had three rounds and of the three, I chose Bing results twice. In real life usage, I would say that it's 50-50 now (whereas I used to use Google exclusively) because Google's results aren't as good as they used to be and Bing's results have improved considerably and sometimes, I find what I'm looking for on one but not the other and vice-versa.

larry213 says:

I have taken the Bing challenge a few times and Bing always wins for me. I do prefer the results I get with Bing over google for my normal day to to day searches but for some reason I still use google to search for porn. Go figure

Gurggles says:

I barely came out with Bing on top.

glassadam says:

I took it a few weeks ago and was surprised that I chose Bing over google 4 out of 5 times. But I wonder, when did MS do their tests that resulted in the "2-1" win? If it were a year or two ago, maybe the change in results today can be explained by the plain ol' passage if time. 2 years ago maybe Bing won, and MS has been touting those results regardless of the fact that perhaps today the results may be different. Just a thought.

shmuntz says:

I'd bing that hot girl in orange shirt all day ;) I can almost hear her wispering softly in my ear "bing it on, bing it on"
...shit, need to listen some old Enigma tracks now after looking at her for too long :D hehe

Skie77 says:

So are these worldwide stats or only US? I have heard that Bing works a lot better in the US in Europe fx, and in that case stats can vary a lot if the tests aren't based on equal populations.

jdawgnoonan says:

I use Bing every day at home and at work. But, when I took the challenge at a Microsoft store and at home, Google won 5 of 5 times both times. I like my free tshirt though and don't mind advertising for Bing.

HM02 says:

Wow, like every other major company in the world doesn't lie to get more consumers to use their products.
http://i.imgur.com/uzOo1Ku.gif
Bing is still better than Google, no matter the 'statistics.'
 

marmau says:

This ayers guy failed at step 1 of statistical/scientific method. Choosing a random sample. Going to mTurk is like asking "how important is the internet in your daily lives?" to 1000 random people on Facebook. Potential legal trouble?! You dont think Microsofts legal team didnt look over this? Microsoft commissioned an independent study and here are the results. Where's the deception in that? Its like 9 or 10 dentists recommending Crest. Finally, releasing the results of an uncontrolled website like bingiton.com is pointless. I guarantee the study participants didnt use it or even know what they were testing. Then again im biased. Haven't used Google in 6+ years. I'm not saying google isn't better (for relevancy at least), or Bing is far superior. But I do work for Microsoft (not the Bing team mind you). Anyway like I said im biased but this guy has no place to call BS on the MS study because his is BS - he failed at step 1.

Jazmac says:

Google is shaken up by Microsoft's march.

skshandilya says:

I used to use google and off late, all the searches I made returned stuff related to ads rather than what I actually wanted to search. I switched to bing, I am seeing more meaningful results.. but location searches suck in bing. I still use google for that. It seems to be improving. I stopped signing in to google. Google is definetly evil. M$ is also evil but is not as evil as google :)

WPCrook says:

Well, consider me one frequent user of Bing.

_Emi_ says:

I dont know why this article is even here... really if the guy steals people words in his books and try it to pass them as if he said them... you know, if you write a book, everyone know quotation marks have to be included... didnt this guy went to university?
 
so if he cant even do a simple thing like quotation marks, why should we listen to what he says? he doesnt eve know how statistics work, even I learnt about it reading couple pages this year. you need to optimize and balance everything, if this guy doesnt know about it... why does he even talk about it? does this guy think, when you see a presidential poll EVERYWHERE around the world, they ask millions of people what they think? no, they make small samples, and its enough to know what people think. this guy seems not to use much his brain :)

 

rasa_jitu says:

In term of search engine Bing is suck....in Malaysia it totally blocks the adult sites. Google isn't doing the same thing. In doing so it has blocked berita harian site, an innocent daily newspaper site..suck it on

chmun77 says:

Bing sucks to the core. Period. It never works well outside USA. Crap piece of engine.

falconrap says:

Michael should be ashamed of this article. If one reviews MS' full response and looks at how the Professor came up with his results versus MS, it's clear MS did things the right way and likely got a better representative sample, save the sex of the searcher. This article should have printed MS' response, as another WP site did.
 

MaulerX says:

I agree. Even other popular tech sites put things in perspective with Microsofts answer. This was presented in a one sided way and in fact there is no "evidence" of anything.

Reflexx says:

Agreed.
This is a bad, poorly researched, article. 
I'm really surprised that this was actually published on WPCentral.

> Have you taken the “Bing It On” challenge?
Yes. Google 5-Bing 0. My regular search engine is Google.

SleepyTheDon says:

If Google had a case im sure their lawyers would have sued along time ago and requested copies of their data pulls

SleepyTheDon says:

I've taken it a lot of times and I have to be honest... Its 50|50 split. Out of 40+ searches im split between the two. Who cares if MS stretched the truth I don't. I use both they both have their +'s & -'s

There could be a simple answer to this: selection bias. Microsoft's sample of the population is probably more broad, even if it might not be representative of the entire population. However, by turning to the Mechanical Turk, you have a group of people who are probably similar in several characteristics who get paid to work from home; I'm going to assume the age range and computer skills of this group lean toward younger and more computer savy folks.
 
Just my 2 cents.

slavrenz says:

Yeah, I don't feel like reading through all of the comments, but can you say self selectivity bias?

Pages