Colombia v Columbia: How many people get it wrong?

tldr; I reckon about a quarter of people get it wrong but I’m open to being corrected…

I was recently in Colombia (not Columbia) and whilst I was out there I noticed wristbands and T-Shirts etc for sale with the “It’s Colombia not Columbia” branding similar to below.

Screen Shot 2016-11-29 at 13.31.36.png
Credit: Miami Herald 

Colombians were understandably annoyed by how their country was consistently miss-spelt and so this campaign was an attempt to highlight and change this. When I saw the campaign, I was curious to know the numbers behind the miss-spelling.

The tragic news of the Colombian plane crash gave me a chance to look try to do a quick and dirty analysis of the scale of the problem.

The plane crash being such a big, worldwide news event meant that the search data for that time around people searching for Colombia V Columbia would be likely to be overwhelmingly about Colombia, the country rather than Columbia the university, clothing company or the state.

To see quantify the scale of the problem and look at the worst culprits I compared Colombia and Columbia in Google trends for a timespan of one day.

Screen Shot 2016-11-29 at 13.49.26.pngscreen-shot-2016-11-29-at-13-50-15screen-shot-2016-11-29-at-13-49-36

If we assume that most searches around this day were about the plane crash and therefore should have been ‘Colombia’ then you could see that most people were getting it right when they searched with a hefty chunk getting it wrong. But that’s a big assumption, that most people were searching for the plane crash.

So I then looked at just those News searches in Google trends by narrowing down the category. That’s much less of an assumption.

Below shows searches for Colombia (blue) and Columbia (red) over the period of one day (the spikes are when news broke of the crash). On breaking news, the red peaks at around 35 relative to the blue’s 100 so 35 as a percentage of 35+100 = 26% = that feels like a fairly good finger in the air estimate of the volume of people who miss-spell Colombia. 

Screen Shot 2016-11-29 at 13.54.53.png

A quick Google search showed that even respected news organisations get it wrong over the spelling of Colombia. Interestingly, when I clicked on the links on Google’s In the news results, below, it looked like the Columbia spelling had been corrected on the Business Insider site (but not in its page title), on the Mirror (which presumably had initially published with Columbia unless Google autocorrected) and was still miss-spelt on the Manchester Evening News.

Screen Shot 2016-11-29 at 13.40.32.png
Google autocorrecting or news editors correcting?

 

 

Can search engines predict the future?

When we analyse what people search for on the internet, we’re trying to pick up on trends and topics people are interested in.

Man with crystal ball

A recent post I did about the rise of depression in the UK showed a rise in people searching for depression treatments in the UK over the last four years and just yesterday, a report into the rise of depression by the BBC came to the same conclusion.

On reviewing the two reports, I was struck by accuracy of  my search insight when placed alongside the BBC’s data.  Searches for depression treatment had indeed risen by around 40% mirroring the reports data from the department of health almost exactly.

So that got me thinking.  If using a freely available tool such as Google insights for search can predict with accuracy the state of the nation, months before official figures (as Google’s flu prediction tool can).  Can it be used to predict the future?

Taking stock

Studies have already tentatively shown that twitter could be used to predict stock market ups and downs and Google are obviously looking into something similar here  but have probably concluded that it would be illegal to do such a thing and no doubt would keep quiet about it even if they did find a trend.

For anyone wanting to persue the idea of using search behaviour to predict stock market trends however there is hope This study found a correlation between rises in search volume for a particular company and the volume of shares traded in that company the following week however it couldn’t predict the price (rise or fall).  I suspect that further sentiment analysis of those search terms however could well provide indicators as to whether a share price would rise or fall (so if suddenly a large number of people are generally saying IBM sucks or searching for refunds online, share prices would be expected to dip).

It’s not just the stock market that can be predicted though.  The health of the nation, what we eat, crowd numbers, housing demand, political motivations… The list is only limited by our imaginations and how much access we have to Google’s data. 

I strongly believe that teaching people how to mine this data will help to empower citizens and help them to make their own decisions about what’s going on with the country rather than relying on spin and the media.  The data revolution starts here!*  Power to the people!

Oh, and by the way, if anyone is inspired to carry out further research into predicting the stock market by this blog post and is succesful,  just remember who inspired you and that my cut is 10%!

*Note to future researchers trying to find out where the data revolution started – it wasn’t here, sorry.

The intent behind search – Top 6 ways to tell

Do posh people get into trouble when they look for ways to ‘groom’ their children online? 

 “Officer, I was merely searching for Brylcreem and Old Spice I can assure you”.

What we say isn’t always what we mean, especially as sometimes the English language seems to be deliberately developed for the sole purpose of carry on style double entendres. 

Fnar! Fnar!

People who study and try to make sense of internet searches (like me) can sometimes find getting to the actual meaning of searches difficult (for example does someone searching for ‘laptop’ want to buy one? get one repaired? read reviews? complain or did they mean: lap dance?   

Even the boffins at Google HQ find this a difficult subject.  Their ‘did you mean’ function is used in a surprisingly small amount of instances and mainly to correct spellings (which are relatively easy ground).  When they stray from this safe ground, they find that determining intent behind people’s searches through a computer algorythm can be incredibly difficult (and sometimes amusing – see below). 

Google's latest 'sexist' version didn't quite take off in the way they'd hoped

So why is this a problem?  In a nutshell time and money.  Getting the intent behind a search wrong costs the user time (they have to redo their search or scroll through a heap of irrelevant results) which then costs the search engine money as there’s a risk the user will use a rival or get frustrated and give up altogether.  

The problem is  decreasing as searchers become more sophisticated though there will always be a need to determine search intent. 

So here are my top 6 tips on ‘determing search intent’ (or simply finding out what people mean)

1.  Keyword research.  

 Often, when people search, they’ll quickly follow it with a more defined search as they realise their original search was maybe too generic (so instead of the brain thinking hmmm ‘laptop’, ‘buy laptop’, ‘buy sony laptop’, ‘buy sony laptop on finance’ many people will actually search for thee phrases in succession without clicking a result so in effect, they’re using the search engine to bounce ideas off of as they go. 

We can capitalise on this by using keyword researh tools that allow us to order our keywords by search volume: 

 

So we can surmise by looking at the right hand number column that the large proportion of people looking for ‘laptop’ will be split between ‘laptop bags’ and ‘laptops’ followed by people searching by brand and so on.  

2.  Search by category

Searching by category (as shown below) can also help you to see the intent behind a search.  For example, a search for laptop under computer hardware would give you a fair indication behind the intent i.e. They were after hardware over accessories. 

 

3.  Allow for country variations

Searches will have different intent depending upon the country of origin of that search.  For example when we compare UK and Russian searches, we note that Russians have different brand intent as well as type intent – i.e. The vast majority of Russian laptop searchers actually mean they want a notebook. 

UK laptop searches
Russian laptop searches

4.  Allow for news

It’s important to know that your numbers aren’t being skewed in anyway when you look at them for search intent.  For example, if we were looking at the intent behind a search for oil using figures from over the last month or so, we may well be fooled into thinking that most people are concerned about oil spills rather than oil shares due to the amount of searches this takes up.  Google insights for search can show us peaks and troughs in search patterns and allows us to see where major headlines may have skewed things slightly so we can take this into account. 

Google insights for search

5.  Use other stats

If you use Pay Per Click advertising, you can test messages in realtime and see how it affects your numbers.  For example, including ‘free delivery’ in one advert and ‘cheapest on the web’ in another could give you some indication as to a customers preferences depending on which one the majority click on.  

Similarly, look at your own web visitor stats to see where people went on your site after using particular keyword. 

6.  Get advanced

There are new tools coming out on the market that aim to help us with search intent.  If you can afford the likes of Hitwise, then you’ll already have a valuable source of information at your fingertips in the form of ‘success rates’ or the percentage of people who clicked on a result after searching for a particular keyword.  

Microsoft have recently introduced a new tool that aims to tell you what people’s commercial intentions are behind a search.  The commercial intent tool shows the likelihood of people purchasing after typing in a particular keyword.

So we can’t please all of the people all of the time

But take heart.  As long as we keep this in mind when writing and creating content for searchers, there are ways and means to make sure that we please the majority of people, the majority of times.

Why don’t you take a look at some of the other posts on my blog?  Or make a comment below?  You can also subscribe to it by email using the really simple form in the top right corner of this page.

Top UK searches

But where's the porn?

An unusually sensible bunch of ‘hot topics’ from both the UK and US today.  Searches for ‘real news’ have overtaken the usual celebrities that clutter up the top spaces which is either a refreshing change or worrisome depending on your outlook.

However, were these trends taken literally by researchers looking back on them in the future they’d still reveal that ‘slipknot’ was slightly more important to us than the impending outbreak of nuclear armageddon in North Korea.

The trends are updated regularly here.