Dec/090
Pictograms or 3D plots? Neither – Just stick to 2D:
In general any visual representation of data is the most efficient form of putting your point across. A picture is worth a thousand words. It is true, not only in the art world but also in the business & academic worlds. Graphs, all sorts of them, stem, box, scatter, histogram, bar, pie & 3D are all used as windows for looking deep into the otherwise shallow looking & less informative pool of data points. Graphs have the power to harness the might of the human neural network to discover new dimensions which otherwise would always be inherent. This is exactly why one should be really careful while choosing the type of graph in order to paint the ‘best picture’
Many online & offline magazines make use of Pictograms and 3D plots in order to present data to the public. Pictograms and 3D plots though may look very appealing can usually mislead us while interpreting the data. A couple of examples of such cases are shown below.
The above example is a pictogram that aims to show the distribution of advertising budgets across TIME, Newsweek & US News. What is clear is that TIME attracts the largest amount of advertising spending followed by Newsweek & then US News. What is interesting to note is that having a look at the figures quoted in the picture tells you that the ratio of TIME’s to Newsweek’s share is around 1.6 and that of TIME/US News is around 2.9. This means that TIME should be only about 1.6 times larger than Newsweek and only 2.9 times larger that US News. But according to the pictogram TIME looks definitely more than the above ratios. What is this visual discrepancy?
Well what actually happened is that the picture in the pictogram was magnified and used for the three data points. While magnifying the picture both the height and width have been increased proportionally to avoid distortion. This means that the actual increase in size between the TIME’s figure and Newsweek’s figure is actually 2.56 (1.6*1.6, accounting for both height & width). Similarly the increase in size between Time’s figure and US News figure is 8.4 instead of 2.9.
So, our eyes, which capture the area of the pens rather than only the height, are misled to think that TIME’s share of budgets is way ahead of its competitors while it actually is not
The following is another example of an unnecessary use of 3D plot to present the data.
The aim of this graph is to showcase the drastic increase in the number of Starbucks outlets around the world. If you look at the graph carefully between 1999 & 2003 there is an increase from about 2000 stores to a little more than 6000. This is effectively an increase of about 200%. But the from the graph below it appears that the increase is much bigger than what the actual numbers suggest.
Using a 3D graphs brings in with it another problem, the third dimension. So when a 3D picture is magnified the height, width & breadth are all simultaneously increase to avoid distortion. This is the reason the picture looks much bigger than the 200% increase suggested by the numbers.
So essentially make use of the traditional 2D plots for the best and most accurate representation of your data.
Dec/090
Data Mining & its cousin – Inferential Statistics
Data mining as a field of significant academic research is on an exponential curve. Though the theoretical underpinnings of this concept have been around for a while the practical use of this field quickly reached a level where researchers and businesses are finding great value. Information explosion due to the success of the world wide web, general purchasing power of the common man, growing population, Globalization of commerce are only a few of the various reasons for this field to gain such popularity in such a short time.
Initially there was a lot of confusion about the difference between the fields of Inferential Statistics and Data Mining and researchers around the world immediately started working on producing tons of literature to address this issue. Though there is no single universally accepted definition for either of them, many researchers have defined these fields through their individual application perspectives. From what I’ve gathered and known about these fields, this is how I draw a line between them.
Difference between Data mining & Inferential Statistics:
- Data Mining is a field where you discover hidden patterns from already existing large data sets. These hidden patterns discovered are later used for analysis and decision making scenarios in the area of concern. The process of using the discovered hidden patterns is also called as ‘Knowledge Discovery’. Inferential Statistics is the field where you prove or refute a pre-conceived hypothesis (or a null hypothesis) by performing classical statistical methods on a sample of a given population size.
- Data mining starts at an already existing database (usually large datasets) and Inferential Statistics generates its own database using sampling methods on data set.
- Data mining methods employed like classification, clustering, etc., scan the entire dataset in search of hidden patterns while classical statistical methods are run over only a small section of the dataset (the sample).
The above point also infers that Data mining methods are more computer intensive as they have to run through large data sets and hence should be used only when really needed.
So when do you use data mining over inferential statistics? Well the answer is simple. If you don’t know what you are looking but want to make the best sense of the data you have then use data mining. If you know what you are looking for and want to back it up with proof by checking the data you have you should use inferential statistics.
Nov/092
Fast food Joints – Best places to learn about Team Work
Now, I know that your Doctor/Mom/ and the general 'elite' of the society are usually against the notion that there acutally can be something useful that could be derived from a fast food joint. But trust me on this, there is something that I did find something like that. Something that I probably would not ever find even after years of searching. No its not all the yummy looking/smelling but not good for health food but its the bunch of people that work as a team that is holding the joint together. Fast food joints are the best places on this planet to learn about an important trait that young starters need. - Team Work.
I'm completely convinced that Fast food joints are the best places for the young folk that most throng these joints to take a few lessons on Team Work. This is not a random observation. This is a hypothesis that I proved to be true after years of collecting data of statistical significance 
. No, seriously jokes apart I've been visiting this fast food joint since a loong time and I've visited at random times, a good combination of early mornings, late mornings, evenings and on both on weekdays and weekends. I can say that even at peak times never was my order held for more than 15 minutes, averaging at around 7 minutes in general. Believe me when I say peak time I'm speaking about at least around 10 orders being placed per minute and sometimes each order in bulk, Ex: 10 plates idli, 5 plates Dosa etc., Another observation made was that its always the same 4 people who takes the oders at the counter and then another couple of people inside the kitchen actually executing the order. What is another constant is there is continual chaos at these food joints irrespective of the time of the day. With all these unfavourable parameters one would would think how these folks have been managing to sustain its customers for all these years. The answer is simple ! They work as a team. They have all the qualities that an efficient team should have. I've read a few online resources that talk about the traits of a great team and when I tried to align these traits along with the few observations that I made at these fast food joint folks I actually got a good overlap of these traits.
These are some of the often found traits of members of a 'great' team:
1. Sharing of responsibilities & rotating roles as needed.
2. Showing of support, respect and trust.
3. Great understanding, immaculate communication and collaboration
4. Having a unified goal oriented approach in their actions/work.
Another very inspiring story is that of the mumbai 'Dabbawallahs' whose business model was studied by big business schools around the world. You can find more information on them here:
Nov/090
Getting a Police Clearance Certificate is such a pain – Stinking Bureaucracy
India - the second largest nation in terms of population,
- the seventh largest nation in terms of area,
- one of the fastest growing GDPs in the world,
- more than 500o years of rich cultural history,
- one of the world's highest producers of engineers & doctors,
- a nuclear superpower and a soon to be economic super power,
- largest democracy in the world,
and the list goes on...
All these are indeed things to be proud of, things that can instill an instant dose of patriotism in anyone. But all of this, everything stated above just vanishes into thin air when you see that things really don't work the way its supposed to at the grass root level. And I put the entire blame on the Indian Bureaucracy. This stinking organization has been pulling India back 2 steps for every 1 step that it puts forward. Rajiv Gandhi once openly stated that for every rupee that the Government puts aside to spend on its people, only 15 paise or 15% of it actually comes through. This is an observation that was made in the year 1988. 20 years since a lot of things have changed , most of it superficially. You try to dig a bit deep and you see the dirty side of the country.
Now a bit more context to my random ranting. I've recently had to deal with a government organization for my cousin's personal work. The job at hand was to obtain a police clearance certificate for this cousin living in another country to extend his visa. Initially, I was glad to get to know that there was an online application to be submitted. Progress ! right ??? Wrong what I had to do really is after I've submitted that application online, I still had to go to the office to submit the printed version of the online application form. Duhh!! Anyway, I did that assuming that it was just an 7 day ordeal as the website stated that the certificate could be collected on the 7th day after submitting the application. After the police verification was done it is the 3rd week now and I have received it yet and every time I call them to check the status, the information I get is that the report has not even reached them. When I ask them what the reason is, I get the most insensitive answer. I mean how difficult is it for a citizen of this country, who has not had even a single complaint against him ever, has all the right documentation to get a clearance letter from the local police stating his good conduct?? Well, it is clear that it is very very difficult. My friends and parents say that if I bribe them my work would mover faster, why should I give any extra money to anybody?
I understand many people love this country and would have a bucket full of great advices for me to join the 'system' and be upright and shine as an example. Why should I ? I mean I don't like to be part of the system, its a personal choice. Whats wrong with that? If everyone who has my opinion does the same thing then you will have a billion people who would be looking for jobs in this organization. I wonder what will change us, it cannot be organizational , it has to be spiritual. God save us.
Sep/090
Issue of Click Fraud
I've been reading a lot about Click Fraud lately and its been really interesting. I wanted to share a bit of my perspective on the whole issue and point you guys to some useful resources that I've found extremely useful. There are tons of news articles that project this concept as theDark side of the online advertising . Many also blame that efforts to mitigate this type of fraud is not in the interest of the Ad Networks as increased clicks result in incremental revenue. The key to understanding the implications of click fraud, as it is in any other concept, is in having increased awareness of the issue.
Click Forensics, a leading solutions provider for improving traffic quality for online ad campaigns, reported that though there is an overall dip in click fraud rate in Q2 2009 over Q1, click fraud traffic from sophisticated sources and scripted programs rose again in Q2 2009. This is evident from the rise of Publisher Collusion Fraud where publishers are employing sophisticated IP rotation methods or botnets in oder to inflate their commissions.
There is a huge incentive for the Ad Networks and Search Engines to look at this as a serious concern because any hint of discomfort advertiser will lead the advertiser and/or publisher to defect from the system which will eventually lead to revenue losses. Infact, Click Forensics President, Tom Cuthbert states that the increased diligence from the Ad Networks and Search Engines is the main reason for the click fraud rates to steadily decrease over the past few quarters. Google for example has set up a resource center to raise awareness about this issue. All the docs, presos and videos at this link are excellent reads on the topic.
So, from a technology standpoint, Search Engines and Ad Networks are doing their bit by developing tools and techniques in order to mitigate the effects of click fraud. What is lacking though are studies that look at user, publisher and advertiser behavior towards click fraud. Do they know what click fraud is? How do they validate erratic behavior on their campaigns? Is there a certain level of awareness to help them make informed decisions ?etc., There is enough proof to tell you that often advertisers complain that a seemingly erratic behavior in their campaigns is click fraud while the real reasons for such a behavior might be something completely different. Conducting research hence is important for the Search Engine and Ad Networks to develop a strategy around increasing awareness around this issue that actually is effective.
Sep/091
Prime Minister curious about Twitter
Apparently our beloved Prime Minister Dr.Manmohan Singh wants to know more about the twitter phenomenon and has asked the Minister of State for External affairs minister Dr. Sashi Tharoor, who by the way is one of the celebrity power users of twitter in India. Though its interesting how twitter is attracting celebrities to signup, what is more important is that by this means scores of people are getting to talk to these celebrities and like wise the celebrities get to connect to people more on a one-one setting. This is truly a powerful medium for communication and I hope more celebrities, especially politicians take a leaf out of the Dr.Singhs book and start thinking about adopting and using this powerful tool. I've complied a small list of some of our Indian celebrities on twitter, feel free to add to the list if I missed someone.
Indian celebrities on twitter:
Aug/090
SMX in India
SMX, one of the more popular and niche search marketing conferences in the world in coming to India. According to their website it is scheduled for November 2009. This is great news for the fledgling SEM industry in our country. This is a chance for folks in the SEM & SEO domain to talk, listen and network with some of the best brains in the industry. The SMX India site is not up yet but bookmark this space: http://www.searchmarketingexpo.co.in/
They also seem to have set up a twitter account here: http://twitter.com/smxindia
I'll try to update this space regularly regarding this event.
Time to get excited?
Aug/091
A fresh start !
A new start with a new website ! I did move all of the content from my old blog:
to this new place.
http://www.biztrivia.org/blog/
I will be more regular in updating this space and hopefully will meet and make many good friends
Aug/0711
Entrepreneurial India, Utopia??
Ok. So my last post was about the changing fortunes of the Indian IT companies due to a decline in the conversion rate. My closing line was quoting a hint to the Indian government to rise to the occasion to look for means and ways to prevent the IT boom going bust. But that’s not an easy effort. There are more than 1 billion people in the country. Out of this there are about 500 million children who are currently under the age of 15 who will soon be joining the workforce. Neither the government nor any large business empire can create avenues for employment and growth for these people. It is only possible by budding entrepreneurs.
There are far too few cases to talk about when it comes to Entrepreneurship in India. But the few cases are worth talking about. V.G Siddharth setting up a chain of outlets called the Cafe coffee day is a classic example. In days when coffee was considered to be a beverage only for the elder generation, Siddharth brought in a whole new hip look for the younger generation and having a coffee mug in your hand a style statement. He wanted the youth to come in and have their own choice of coffee, munch on their favorite cake or sandwich and even check their emails. Today the Coffee day is a successful chain not only providing delight to its customers but also providing thousands of employment opportunities.
A graduate of NDA (National Defence Academy) and after serving for some years in Indian army in the aviation branch, Captain Gorur.R.Gopinath decided to take voluntary retirement and return to Bangalore. One day he heard about a story of a Vietnamese woman who returned from Paris to start a helicopter service in Vietnam to provide access to remote locations. That made sense to Gopinath. He felt that the same concept was applicable to India as well where infrastructure remains poor and many parts of the country inaccessible. Air Deccan as an idea was born then. Today it is known as company that started a revolution in the Indian Aviation industry and as a pioneer in budgeted airline service bringing in the skies closer to the common man. Needless to say this company has brought in hundreds of thousands of jobs along with it.
Entrepreneurs create jobs. Jobs provide livelihoods. Livelihoods provide a sense of confidence to provide emotional security to people who depend on us. So, encouraging entrepreneurship should be a top priority for any developing country. If India has to see herself as a world economic super power then there has to be focus on encouraging Entrepreneurs. Institutes like EDI (Entrepreneurship Development Institute) are being established with a mission to provide training and guidance to encourage and ignite young minds to take up this responsibility. It’s an ideal situation a Utopia of sorts but in the end it’s a call the Youth of this Nation has to take!