TRENDING : #VishalSikka

Why gut feel matters

Mathematical models anchor us to paradigms and inhibit innovation. However, human enterprise has the ability to bet on the success of a new technology.

K. Ramkumar
Published: 17, Feb 2013

K Ramkumar, Executive Director and Head of Operations & Human Resources at ICICI Bank, loves examining the other side of the traditionally accepted views. He examines everything that comes in his way. That has helped him to broaden his perspectives, something that he would have otherwise never done. Even while reading, he debates with the author by writing on the margins. Ram, as he’s popularly known to his colleagues and friends, believes that there is nothing more joyous than having an open debate with an equally passionate and experimental individual. The outcome is not important. What matters is a counter point - the other perspective to every viewpoint. A science graduate and a post graduate in Personnel management and Industrial relations, Ram is an ardent sports fan. He prefers to be in the game as it keeps him engaged with others. He also enjoys making short documentary films.

Early last month, the Washington Post linked to a remarkable mea culpa by Olivier Blanchard, chief economist at the International Monetary Fund (IMF). He admitted their predictions on which way the Greek economy would go had gone awry. [See the IMF paper.]

To tide over the economic crisis Greece is now going through, the European Central Bank and the European Union (EU) mandated the Greek government to cut spending. Econometrics wonks at the IMF got down to work and deployed mathematical models at their disposal to predict what could possibly happen. When the number-crunching was done, their models indicated if the Greeks cut spending by 100 percent, the economy would shrink 50 percent. As things turned out, it shrunk 150 percent. And that is the problem with mathematical models.

We all have this strange delusion that anything stated in numbers that has a formula or an algorithm must be objective and a statement of fact. Any data or model that is not put through the crucible of human judgement is useless, when it comes to making calls about the future.

The big picture can never be gleaned from just analytics. Models create sameness, while human judgement supported by analytics brings in enterprise. Models, people believe, cut down risks. But because of standardisation they actually amplify risk.

What do I mean by that? Think fixed telephone lines. If a mathematician were to build a model on how the business of telephony would evolve, it would perhaps take in all of the variables and come up with a set of numbers that a business can be built on. But could any model have predicted the emergence of cellular phones? For that matter, a hundred years ago, what mathematical model could have predicted air-planes would be the dominant mode of global travel?

Models anchor us to paradigms and inhibit innovation. However, human enterprise has the ability to bet on the success of a new technology. At worst, it puts to risk a single organisation, never the whole system.

Jean Piaget the legendary French philosopher, in his treatise on thinking, argued that analytics helps in comprehending the relationship between cause and effect, thus helping solve problems. Analytics connect the past to the present consequences, much in the same way as the rear-view mirror of a car. He established that conceptualisation on the other hand, helps in figuring out weak lines that connect the present to the future and thus aids prediction. Conceptualisation also amplifies weak signals in the environment. Analytics is akin to the rear-view mirror.

Nobel Laureate and behavioural economist Daniel Kahneman dwells in detail about the futility of trusting the analytical mind to the exclusion of the intuitive mind in his book, Thinking Fast and Slow. He, through experiments, establishes that the analytical mind is useful in assessing risks under standard and repeatable conditions. He proves it is ineffective in assessing both risks and opportunities, when the environment is non-standard or involves assessing human behaviour or is volatile.

It is here that the intuitive judgement trumps over analytical models. He argues eloquently that markets and humans often behave irrationally. Hence over-dependence on rational models can lead us to missed opportunities or ignored threats.

Only in science can data be extrapolated into the future and reasonably valid predictions be made. Even here scientific models struggle to predict volatile events such as weather and earthquakes. This is where scientists who have a “nose” to support their inferences from the models, win over pure model-based ones. Try predicting the course of a sporting event based on any model, and the perils of model-based thinking will hit you hard.

We will call a few wrong but will also get a few correct. But if we call everything on the basis of the need for certainty of results, then we will call everything late and lose competitive advantage. When business leaders get stuck to models, especially those built by others or from the past, they will have nothing different and concrete to offer. Sticking to the past cuts risks or normalises it, but will not help call the future and break out. History tells us that no one learns from others or from their own past.

  • Balasubramanian Iyer

    Extremely interesting write-up on GUT feeling. GUT feeling or intuition is the foundation or the base for many of the inventions, discoveries or the mathematical models which are mentioned here. When people say that certain inventions or discoveries happened by accident I would say people trusted their GUTS or intuition or to make certain things happen.

    on Nov 6, 2013
  • Why gut feel matters | The Other View

    [...] We all have this strange delusion that anything stated in numbers that has a formula or an algorithm must be objective and a statement of fact. Any data or model that is not put through the crucible of human judgement is useless, when it comes to making calls about the future. Read more [...]

    on Aug 12, 2013
  • Salil Chugh

    I agree with you on the premise that everything stated in a formula is not a statement of fact. Also, would agree that judgement should lead rather than be led by models.
    Had a couple of points to add on the subject. The way people use/ overuse/ under-use these techniques tells us interesting things about human behavior. These are in a way bigger reasons for unfavorable outcomes rather than the models themselves. : -
    1. Models are a tool for comprehension and not meant for deterministic prediction- Models are as good as any other tool available for comprehending relationships between events. The only difference is description of relationships is through quantification rather than through the use of literary language. Hence if restricted to comprehending relationships they actually aid decision making. The operative word still is aid. Enough examples exist also when models used correctly have actually provided competitive advantage to organizations.
    2. Manifestation of halo effect leads to misuse of models -In a way over dependence on models is a manifestation of the halo effect-not of people but of models. In such a situation they cease to be tools but become brands. Somewhat akin to popular stock pickers having a huge fan following. In both situations, people want to outsource comprehension of situations to something/someone else. Over a long period of time this behavior of few individuals gets institutionalized in the form of incentives, leading to the problem faced in the next point.
    3. Incentives leading to misuse of models– Often, incentives lead to misuse of models- standardization of processes based on models rather than on empirically proven results ( models do have empirically proven results as part of their solution). Example- During the recent crisis, the banks went by the rating agencies that rated the pools. Internal processes would work in a way to insure the buyer of the risk as long as a reputed third party rated the model based pools well. That led to over-dependence on outsourced wisdom. This was a bigger issue than how the wisdom was formed.
    4. Decision on application situations of models- In a way how and in what situation one applies a model to is a combination of art and science. Is the maker deploying a probabilistic output to give a deterministic result? That would be a wrong modelling decision. Here, the modeller’s decision to apply it to a situation without addressing assumptions needs to be questioned rather than the technique.
    5.Gut and its synergy with comprehension- Finally, there is a strong role that the subconscious mind plays in developing gut based decisions. Also, as discussed above, data/ analytics/ models help in comprehending the situation- in a way aiding the subconscious mind to evaluate the pros and cons of different actions. These linger in the person’s mind. The “Slow Hunch”- the example of Darwin coming up with the Natural selection theory much earlier before he realized it is a case in point. Refer the link:
    Hence, in most situations, as long as models aid and not replace comprehension they may help to develop gut based decision making. Therefore, in those situations, the two may actually work better together as long as both play their desired roles in the decision.

    on Mar 2, 2013
  • Sudipto Nag

    Sir…Indeed a very provoking thought,history shows more of devastating failure than greatest victories are backed by data analytics and mathematical models.Mathematical models also limits and orchestrates the thinking paradigm.The more experienced and skilled you are the more information you can unwittingly process and react to. Often seen gut feeling is positive about finding new opportunities and writing success never witnessed before.Renowned researcher and a famous author Garry Klein reveals in fact 90% of critical decisions we make is based on our gut or instinct.

    on Feb 28, 2013
  • Anindya Karmakar

    This blog in NYT titled "What Data Can't Do" by David Brooks provides another forceful argument in succinct points. While it is well articulated by many that data struggles with social and context (more in the link), what I liked were the points on how data creates bigger haystacks and how big data cant help in solving bigger problems.

    on Feb 27, 2013
  • Mandar M Tambe

    very true. Gut feel does matter. Otherwise Steve Job would not have pursued his idea for i phone, i pad and i pod. There was no market survey for the customers' needs and requirements. What Steve offered was not 'measurable' at that time with market survey. It is well-known that what is not measurable, is not predictable. It was his gut feel about the customers' future inclination and appreciation, created one of the greatest innovative companies - Apple.

    on Feb 26, 2013
  • gopalan

    Key is unpredictability of the environment where models would possibly fail. If it is always possible to use a mathematical model, then there is no need for business leaders. But there is always a downside risk to certain decisions, which get validated by these models. I would feel that there are enough gut feel going wrong as much as model-based decisions being ineffective.

    on Feb 25, 2013
  • Vishal BATRA

    Sir It's been very well put across by you on gut. My belief is that while what ever you have gained as an experience in your professional career, there has been data and analytics involved when you are playing with the numbers and that is just a direction which set you in motion to take a view but in the working environment, there are a host of intangible factors which actually needs the gut, the intuition which you develop over which acts as a catalyst and garnishes the action to be taken. In today's dynamic business environment when there are enough complexities, one needs to look at other variables surrounding the environment and bring in the threads together coupled with data, past history and the gut for moving forward. It's always said in the markets that past performed may not always be factored while looking at future returns. Just wanted to add that had it been that my brother Capt Vikarm BATRA (PVC) during the assault on peak 4875 looked at previous attacks where in we lost officers, he would have never led the attack from the front but it was his gut that he would win the peak for India and that made him attack and hoist the Indian flag there.

    on Feb 25, 2013
  • Pradeep Mukerjeepradeepmukerjee

    Like everything else in life, balance is the key. The data, the analytics sharpens the gut.....which is required to take the decisions based on an appreciation of the bigger picture.... Look forward to reading the others Ram.

    on Feb 25, 2013
  • Chandra

    Well said Ram. Gut feel matters. But like all else, only gut feel may not be enough. If we can 'garnish' a well cooked analytics model with that rare 'gut' that we can hone only over years- you surely will have the best of decision dishes.
    Gut feel tell me arguing for gut feel is rather rational!
    What says you?
    Chandra @ IBM
    P.S after writing down my post I scrolled through a whole lot of unconditional agreement with your wonderful post. Wondering if I am being a bit of spoil sport. But 'gut' tell me all will understand.

    on Feb 23, 2013
  • sujata deshmukh

    Very nice start Ram. Looking forward to the follwing 29 blogs. Interesting experiments leveraging gut feel or intution of groups of people have been shown to be more accurate than most mathematical models. This applies to estimating the weight of a bull as well as whether a new clothing line would sell well. Most acts of creation are about stepping out of boundary and loooking at possibilities instead of probabilities. Here is raising a cup to possibilities and putting words to gut.

    on Feb 22, 2013
  • Srinivasulu

    Ram, No need for me to reiterate on the "Great Start ".....
    Your thoughts truly balance a Leader between one overly dependent on data and Analytics and the other predominantly chosing "GUT or Intuition or Over all Feel /Judgment "
    We come across many leaders with never ending appetite for data, only to become more indecisive or get their decision wrong at the end... There are friends at other end of spectrum who are allergic to data ( as they feel it as threat to their judgment capability/authority ) and tend to rely on GUT feel.,
    In dynamic business scenarios, the problems are also varied in complexities and variables and its critical to connect the dots for the Big picture to emerge.
    Can't say "Dil mange more" having been with Coke!!!

    on Feb 22, 2013
  • M.S.Krishnan

    Congratulations on a great start Ram!! I really enjoyed reading your blog and the numerous comments from other members. As noted by all here, I cannot agree with you is impossible to capture the complexities of economies or the current business ecosystems in models. Hence you are on the dot in cautioning business leaders in their dependence on analytics... However I believe this applies more to "predictive analytics". In the present context, when we talk about Big Data Analytics, there are usually two components to it: the first is about bringing new information almost realtime to the decision makers e.g. pattern of customer voices on the social network platforms or an unstructured data on the actions of a competitor, these need not always be plugged directly into predictive model to pop out a decision. In fact the new information (even though noisy) and the increased level of transparency about the ecosystem in which a decision maker/business operates can aid the "gut feel" of the decision maker to connect the dots by amplifying weak signals. The second component of analytics is about predictive modeling that draws from past data.
    In my opinion you are correct in stating that predictive models based on past data may not be relevant any longer given the pace of changes in our business ecosystem. However we certainly need to explore the emerging technologies in analytics that enable businesses to get real time insights on the changes in its fact this first component of analytics can in fact help the decision maker in his judgement. In the past it was impossible for us to get these insights efficiently and the emerging technologies around Big Data can enable this.

    on Feb 21, 2013
  • Arun Raman

    Its a mix of both. A totally blue sky approach will not help. Equally, too much data (paralysis of analysis) is not helpful. The situation should be examined and based on it, the decision model to be used. Few persons get it right all the time.

    on Feb 21, 2013
  • K.Ramkumar

    Dear Friends, I am very happy that so many eminent people have joined in and are enriching the theme and the experience for all of us. In the final analysis if we are asked to choose amongst our 5 senses, will we prefer any one more that the other? I guess no. Each faculty has its value and hence cannot be traded off. However in the same breadth if we over emphasise one over the other then over a period the unused or infrequently used faculty looses its potency. It is also the appropriateness of which one and when. Ear for the music, painting for the eye and taste for the tounge. if we want to use ear for everything to the exclusion of the others we will miss a lot. So gut feel matters as much if not more than analytical faculty. The argument is not that one is superior to the other. Conceptualisation and Gut has greater value to comprehend future and analytics aid us with problem solving in the present and also feed in to assess consequences of choices made. Thanks once again for this enriching contribution by you all. I hope most readers jump in and have fun expressing themselves.

    on Feb 21, 2013
  • V Manjunath Rao

    Fantastic Start Ram ! I thought you summed up well when you said Quote "When business leaders get stuck to models, especially those built by others or from the past, they will have nothing different and concrete to offer. Sticking to the past cuts risks or normalises it, but will not help call the future and break out. History tells us that no one learns from others or from their own past" Unquote. This is the key wherin many Indian Business loose to competition since the past has not been great and you build pillars on it.You will reach no were!! Looking forward to the next!!

    on Feb 21, 2013
  • Anindya Karmakar

    The numerous experiements conducted by Daniel Kahneman and Dan Ariely prove that events big or small can rarely be predicted using rational models. Decisions are made on irrational grounds and models but cannot factor in all the variables in one go. Models often bind us together for ease of predictions, but often these predictions are in the wrong direction. Predicting the future is almost impossible, but human beings are addicted to predictions. No one can blame them. After all the future matters more to us than the past. But whenever scientist, economists, experts have tried to predict using mathematical models, they have come out short. As Nassim Nicholas Taleb mentions in his book The Black Swan: If you hear a "prominent" economist using the word equilibrium or normal distribution, dont argue with him. Just ignore him.

    on Feb 20, 2013
  • Satish Pradhan

    Well said Ram! the challenge with what we call "logic" on which analytics is predicated is tyrannically restrictive in "sense making" ! Making "sense" has different "logic" which seems inexplicable in logical terms and hence is labelled "gut" or loosely illogical? That there is a different "sense making logic" that is about insights, inference, connecting apparently unconnected things and phenomena is very challenging to structured followers of the logical analytics school. And yet when the insights are expressed they often seem so self-evident! You rightly push us to discover and legitimise that capacity in ourselves.

    on Feb 20, 2013
  • karan jaswal

    Ram, I think GUT is inherently the driver of all decisions. GUT is that feeling in the subconscious which too has done analysis over a lot of data over a period of time. For example, in the financial markets, traders across the world take decisions based on granularly analysed and published data and they refer to same set, and end up as sellers and buyers - well markets will cease to exist if they all took similar decisions - so, is this not "gut". Whose is right and whose is wrong is there with the institutions to see - it's zero-sum at any point in time. Secondly, the output of data analytics is to ascertain if the GUT is right. Yes we may often be at cross-roads of GUT and Data. Which road one takes when analysed in hindsight reveals , if was right or not so right, profitable or not so profitable.

    on Feb 20, 2013
  • Shiv Goyal

    Leadership is all about having a gut feeling which has been bred on knowledge, experience, openess to change and "some" data - empirical or social. If life was mathematical and restricted to econometric models, there would never be quantum jumps, disruptive thoughts and technologies and men would have understood women !

    on Feb 20, 2013
  • jayakumar

    Some builds Ram.. - Hind sight is akin to 6X6 vision and hence most of us intrepret past data very well (all most to perfection and is infact considered a great measure of intelligence). A good example are the books written on the global financial crises where in every author seems to display his/her ability in perfect understanding of the reasons for the crisis. - However past data does provide an ANCHOR and some guidelines for looking at the future - Looking at the future without knowledge of the past is like walking into an unknown dark room... - Analytical ability indeed PREPARES us to predict future situations better and hence cant be discounted. Having said these, why "GUT" is important in decision making is because of the fact that No 2 situations are exactly similar and hence intution plays a vital role in decision making..... - finally; some thing to ponder about... if one were to group together all the decisions that have being made into two buckets - decisions based on data and decisions based on intuition/gut. which way will the scale tilt..... whats the view.....

    on Feb 20, 2013
  • H.N.Shrinivas

    You are so true, Ram. Incidentally, I was reading a very interesting book which echoes your views, called, A WHOLE NEW MIND, by Daniel Pink. He talks about the right and the left brain, and how, in the last 4 to 5 decades we have over emphasised on the left brain "intelligent" people for key positions in various walks of life, and how it is not proving to be the right choice. He provides research findings on companies who have taken a non traditional path to find talent who have a balanced approach to life with a well developed right brain qualities in addition to good analytical skills. I cannot agree with you more that gut feel plays an important role if not decisive in major decisions in life and business.

    on Feb 19, 2013
  • Vinayak M More

    GUT feel does matter ! GUT is not a replacement for the mathmatical models but it is a very important factor in decision making. Decision taken based on data and gut always leads to more accurate results than the decision taken based on data alone. Data analytics is based on past and past environment. However the same may not be applicable for times down the line as the environment is constantly changing and the factors responsible for the change varies.

    on Feb 19, 2013
  • Prof R S S Mani

    excellent !! very thought provoking & unique piece..

    on Feb 19, 2013
  • Kirtiraj Gohil

    So true.... I believe even with all the statistics and analysis - we still take our decissions with gut feelings - primarily as mentioned " markets and humans often behave irrationally. Hence over-dependence on rational models can lead us to missed opportunities or ignored threats" Sums it all... Aptly written...

    on Feb 19, 2013
  • Prabhakar

    Absolutely relevant and close to my own thinking ! I can surely feel and connect here. There can never be any substitute to the ' Gut Feeling' despite any amount of analytics and decision support model being available around. In fact, i would say that these decision tools are in some nature imaginative and largely influenced by the gut feeling of the designer. particularly, when there is point where one has to see the future radically different or patterm of human behaviour/lifestyle changes, connecting the dot is the key success factor and gut does play a key role.

    on Feb 19, 2013
  • Dr P V Ramana, ITM Group

    I agree completely with Ramkumar Some ideas that I have in this area of contemplation - One needs to be immersed in the business activity and be close to the ground for GutFeel to work. We are unconsciously absorbing various inputs, while simultaneously engaged in studying spreadsheets and project reports. When intuition says one thing and the numbers are speaking something else, I tend to follow my feeling. In my 51 years of work life I have seen this work very often. - When I need to follow up on my intuition and go against given wisdom, I tend to bet on the nearest "good" person I can lay my hands on and trust him or her to deliver the goods. - This ability grows as you grow older, and when Intution and a Good person work together every thing falls into place Ramana

    on Feb 19, 2013
  • Anand Vardarajan

    Mark Twain famously said "history never repeats but it rhymes for sure". We have a tendency to drive looking at the rear view mirror and end up ignoring the windscreen. Be it financial markets, businesses or sports the only golden rule according to me is "this time it's different". Mathematical models have the ability only to do incrementalism not innovation. It tends to ignore the human angle and takes a linear spreadsheet approach.. Like they say all that counts may not be countable and that is countable may not count. Take the case of a MNC soft drink brand which chartered outs it's plan saying an average Indian drinks 1litre water per day. We want only 5% market share of that. Like to end by quoting the same Mark Twain "lies, utter lies and statistics".

    on Feb 19, 2013
  • Sujatha Rao

    Very well articulated article regarding the significance of gut feeling especially with reference to matters relating to predicting future events or taking calls about the future for the following reasons: (a) The future is unpredictable many a time, thanks to the constant innovations or unknown calamities which take us by surprise or shock. (b) All the variables regarding the past event may not be available for the model to make an accurate prediction about the future. (c) We can not overlook the fact of human beings not being 100% rational beings, incapable of thinking of success or failure of an event in percentages of probabilities and hence veer towards one or the other. This human tendency intrudes with the process of over-reliance on rational models in decision making. (d) It perhaps may not be out of place here to mention that gut feeling also is an outcome of one’s overall experience and to that extent has a rational base to it, despite our inability to quantify the same in terms of each variable therein. Thanks Ram for making us think about this topic.

    on Feb 19, 2013
  • Prashant Gondhalekar

    Yes , I fully agree with the views expressed here. Its GUT feel which is very essential in decision making. I would like to draw an example from mythology of Mahabharata. Pandavas were asked to select from two options for fighting war... i.e Kourav Sena OR Lord Krishna. The pandavas opted to go with Lord Krishna and we all know what happened. It was a gut feel of the Pandavas that allowed them to take a out of box decision and it helped them win the war.... Regards

    on Feb 19, 2013
  • anirudh kamani

    i completely agree with this. Mahematical models are always built on past traces. It creates a vicious cycle and you may never come out of it. Challenging this becomes more pertinent in the fast changing environment around us. Working on first priciples, and having a zero base approach can help people get out of this. I have tried it successfully many times. Of course, one has to be prepared to be ridiculed if you wish to create the unheard or they say.....any idea which is not riducled may not be a path breaking one.

    on Feb 19, 2013
  • Ajay Pandey

    While the thoughts expressed in this article are completely contra to the general belief that exists and challenges many common schools of thought, I completely agree with it. Ultimately it is the GUT that matters. one cannot be trained to handle all situations and neither can one remember everything that has happened in the past when faced with an immediate challenge or a dilemma. The decision to manage the challenge in the as is moment only happens through the GUT, without considering any past analytics or future predictions. It is true it sports or corporates or for that matter our own lives.

    on Feb 19, 2013
  • Shashank Kasliwal

    Absolutely Sir, this is real intuitive living that I am practicing myself. Thank you for fortifying my way of living. Regards Shashank

    on Feb 19, 2013
  • Krishnan RA

    Thought provoking and encourages one to give a try some thing(k) new. 2. History also tells that referring to the past gives a learning or tips on what not to try ! . . .

    on Feb 19, 2013
Hindustan Unilever's Double Whammy
Will you buy the Mahindra Reva e2o?