Post Your Comment
The way ahead: Non-intrusive big data
With the impending, all pervasive data analysis techniques, there is now a growing fear whether too many lives will be controlled by too few
The coming years will see an increased debate on data privacy and secrecy, regulations that are structural as well as knee jerk reactions and most importantly, approaches that are ‘non-intrusive’ Image: Shutterstock
The three buzz words - big data, artificial intelligence and machine learning are getting even more prominent and at the cost of getting most of us jittery. If there was any unfound fear about AI replacing human intelligence initially and then dominating it, the recent annihilation of the chess engine, Stockfish, by Google’s AI-based Alpha Zero only adds credence to it. Stockfish, a chess engine that Grand Masters hone their skill against, is programmed with a collection of all the games played by the very best over centuries, was handed out a drubbing by an AI-based engine that only practised with itself for 4 hours an just knew the rules of chess! The implications are tremendous, the best of collective human intelligence stands little chance against the machine! AI would slice, analyse and process data which human beings cannot fathom! This is already happening. Where does it leave us? Most importantly, where do we see the next big areas of concern and the next innovations therefore?
Consider the case of farmer distress. Currently large areas in India are under severe agrarian stress while there are many more farmers, not hailing from distressed areas but nonetheless face perhaps greater crisis. Currently, our approach to solving these problems are time consuming, intrusive as well as reactionary. We spend an enormous amount of resource to identify the distressed farmers and in most cases even the distressed areas. As a result, any response by the Government is reactionary, made ineffective by leakages. With available resources, the big data analytics has already started to give us a fresh way of approaching the issue. One uses Satellite image data, applies AI on it to be able to generate real time productivity losses at individual plot levels! Increasingly, the benefits of using unstructured data to solve traditional problems are dominating the landscape. Techniques like these are endogenous response to the issue of data privacy and secrecy!
To leverage this what we need to see now, is the future of data analytics in conjunction with appreciating the business problem and understanding the data extraction and structuring process. One can expect a greater synergy of the three going forward, reflected both in terms of how we train the professionals as well as how they solve the problems. That is the future. Leverage the human intelligence with AI coupled with computational storage!
-Dr Bappaditya Mukhopadhyay is a Professor at Great Lakes Institute of Management, Gurgaon
