Skip to main content

Are You Data Driven!

Introduction
  During survival training the commandoes are taught techniques to avoid starving when cut off from human habitats while operating in dense forests, unending deserts and snow-clad mountains due to enemy action. The basic principle is that human beings can consume everything that is eaten by mammals. Wild or unknown red-coloured fruits and plants that exude white juice should be considered poisonous and avoided. The commandoes are trained to catch and eat even snakes. The trick is to catch the snakes alive and remove their poisonous, rotten and harmful glands before killing them and preparing a delicious meal. Similarly, the data needs to be defanged before consuming, lest it dooms the user. It needs to be sifted, vetted, analysed and evaluated.
  Information is power. Google’s mission statement is to “Organise the world’s information and make it universally accessible and useful”. The ever evolving technology makes it possible to compile and process infinite information, in near real time. These compilations can be accessed across the globe with the internet. Smart phones have empowered individuals to summon all the information on the internet instantaneously. Computers, internet, satellite based communications and smart phones together have provided wings to the information by generating high speed, volume and efficiency of data transmission that can create turmoil in seconds. Smart phones and the internet have made available a personal library, bigger than any physical library to each individual and have networked all individuals into one whole. Artificial intelligence (AI) is now providing an independent brain to this networked information that can be summoned instantaneously. The data is so intensely entwined with the communication technology, computers, internet, media, and AI that it is futile to discuss one without others. 
Nature of Data
  The data can be incomplete, inaccurate, obsolete, biased, insufficient, motivated or irrelevant and its compilation can be through archaic techniques. There may be gaps in the information that need to be plugged. Historical data on which the present information has been built, may lead to biases. Today there is more data that is easily and quickly accessible through sensors, space crafts and other devices. It can be disseminated directly over a variety of media. Big or more data does not necessarily mean objective or accurate information. Quality is always more desirable than quantity. 
  Data has a unique attribute. It usually creates an illusion of certainty and conviction. The numbers seem to be more accurate compared to other factors that are somewhat vague and generic, thus inducing confidence and over dependence on numbers. Over confidence and human judgement can lead to erroneous conclusions. Data evolves over a number of iterations. However, usually there is a rush to react to information even before it is analysed or interpreted, as there is continuous pressure to outsmart rivals and competitors. Our inability to exploit the data optimally can create mirages. 
  The source, motive and purpose of data compilation are very important. BREXIT is a classic case of data generation through ineffectual source. The United Kingdom held a referendum on 23 Jun 2016 and voted to leave the European Union (EU). The next day, it was confirmed that 51.9% of the British population had voted in favour of leaving the Union. This decision taken through a simple majority, wherein 48.1% had voted to maintain status quo, shook the international financial world.
  It is unfortunate that the decision for BREXIT was taken on the basis of simple majority vote by common men and women, who barring a small percentage, were neither fully aware nor competent to understand the implications of such a pull out. A citizen is more engaged in practicing and mastering his profession, earning livelihood and raising a family. His judgement is largely influenced by opinion makers. He has minimal understanding of the impact of international coalitions on national socio-economics, politics and security. 
  The EU comprising less than one tenth of the world population generates nearly one fifth of global GDP. Barring a couple of exceptions, the member countries have a very high Human Development Index. However, the United Kingdom government divorced, as apparently in the British opinion, the Union had failed to deliver prosperity to them, a premise for its creation, a fallacy that a common citizen is incompetent to appreciate. 
  The separation has manifested more to the detriment of Great Britain than the EU or the world order. It shall continue to have long term political, economical and social ramifications. It should therefore have been the prerogative of the bodies of experts to make recommendations on such an intense, technical and vexed issue after in-depth and detailed deliberations. Thereafter, the government should have ensured that decision to sever the cord was taken near unanimously as a simple majority in the instant case meant though less, but nearly equal numbers had opposed the proposal. Let us not forget that the United Kingdom had played a pivotal role in the raising and shaping of the Union, without any referendum. 
Information Technology and the Future
  Machine learning, quantum computing and AI will shape the future impact of data. Cyber attacks, hacking important communication systems and social media platforms are exploited both by the nations and non state actors. Social media acts as both a source of legitimate information and a conduit for misinformation and fake news. In a latest development, META (earlier ‘FaceBook’) a major social media company has shifted the fact check burden from professional fact checkers to users in a move to democratising information and eliminating allegations of biased content management. META’s withdrawal from professional fact checking is a watershed moment in the history of internet with far reaching repercussions. This action framed as an effort to defend free expression runs the risk of creating an unchecked atmosphere for increase in health related misinformation, politically tinged falsehood and security threats to thrive. It has opened the floodgates for lies, hate, disinformation, propaganda and conspiracies by firing all the expert fact checkers and declaring that the old ‘containment’ system did not work. The challenge especially due to lack of internet accessibility in geographical variations, rural and poor population who rely on unverified sources and linguistic and literacy diversity will be to sift the truth from lies. Developing AI powered solutions to detect misinformation and permanently placing teams of experts in the regulatory bodies are some of the possible solutions for damage control. Thus far there is universal conundrum to social media laws and regulations that needs to be now addressed with all seriousness.
  Geoffrey Hinton, winner of Nobel Prize in Physics 2024, also known as the god father of AI, has renounced all his work and job apprehending the terrible danger of AI that may soon engulf mankind. AI systems are intelligent, can understand and make decisions based on their experiences. In time to come, these systems will also have self awareness. First time on the planet, humans will be second most intelligent, as then AI will be more intelligent than us. These are algorithms based systems. When these algorithms interact with data, it produces complicated neural networks good at doing things. We are however; unable to decipher the process of their operations like we are unaware of the functioning of someone’s brain. One of the ways these systems can escape our control is when they start writing their own computer codes to modify themselves. That is something we need to earnestly worry about. 
Data Warfare
  Information has also been labelled as a weapon. Data Warfare, aptly named Information Warfare (IW) has been prevalent since ancient times. It involves generating rumours, spreading misinformation and false reports to break enemy’s morale and create confusion. It also involves gaining enemy information while denying the same to the adversary forces. It involves control and perception management. It is a conflict in which information and information systems act as both weapons and targets.
  Emerging technologies usually get embroiled to develop lethal arsenal during wars for gaining battlefield ascendency over the adversary before she is able to outsmart you. Information and communication technologies enabled capacity was also enmeshed to achieve strategic impact and tactical advantage over opponents during the world wars. It facilitated conduct of war on the global footing for the first time.
  Subsequently, there was a prolonged period of cold war. Cold war is a constant war. During this period information warfare that has now graduated to cyber warfare continues. This warfare is geographically boundless. It is a relentless campaign that commences years prior to the actual conflict, lasts through the war and persists beyond. It encompasses national, diplomatic, economic and military power. Psychological manipulation and cyber operations impact the outcome of operations and public perception.
Data and Decision Making
  Decision making is both an art and science of critical importance, involving a variety of operational, technical, financial, environmental and social, besides a host of other tangible and more intangible issues. Judgmental errors of even well thought out decisions, may have catastrophic consequences. Data is only a tool that helps to analyse a problem and facilitates decision making. It is not a solution to the problem. This will be aptly elucidated with a causality evacuation operation that went wrong with disastrous consequences. 
  A group of men was operating on a mountain range at very high altitude. The sun was right up and bright but the temperature down below was about minus twenty five degrees Celsius. In the coming days all the routes to these mountain ridges would be buried under more than twenty feet of snow and the mountain passes will be closed till the next summer. The only hope for reaching out to those deployed on these mountains will be helicopters that will operate only during fair weather conditions and daylight. 
  As the sun was setting, those housed at the highest ridge reported that a man was having severe chest pain and needed immediate medical care and evacuation. The boss of the organisation whose employees were operating on these peaks, was now in a dilemma as that was the last day for making any movement. Standing operating procedures drawn on the data of over a hundred of years of experience in this inhospitable terrain prohibited any movement from the midnight that day for the next six months as the snowfall could start any day. The first snowfall is usually very heavy and ruthless causing maximum devastation. If the ailing man was left to his fate, it could be fatal. After some brainstorming, he decided to attempt foot evacuation as helicopters did not operate in the dark. Two parties, one each from either end were launched. These parties would marry up at the midpoint and exchange further evacuation of the critically ailing casuality and return to their bases. The parties married up at the midpoint. Just then the weather suddenly turned hostile and a strong snow blizzard swept the area. Soon contact with the evacuation teams was lost. By the next morning, all the mountains were covered with thick piles of fresh snow. When none of the teams returned to their bases till late, helicopters were launched to look for the rescue teams. Helicopters could not spot anyone except the casualty who was lying on a rock top, nicely packed in a sleeping bag, tied to a stretcher. Apparently the rescuers had all gotten buried alive under the snow. Their bodies were recovered next summer when the snow melted. The Boss became insane reflecting on the horrific outcome of his decision to evacuate. It, however left an indelible mark of camaraderie on the team members as they never desert their wounded. There are folklores exemplifying this incidence that are still recounted at  functions. Despite years of data based caution, they suffered that night because in a given situation there can be innumerable other intangibles that get predominantly compelling for the decision maker, than merely data based decisions. 
Ethical Issues and Data Regulations 
  These days every organisation big or small; whether political, social, military, commercial or medical has their independent cyber cells. It is naive to expect ethical behaviour from these teams whose genesis is unethical. Besides their many other tasks, cyber teams are continuously engaged in media campaigns and perception management which in simple words means telling and spreading well thought out series of plausible lies with the intent to influence their target audience. The principal economic hustle and bustle; the share markets, the world over are manoeuvred with data manipulation. The data here works for its creators and injures the innocent, incompetent, unaware and unsuspecting end users who unwittingly make decisions reposing faith in the misleading information. Political parties engage cyber teams for their propaganda and governments manage perception of citizens by feeding biased data. Most of the media houses are owned by business tycoons and political parties that keep blaring misleading information. 
  Smart phones can snoop on the conversation in their proximity. Conversation between a couple in the privacy of their bedroom or the locations they visit with their smart phones in their pockets or bags get picked up on internet through their phones. Accordingly, conversation specific or location relevant messages and advertisements start popping on their phones. It is therefore a sham to talk about the ethical dilemma of data collection and its usage. In the digital era, individual privacy is a myth. Sensitive personal data is in the open public domain. Harvesting data from unauthorised sources to target individuals with structured narratives to influence their opinions, polarisation and sensitisation, as also for improved understanding of the public needs for better governance is customary practice. During technology driven wars, ethics always plays catch up as the conventions for the usage of any new technology get drawn in a later time frame after it has already been commissioned.
  During Covid 19 social media networks played a vital role in informing millions about the safety precautions and vaccine. On the other hand, social media is a breeding ground for health misinformation. World Health Organisation has dubbed social media as the pandemic’s ‘Infodemic’, highlighting the harmful effects of incorrect information spread on social media at lightning speed. Effects of miracle cures, vaccine side effects and pseudoscientific treatments campaigns are catastrophic, including delayed treatment, increased vaccination reluctance and resultant fatalities.
  Both national and international regulations to prevent misuse of data have become obsolete as the generation and dissemination of data has grown exponentially during the past couple of decades. There has been almost negligible review of the existing regulations, that too, mostly as fallout of the legal suits. Scheduled revision of these regulations to counter the challenges posed by the emerging cyber misinformation campaigns is yet beyond the horizon. There is a definite need to institute measures to counter social media campaigns and cyber troops engaged in subverting the minds of people both from within and outside the country. The challenge however is the competing needs of free speech and censorship. Clear regulations for data collection and its usage and international agreements, treaties and conventions for cyber warfare are essential. 
Human Mind and Data Education
  Human mind is more adapted to remembering stories than sheets of data. Many people may remember the wisdom from one of the epics that a young goat should never be cooked in its mother’s milk. A section of the people interpreted that the young goat can be cooked in the milk of another goat or cow; while the others interpreted that it is unhealthy to consume milk immediately after eating goat meat. Thus the individuals interpret the same information differently as per their perception. We overestimate the predictive power of data as if it is flawless. This can be disastrous. Our minds have confirmation and cognitive biases. Majority of masses have a lack of understanding of statistics, domain knowledge and ability to discern patterns and anomalies.
  Data literacy and education of all professionals who have to rely on data to understand statistical concepts, recognising cognitive biases, ability to critically evaluate data and methodology of data generation is essential. Decision makers should know the limitations of data and the consequences of relying on it. Better data practices involve capability to improve data quality by ensuring accuracy, completeness and true representativeness. It involves exhaustive data collection protocols, storage, analysis and audit for errors and biases. Proficiency in the art of communication avoids both; misunderstanding and misrepresentation of data. Raw data is information. Converting information into intelligence is a technical process. The reliability of the source of data and the likelihood of its correctness needs to be evaluated before analysing and drawing any deductions from data.
Conclusion
  During the past century, Homo-sapiens have graduated to an era of assured mutual destruction where super powers have the nuclear arsenal to counter attack causing a complete nuclear holocaust even after they have been struck lethally. Similarly, data is a powerful double edged sword. Data does not stab us, its misuse and misunderstanding definitely sinks us. Data helps innovation, research, understanding social needs, market trends, individual behaviour, forecasts, medical treatment etc. Data based research helped reduce child mortality rate and increase human life expectancy. The problem is not the data itself but how we use it. Misrepresentation and ethical dilemmas are our pitfalls and not the data. It is a powerful tool that can be both used and misused. 
  There is no objective truth at all. Everyone has their own truth which they wield to vanquish their rival. All social interactions are power struggles. Power is the only reality. Politician’s truth is power; profit is the truth for a businessman. AI generated frauds are already a routine. The societies need to be protected from the evils of AI and yet we have to exploit its goodness. Nations, organisations and individuals who avoid AI will get left behind. We have to remain optimistic about it. This happens with all the new technologies. Blotching technology driven data is similar to blaming the stick in the hands of a monkey for all the destruction caused by the monkey.

Comments

Ravi Shankar said…
Excellent read. Today the world of intelligent data has been scratched on the surface only. Thus the fear of use-misuse is alive in every layman. A regulated framework may reduce fears but policing is no better !! Actually we are at crossroads in personal data danger zones and ignorance of digital tools to protect us !!
Rakesh Singhania said…
Very indepth and thought provoking.
KAYJAY CALLING said…
This is my second reading of this very competitive and analytical Blog. Has very apt examples ..It needs to be read and heeded to by all Data users or consumers. Should also be published in Journals. MUST READ.
Akash Deep Anand said…
A brilliant piece of writing, giving insight into AI and data driven technology!
vishalbhatia said…
I am not IT or tech savy but after reading this, when it percolated down , it made sense. This is distinctly a MUST READ...
Col Anand Sachdeva said…
Excellent and knowledgeable post.... Very well written and well explained
Ghost writer said…
Great summary of how data can be used as both good and evil. There is over reliance on data based decisions and often as mentioned data is just a tool. It is not the solution but provides one input to making the decision.

Popular posts from this blog

ART OF IGNORING

    One day, when I was a Colonel and commanding a combat unit, I returned home very late for lunch. Although, physically back home for some food, my mind was still reeling with the stuff at work. I was unable to notice an old friend who had come on a surprise visit and was sitting in the lawn, chatting with my wife for the past two hours. He read my mind well and smiled. When my wife moved to the kitchen for laying the lunch, we got talking over the pre-lunch drinks.      My friend, also a senior colleague, explained that when we had joined the army twenty years back, as Second Lieutenants, ten men each were placed under our command. These men made one, two or three mistakes every day. The lapses were of simple nature like getting late for parade, untidy turnout, slouching or missing salutations. It was very intimate command. We noticed almost every lapse and were well trained to take corrective actions. Thus, at the end of the day, we were quite satisfied with...

MOTIVATIONAL MOTTOES

     I was ten years old when I joined Sainik School during Jan 1967. It was a boarding school with military like environment and discipline; all very different from the carefree living with the siblings, in the cosy comfort of parental care at home. Here, one had to fend for everything. Early to rise, a cup of tea, followed by physical training, then bath and change into uniform akin to the army, thereafter march to the mess in squads for breakfast, from there march-on to the academics block for classes and march back to the mess for lunch. There was a short rest period post lunch followed by games parade.  In the evenings we again took bath, wore mess dress and observed combined study period. The day concluded with marching for dinner to the mess and back to the hostel, polishing shoes and washing small garments before going to the beds at lights out till reveille early next day morning. There were intermittent doses of instructions and moral lectures everyday to m...