The Unintended Consequences of Generative AI

Generative AI is less a single technology decision than a portfolio of management choices. Leaders need to decide where the business will accept automation risk, where human judgment must remain mandatory, and which use cases justify the cost of extra controls. The practical question is not whether to adopt GenAI, but how to make adoption governable at scale.

That means treating GenAI as part of the operating model, not as a collection of isolated experiments. Decision rights should be explicit: who approves a use case, who owns the data inputs, who signs off on outputs used externally, and who carries the risk when the model is wrong. Without that clarity, productivity gains can be offset by inconsistent use, weak accountability and avoidable escalation later.

The trade-off is between speed and assurance. Tight controls can slow delivery, but loose controls increase the likelihood of IP leakage, poor-quality content and reputational harm. CIOs and transformation leaders should press for a tiered policy model that distinguishes low-risk internal assistance from higher-risk customer, regulatory or financial use cases. That also creates a basis for prioritising investment in review workflows, logging and staff training.

A second management issue is capability preservation. If GenAI routinely drafts, summarises or recommends, what skills still need to be exercised by teams to avoid long-term erosion of reasoning and domain judgment? Next questions for leadership: which processes require human explanation by design, what evidence is needed before expanding use, and how will benefits be measured alongside risk, quality and workforce impact?

With great power comes great risks.
The water that carries a boat can also sink it, so goes the ancient Chinese saying. Generative AI is todayโ€™s proverbial water that can both propel humankind to greater heights and โ€“ if weโ€™re not careful โ€“ overwhelm us with its power.ย  One of the ways the technology can inadvertently cause harm is by worsening the climate crisis by way of energy consumption, say INSEAD professors inย thisย INSEAD Explainsย video series. Another is the erosion and even loss of problem-solving skills and creativity. Then thereโ€™s also the societal and legal implications of questionable and inaccurate content.ย  ย  ย 

1. Worsening climate change Phanish Puranam, Professor of Strategy GenAI presents two key challenges for businesses. First, the massive energy consumption required to train models like ChatGPT exacerbates the climate crisis. Unless engineers can develop more efficient hardware, this technology may hinder rather than help us achieve environmental goals.ย  Second, over-reliance on AI may cause us to neglect, even lose, valuable skills like problem-solving and creativity. Businesses must strategically determine which skills to retain and which to outsource to AI, balancing efficiency with the preservation of uniquely human capabilities. These are not just about economics but also identity in an AI-driven world.
2. Weakening higher-level reasoning Hyunjin Kim, Assistant Professor of Strategy While AI can automate tasks and enhance decision-making, early evidence suggests it may also impair higher-level reasoning skills. For instance, in financial firms, AI-powered predictions may improve investment decisions, but analysts’ ability to explain and justify those decisions might decline. This poses a challenge for businesses, as the ability to reason and communicate effectively is vital for stakeholder engagement. The key is to strategically integrate AI into workflows, ensuring that the technology enhances human capabilities rather than replace them. This may involve redesigning processes to emphasise human reasoning and explanation, even as AI improves decision-making.ย 
3. Producing harmful content and leaking sensitive information Theos Evgenious, Professor of Decision Sciences and Technology Management GenAIโ€™s ability to create vast amounts of questionable or inaccurate content can undermine trust in information sources and complicate efforts to combat misinformation. Harmful GenAI outputs, including hate speech, illegal content or polarising information can also have social and legal repercussions. Additionally, AI-generated content may infringe intellectual property rights or compromise individual privacy. Information leaks pose another concern. When users input proprietary code or sensitive data into AI models, there’s a risk of unintended disclosure. Until we have effective control mechanisms, businesses must carefully consider the information they share with these systems, and prioritise the development and deployment of ethical AI.

Edited by:

Seok Hwai Lee

About the author(s)

Phanish Puranam
is the Roland Berger Chaired Professor of Strategy and Organisation Design at INSEAD.
Hyunjin Kim
is an Assistant Professor of Strategy at INSEAD. She researches how firms can manage data and algorithms to improve their strategic decision making, and how these technologies change how firms compete and build competitive advantage.
Theodoros Evgeniou
is a Professor of Decision Sciences and Technology Management at INSEAD. He has been working on machine learning and AI for over 25 years.
Original Post

Enjoyed this article? Sign up for our newsletter to receive regular insights and stay connected.