AI Productivity Unleashed: BCG Study Confirms Greater Efficiency With Chat-GPT

by | Generative AI

midjourney depiction of two workers utilizing AI Productivity tools
GenAI promises to provide greater productivity to knowledge work and a recent study on AI productivity may be the conclusive proof organizations need to invest in AI as a strategy. Harvard Business School conducted a study with Boston Consulting Group (BCG) measuring the performance of consultants assigned to tasks. The results were astounding. GenAI not only improved the quality, efficiency, and overall performance of high-skill workers, but it dramatically improved these productivity metrics for lower-performing individuals.
AI Productivity Unleashed: BCG Study Confirms Greater Efficiency With Chat-GPT: image 1
midjourney depiction of two workers utilizing AI Productivity tools

The AI Productivity Study

The Harvard Business School’s AI productivity study assigned 18 consulting tasks (verified by professionals within the industry) to over 700 participants from BCG to perform within a set window of time. The participants were required to take a survey assessing their abilities and identified as “lower performers” and “higher performers” compared to other participants.

These tasks were defined by four different core competencies: creativity, analytical thinking, writing proficiency, and persuasiveness. All 700+ participants were divided into three groups: a control group with no AI tools, a group with access to Chat GPT-4, and another group with access to Chat GPT-4 that also received an overview on prompt engineering (prompt engineering is method of utilizing Chat-GPT that results in more accurate and useful answers).

The AI productivity study conceptualizes the capabilities of AI as a “jagged technological frontier.” This concept is meant to evoke a visual of AI’s capabilities (or challenges). Some tasks are surprisingly easy for an AI to perform (such as concept generation) whereas others are surprisingly difficult (such as basic math). The study asks participants to complete tasks within two phases. The first phase consisted entirely of tasks within the frontier, whereas the second phase intentionally provided tasks beyond the frontier — anticipating the AI-assisted groups may perform better in the first phase and worse in the second phase.

The tasks involved conceptualizing and pitching a new product within the beverage and shoe categories. This included proposing new ideas for a market, segmenting the industry based on users, drafting a press release on the product’s announcement, and writing an inspirational memo to employees on why the product would succeed.

5 Point RAG Strategy Guide to Prevent Hallucinations & Bad Answers This guide designed to help teams working on GenAI Initiatives gives you five actionable strategies for RAG pipelines that will improve answer quality and prevent hallucinations.

Study Shows Augmented Work Results in Greater AI Productivity

Participants using AI demonstrated remarkable improvements in productivity and quality. For the phase that required a number of tasks that had to be completed within a set time limit, the AI-assisted groups completed 12.2% more tasks on average. These tasks were also assessed for quality and the AI-assisted group received quality scores 40% higher than the control group with no AI assistance.

All AI-assisted groups in the study experienced increased performance, but — remarkably — higher performers saw a 17% increase in productivity whereas lower performers saw a 43% increase in productivity. This result suggests the impact of AI productivity could unlock significantly benefit organizations integrating these tools into their workforce.

graph depicting AI productivity across study groups

Source: Harvard Business School

Study Shows All Workers Benefit from AI Productivity Tools

In the AI productivity study, AI-assisted workers outperformed the control group with the exception of one measurement: “correctness.” The study assigned a binary score of “1” or “0” indicating if the recommendations provided by participants were correct or not (verified by a professional within the field). This “correctness” measurement was distinct from “quality” or “productivity” measurements.

For tasks positioned outside the AI frontier, participants using AI were 19% less likely to produce correct solutions compared to those working without AI. This suggests that while AI excels within its capabilities, it struggles when confronted with tasks that require human reasoning, intuition, and creativity.

However, the study suggests “correctness” errors can be resolved if the participant has experience within the field to identify them before completion.

Conclusion

The AI productivity study’s implications are substantial. The AI-assisted groups consistently outperformed the control group in terms of response quality, efficiency, and productivity. The groups receiving prompt engineering performed even better than the group without any education on how to utilize Chat-GPT. The significant double-digit increase in productivity for all participants (especially for lower performing participants) has dramatic implications on how AI-assisted work can transform an organization’s output.

The study’s conclusion is clear: AI can and should play a vital role in knowledge work. The impact is so significant, organizations may want to consider how to integrate AI into their workforce. This may include creating new roles augmented by AI, restructuring the workforce around these new capabilities, educating workers on prompt engineering, and leveraging the technology across the organization for greater productivity gains.

This AI productivity study is further proof the AI revolution is poised to reshape how we think and work. It will reduce the costs associated with human thinking and reasoning and lead to transformative effects on productivity and problem-solving.

AI Productivity Unleashed: BCG Study Confirms Greater Efficiency With Chat-GPT: image 1
Shelf prepares organizations for a future with artificial intelligence. Our products offer a reliable, trusted, and secure way to scale organizational knowledge, boost performance, and improve customer satisfaction. Learn more about our transformative products and subscribe below to get updates from Shelf.
AI Productivity Unleashed: BCG Study Confirms Greater Efficiency With Chat-GPT: image 3

Read more from Shelf

May 23, 2024RAG
AI Productivity Unleashed: BCG Study Confirms Greater Efficiency With Chat-GPT: image 4 10-Step RAG System Audit to Eradicate Bias and Toxicity
As the use of Retrieval-Augmented Generation (RAG) systems becomes more common in countless industries, ensuring their performance and fairness has become more critical than ever. RAG systems, which enhance content generation by integrating retrieval mechanisms, are powerful tools to improve...

By Vish Khanna

May 23, 2024Generative AI
AI Productivity Unleashed: BCG Study Confirms Greater Efficiency With Chat-GPT: image 5 Prevent Costly GenAI Errors with Rigorous Output Evaluation — Here’s How
Output evaluation is the process through which the functionality and efficiency of AI-generated responses are rigorously assessed against a set of predefined criteria. It ensures that AI systems are not only technically proficient but also tailored to meet the nuanced demands of specific...

By Vish Khanna

May 22, 2024News/Events
AI Productivity Unleashed: BCG Study Confirms Greater Efficiency With Chat-GPT: image 6 Mannequin Medicine Makes Perfect, OpenAI’s Shifting Priorities, Google Search Goes Generative
AI Weekly Breakthroughs | Issue 11 | May 22, 2024 Welcome to AI Weekly Breakthroughs, a roundup of the news, technologies, and companies changing the way we work and live. Mannequin Medicine Makes Perfect Darlington College has introduced AI-powered mannequins to train its health and social care...

By Oksana Zdrok

AI Productivity Unleashed: BCG Study Confirms Greater Efficiency With Chat-GPT: image 7
The Definitive Guide to Improving Your Unstructured Data How to's, tips, and tactics for creating better LLM outputs