GenAI promises to provide greater productivity to knowledge work and a recent study on AI productivity may be the conclusive proof organizations need to invest in AI as a strategy. Harvard Business School conducted a study with Boston Consulting Group (BCG) measuring the performance of consultants assigned to tasks. The results were astounding. GenAI not only improved the quality, efficiency, and overall performance of high-skill workers, but it dramatically improved these productivity metrics for lower-performing individuals.
The AI Productivity Study
The Harvard Business School’s AI productivity study assigned 18 consulting tasks (verified by professionals within the industry) to over 700 participants from BCG to perform within a set window of time. The participants were required to take a survey assessing their abilities and identified as “lower performers” and “higher performers” compared to other participants.
These tasks were defined by four different core competencies: creativity, analytical thinking, writing proficiency, and persuasiveness. All 700+ participants were divided into three groups: a control group with no AI tools, a group with access to Chat GPT-4, and another group with access to Chat GPT-4 that also received an overview on prompt engineering (prompt engineering is method of utilizing Chat-GPT that results in more accurate and useful answers).
The AI productivity study conceptualizes the capabilities of AI as a “jagged technological frontier.” This concept is meant to evoke a visual of AI’s capabilities (or challenges). Some tasks are surprisingly easy for an AI to perform (such as concept generation) whereas others are surprisingly difficult (such as basic math). The study asks participants to complete tasks within two phases. The first phase consisted entirely of tasks within the frontier, whereas the second phase intentionally provided tasks beyond the frontier — anticipating the AI-assisted groups may perform better in the first phase and worse in the second phase.
The tasks involved conceptualizing and pitching a new product within the beverage and shoe categories. This included proposing new ideas for a market, segmenting the industry based on users, drafting a press release on the product’s announcement, and writing an inspirational memo to employees on why the product would succeed.
Study Shows Augmented Work Results in Greater AI Productivity
Participants using AI demonstrated remarkable improvements in productivity and quality. For the phase that required a number of tasks that had to be completed within a set time limit, the AI-assisted groups completed 12.2% more tasks on average. These tasks were also assessed for quality and the AI-assisted group received quality scores 40% higher than the control group with no AI assistance.
All AI-assisted groups in the study experienced increased performance, but — remarkably — higher performers saw a 17% increase in productivity whereas lower performers saw a 43% increase in productivity. This result suggests the impact of AI productivity could unlock significantly benefit organizations integrating these tools into their workforce.
Source: Harvard Business School
Study Shows All Workers Benefit from AI Productivity Tools
In the AI productivity study, AI-assisted workers outperformed the control group with the exception of one measurement: “correctness.” The study assigned a binary score of “1” or “0” indicating if the recommendations provided by participants were correct or not (verified by a professional within the field). This “correctness” measurement was distinct from “quality” or “productivity” measurements.
For tasks positioned outside the AI frontier, participants using AI were 19% less likely to produce correct solutions compared to those working without AI. This suggests that while AI excels within its capabilities, it struggles when confronted with tasks that require human reasoning, intuition, and creativity.
However, the study suggests “correctness” errors can be resolved if the participant has experience within the field to identify them before completion.
Conclusion
The AI productivity study’s implications are substantial. The AI-assisted groups consistently outperformed the control group in terms of response quality, efficiency, and productivity. The groups receiving prompt engineering performed even better than the group without any education on how to utilize Chat-GPT. The significant double-digit increase in productivity for all participants (especially for lower performing participants) has dramatic implications on how AI-assisted work can transform an organization’s output.
The study’s conclusion is clear: AI can and should play a vital role in knowledge work. The impact is so significant, organizations may want to consider how to integrate AI into their workforce. This may include creating new roles augmented by AI, restructuring the workforce around these new capabilities, educating workers on prompt engineering, and leveraging the technology across the organization for greater productivity gains.
This AI productivity study is further proof the AI revolution is poised to reshape how we think and work. It will reduce the costs associated with human thinking and reasoning and lead to transformative effects on productivity and problem-solving.
