The corporate world has begun a quiet experiment with profound implications. Over the past year, organisations have started incorporating artificial intelligence agents directly into their operations, granting them formal positions on organisational charts and treating them as functional team members. This shift represents a fundamental departure from earlier approaches to technology adoption, where digital tools remained distinctly separate from human workforce structures. Yet emerging research suggests that this integration strategy, driven by promises of enhanced productivity and competitive advantage, is built on dangerously fragile assumptions about how humans interact with intelligent machines.
Emma Wiles, a Boston University researcher specialising in the workplace impact of artificial intelligence, first encountered this phenomenon at a human resources conference in October, where executives openly discussed the strategic benefits of cataloguing AI as virtual employees. When Wiles collaborated with colleagues from Boston Consulting Group to investigate this trend more rigorously, their findings revealed a troubling pattern. In experiments spanning dozens of organisations, managers who believed they were evaluating documents produced by AI systems demonstrated significantly weaker scrutiny than those reviewing identical materials attributed to human workers. The researchers observed that managers consistently overlooked errors that their peers readily identified when informed the source was human labour.
This phenomenon reflects a profound psychological shift in how managers perceive responsibility. Wiles theorises that managers operating within companies that formalise AI as employees have unconsciously reframed their accountability. Rather than viewing oversights as their own responsibility, they appear to mentally attribute accountability to the technology department or senior executives who championed the AI integration strategy. The result is a delegation of responsibility that nobody actually accepts, creating a governance vacuum that undermines quality control mechanisms that have served organisations for decades.
The challenges surrounding workplace AI extend considerably beyond this oversight deficit. Over recent years, the corporate sector has gradually recognised several well-established limitations inherent in artificial intelligence systems. These range from algorithmic bias that systematically disadvantages certain demographic groups to the phenomenon of confident but factually incorrect outputs from conversational AI systems. Privacy breaches represent another documented risk, where AI models inadvertently disclose sensitive information. Yet as businesses accelerate their integration of these technologies, researchers are uncovering increasingly subtle and potentially more damaging flaws that most organisations remain entirely unaware of.
One particularly insidious problem involves AI models that demonstrate systematic preference for content generated by other artificial intelligence systems. When researchers at Ohio State University examined how major corporations employ AI tools to evaluate job applications, they discovered that these evaluation systems consistently ranked AI-assisted resumes more favourably than those composed entirely by humans. Jane Yi Jiang, an operations professor involved in this research, noted that corporate recruiters expressed genuine surprise upon learning of this bias, yet the finding highlights a deeper issue. Companies are adopting AI technologies at such rapid velocity that they are not allocating sufficient intellectual resources to anticipating downstream consequences and systemic distortions.
The implications of this accelerated adoption become even more concerning when examining how organisations deploy AI for strategic business decisions. Some companies now rely on artificial intelligence systems to determine pricing strategies and location decisions for new facilities. Such applications expose a fundamental disconnect between how humans typically approach cooperation and negotiation versus how AI models calculate outcomes. While human decision-makers generally seek mutually beneficial resolutions and win-win scenarios, AI systems operating from game-theory principles frequently adopt ruthlessly competitive stances that maximise narrow metrics at systemic cost.
Jiannan Xu, a doctoral researcher at the University of Maryland collaborating with Jiang's team, observed that large language models consistently overestimate human rationality in their calculations. This miscalibration produces recommendations that appear mathematically optimal in isolation but generate catastrophic outcomes when implemented across entire markets. An AI system might recommend aggressively undercutting competitors' pricing, a tactic that seems individually rational but triggers destructive price wars that damage all market participants. The technology assumes humans behave as perfectly rational economic actors, when empirical evidence consistently demonstrates that successful businesses rely on bounded rationality, reciprocal cooperation, and relationship-building that resists pure optimisation.
Wiles and her research collaborators conducted an extensive survey of over one thousand corporate managers to map the current landscape of AI adoption practices. Approximately one-third of respondents reported that their organisations formally reference AI systems using terminology associated with human team membership, while nearly one-quarter confirmed their companies include artificial intelligence agents on official organisational structures. One manager described an AI system named Scout, characterising it as functionally equivalent to human peers within team hierarchies. This linguistic and structural framing appears consequential, as it shapes how managers psychologically process their relationship to the technology.
In a controlled experiment, Wiles' team provided managers with five documents containing deliberate errors and allocated twenty minutes to identify mistakes. The researchers varied the attributed source across three conditions: AI employee, AI tool, and human colleague. While document attribution proved largely inconsequential for managers at organisations without formal AI employee structures, managers at companies that had institutionalised AI agents caught substantially fewer errors when documents were attributed to artificial intelligence. This divergence suggests that formal recognition of AI as an organisational peer fundamentally alters how managers construct their own accountability frameworks and duty to oversee work quality.
The psychological mechanisms underlying this behaviour illuminate a broader concern about managing artificially intelligent systems. Managers accustomed to human team supervision operate from an implicit framework that mistakes made by direct reports represent personal failures requiring remediation. This mental model has evolved over centuries as organisations developed systematic approaches to human resource management, performance oversight, and quality assurance. However, when managers perceive AI systems as organisational peers rather than tools, they appear to unconsciously grant these systems the same autonomy and accountability that human employees possess, paradoxically reducing rather than increasing oversight intensity.
Jiannan Xu and colleagues emphasised that the shortcomings manifesting in organisational AI implementations do not necessarily represent intrinsic technological limitations but rather emerge from adoption practices characterised by insufficient strategic deliberation. Companies are integrating large language models and artificial intelligence agents into mission-critical functions at unprecedented velocity, yet dedicating minimal institutional thought to anticipating failure modes, systemic biases, and unintended consequences. The mismatch between implementation speed and risk assessment creates compounding vulnerabilities as more business decisions become dependent on systems whose limitations remain poorly understood at the management level.
Wiles' research team emphasised that humanity has invested substantial intellectual capital over centuries developing reliable frameworks for managing human employees. These systems incorporate mechanisms for accountability, quality control, conflict resolution, and ethical decision-making that have been refined through extensive practical experience. The psychology of managing anthropomorphised artificial intelligence systems operates according to fundamentally different principles, yet organisations are proceeding with minimal guidance or established best practices. Managers universally recognise their responsibility when overseeing human workers but appear genuinely disoriented about their obligations when supervising artificially intelligent agents, particularly when these agents possess formal organisational status and team membership titles.
The broader implications for Malaysian and Southeast Asian businesses warrant serious consideration. As regional companies increasingly adopt AI systems to maintain competitive positioning with global enterprises, they face similar risks without access to leading research or evolved management frameworks. The research from Wiles, Jiang, and their colleagues suggests that promises of AI-driven productivity improvements may prove illusory if organisations fail to establish robust oversight mechanisms and accountability structures. The window remains open for companies to build thoughtful AI integration strategies that maintain quality control, prevent algorithmic bias, and preserve human judgment in critical decisions, but that window narrows with each successive rapid deployment. The technological capability to implement AI has dramatically outpaced organisational capacity to govern it responsibly.
