From the Imitation Game to Techno-Imposters: How AI Became the Ultimate Tool for Psychological Manipulation and Fraud

by Alexander Stein, PhD

 

“People will come to adore the technologies that undo their capacity to think.”
– Aldous Huxley

 

People have been swindling one another for thousands of years. In 300 BC, in what is acknowledged as the earliest recorded case of fraud, Hegestratos and Zenosthemis, two Greek sea merchants, attempted to perpetrate a maritime insurance scam by trying to pocket the proceeds of a loan advanced on an insurance policy covering their ship and its cargo.

While the types and techniques of fraud have evolved over the centuries, its psychological underpinnings remain unchanged. Fraud, as I have long maintained, is a crime of relationships, predicated in universal human propensities to be hoodwinked and manipulated. Trust cannot be broken unless it is first given [1].  One party willfully disadvantaging another through deception, betrayals of trust, or abuses of power is, irrespective of tangible losses or other material harms, intrinsically and dominantly psychological.

Understanding that each of us can be duped or gas-lit into thinking, believing, or feeling anything, is a driving principle in all forms of malevolent creativity – producing innovative or novel solutions with the express intent of harming others rather than for socially constructive purposes. It follows that the lengths people will go to deceive, manipulate, or betray others’ trust has no bounds.

Not surprisingly, advances in technology opened new vistas of opportunity for wrongdoers and brought increasingly more complex forms of fraud unleashed at previously unimagined velocity and scale. Misconduct that once required meticulous labor to execute – physical impersonation, for instance, or the hand-crafted forgery of financial instruments, documents, and signatures – evolved, beginning in the late 1980s, to replicating and skimming credit cards with a keystroke. The rapid expansion of the internet in the 1990s as a publicly accessible global network further enabled bad actors to commit identity theft, phishing scams, and many other sophisticated online fraud schemes through easy access to vast pools of information and the ability to target countless numbers of victims with minimal effort.

Today, advanced and emerging technologies, particularly the mass proliferation of social media and the advent of AI systems, chatbots, and automated agents deployed in propaganda and mis- and disinformation campaigns, are fueling rampant, ascendant fraud and corruption. The sheer compute power and capacity for hyper-scaling has already overwhelmed conventional bulwarks against economic crime, cyber warfare, geo-political destabilization, and other wrong-doing.

There is a substantial and still-growing body of evidence demonstrating how AI is already impacting contemporary life: algorithmic bias, wide-scale disinformation and social manipulation, the pillage of privacy, the weaponization and profiteering of personal data, intrusive apartheid-esque surveillance, impingements on cognitive liberty, deep fakes and the erosion of verifiable objectivity created by a deluge of synthetic slop and plausibly credible but actually incorrect data. There are a plethora of computational applications and services designed to monitor, assess, surveil, predict, manage, influence, and manipulate our thoughts, emotions, and behavior. Policies, regulations, laws, and social guardrails can scarcely keep pace, much less overtake the myriad moral, ethical, legal, and socio-political dilemmas generated by the wide-spread commercial release of these systems.

A Report from the U.S. Homeland Security Operational Analysis Center [2] warned that
“AI systems have the potential to destabilize social, governance, economic, and critical
infrastructure systems, as well as potentially result in human disempowerment” and can amplify “existing catastrophic risks, including risks from nuclear war, pandemics, and climate change.”

Throughout history, the introduction of new technologies, no matter how well intentioned, potentially beneficial, or seemingly benign, has also inevitably brought unforeseen abuses, malicious uses, and unintended consequences. At its simplest, any tool can be weaponized – implements invented for hunting, crafting, or building such as spears, bows, and hammers have been transformed from tools for sustenance and developing social infrastructure to instruments of warfare and destruction.

Debates about where responsibility lies or whom to hold accountable for harms and misuses of various tools and technologies seem irresolvable. The specious question of whether it’s guns that kill or the people shooting them has malingered for decades. Similarly, AI’s most ardent proponents reject that computational systems are in themselves dangerous. Problems, they claim, are usually a consequence of exogenous factors such as misalignment, technophobia, derelict regulation, improper implementation, or are dismissed as acceptable engineering bugs that will eventually be worked out.

A well-established heuristic in systems engineering claims that the purpose of a system is what it does. This view suggests that when a system is operating out of alignment with its intended design any resultant unintended consequences — caused by users or other actors who misunderstand the system’s purpose — ought to be readily correctable. The system just needs to be returned to a state where it can do what it does.

While challenges to ideal implementation can certainly present externally to the system, there is strong evidence that many of the risks associated with introducing AI systems and services into society at scale are largely attributable to wild over-estimations of  their actual capabilities compounded by tech leaders’ insatiable reach for power and wealth. AI is a human enterprise devised, driven, and shaped not only by experimentation and innovation, but by our hopes, fears, foibles, and fantasies. The root causes of most problems are invariably human, not technological.

This being said, AI represents a revolutionary category of human invention. It is not just another entry in the cannon of conventional widgets, services, or tools. It is not a spear or hammer, or even analogous to world-changing innovations like the wheel, electric light, vaccines or antibiotics. Unlike technologies conceived for purely utilitarian purposes, AI is the product of a unique mission to build “intelligent machines that could perform the most advanced human thought activities” as the Dartmouth Group proposed in the mid-1950s. It affects humanity as a whole and is being positioned, as I described in a 2019 article [3] as a paradigm shift in the dominant principles governing human toolmaking and innovation: “our tools evolved from mechanisms of necessity to those which can assist us and enhance our lives to now outsourcing self-awareness, self-knowledge, and self-agency.”

Not every tech product is intended to functionally replace human decision-making or provide a service we’re supposed to experience as essentially human-like. But the ultimate aim of many frontier AI systems is to match or surpass human-level intelligence across a wide range of cognitive tasks, often referred to as Artificial General Intelligence (‘AGI’), and to introduce machine agents that can operate autonomously in society in ways indistinguishable from humans.

In this regard, AI is a unique and unprecedented convergence of technology and psychology. It is a powerful technology designed to convincingly present as – functionally pretend to be – human.
The benchmark for assessing that capability is the Turing Test (originally referred to as The Imitation Game) developed in 1950 by Alan Turing, the renowned British mathematician and computer scientist often called the “father of modern computing,” following publication of his ground-breaking paper Computing Machinery and Intelligence [4] in which he posed the question, “Can machines think?” The test is intended to assess whether a machine could exhibit intelligent behavior equivalent to that of a human. A human evaluator judges a text transcript of a natural-language conversation between a human and a machine. The evaluator tries to identify the machine, and the machine passes if the evaluator cannot reliably tell them apart. The results would not depend on the machine’s ability to answer questions correctly, but on how closely its answers resembled those of a human.

In the mid-1960s, Joseph Weizenbaum, a German-American computer scientist working at MIT, developed an early natural language processing computer program he called ELIZA (after Eliza Doolittle, a working-class character in George Bernard Shaw’s Pygmalion who is taught to improve her communication skills). Originally intended as a method to explore communication between humans and machines, it became one of the first programs capable of attempting the Turing test with many users attributing human-like feelings to it, a phenomenon that came to be called the Eliza effect. Despite Weizenbaum’s insistence that ELIZA could not converse with true understanding, its ability to engage in fluent discourse convinced many early users that it possessed intelligence. Weizenbaum lamented that “ELIZA shows, if nothing else, how easy it is to create and maintain the illusion of understanding, hence perhaps of judgment deserving of credibility. A certain danger lurks there.” [5]

While ELIZA is almost laughably rudimentary in comparison to today’s large language model (LLM) chatbots, our willingness to suspend disbelief and impute prodigious human-ish capabilities which are in fact nonexistent is powerful.

In 1966, Weizenbaum presciently wrote:

It is said that to explain is to explain away. This maxim is nowhere so well fulfilled as in the area of computer programming, especially in what is called heuristic programming and artificial intelligence. For in those realms machines are made to behave in wondrous ways, often sufficient to dazzle even the most experienced observer. But once a particular program is unmasked, once its inner workings are explained in language sufficiently plain to induce understanding, its magic crumbles away; it stands revealed as a mere collection of procedures, each quite comprehensible. [6]

Then as now, we are beguiled by our impulses to anthropomorphize, be seduced by
clever but misleading marketing propaganda that masks and misrepresents a slew of nontrivial capability deficiencies, and become lulled into accepting engineering achievements as independent thought and agency.

A 2024 experiment conducted by researchers out of UC San Diego suggests that people are increasingly challenged to distinguish GPT-4 from a human. Participants had a 5 minute conversation with either a human or an AI and judged whether or not they thought their interlocutor was human. GPT-4 was judged to be a human 54% of the time, outperforming ELIZA (22%) but lagging behind actual humans (67%). The results underscore that LLMs like GPT-4 perform language-based tasks at a level frequently experienced as at parity with humans. People interacting with these chatbots are unable to definitively determine whether they are speaking to a human or a machine. The research [7] provides what is claimed as the first robust empirical
demonstration of an artificial system passing an interactive 2-party Turing test.

This experiment and others like it are interpreted to have implications for debates around machine intelligence and are generally taken to bolster claims that frontier computational systems are unequivocally on the verge of surpassing human capabilities in a range of areas. While this sort of research urgently – and correctly – suggests that deception by current AI systems may go undetected, it also tends to over-value the machine and subordinate the human as causal to that deception.

In my view, the Turing Test has never been about determining anything substantially meaningful regarding the capacity of a computer system but rather has always been an unscientific referendum on our omnipresent human susceptibility to being deceived. The test functions more like a Rorschach test in which the outcome is primarily indicative of the human evaluator’s psychology and provides only notionally significant information about the capabilities or quality of the machine.

Following from this, what if the purpose of the test were, conversely, to try to ascertain the extent to which we can or cannot distinguish what or whom we’re conversing with? The answer to that question could have value in returning important information about ourselves, for example, in creating protections against social engineering exploits, fraud, or other forms of deception and manipulation.

While AI applications can be directed to provide that sort of vulnerability feedback, the test’s primary purpose remains determining how convincingly a machine can mimic human communication, in part, to gather training data to refine systems’ future
unsupervised deployment capabilities.

For technologists and computer scientists, crossing a Turing test threshold is a significant achievement. Not only is the system performing as intended, it further bolsters the aspiration of developing computational systems that exhibit intelligence equivalent to, greater than, or even indistinguishable from, a human.

Beyond intelligence – computational cognition – tech entrepreneurs are now intent on commercializing capabilities for artificial emotions and approximated empathy as more applications enter mainstream use and a familiar, trustworthy human-like interface is expected or required. Of primary additional importance in next generation agentic systems is the extent to which systems and services are perceived not only as safe and relatable to people interacting with them, but which foster instant rapport and thus can blur, perhaps eradicate, any conscious sense of interacting with a non-human entity.

“Technology giveth and technology taketh away, and not always in equal measure. A new technology sometimes creates more than it destroys. Sometimes, it destroys more than it creates. But it is never one-sided.”
– Neil Postman

Will we develop systems and agents that can act and intervene in human affairs responsibly and fairly? Will nation states, governments, investors, commercial enterprises, private actors, judiciaries, legislatures, and other powerful stakeholders reach practical consensus on and be held accountable to violations concerning complex matters of international cooperation and governance, privacy, regulation, and legal, ethical, and other critical protections against and responses to abuses, misuses, and other harms? What are the consequences if we don’t or can’t?

Social communication and information networks running on advanced machine learning-based systems are already deeply embedded in our lives and are susceptible to both unwitting and deliberate abuses and misuses. User data – our most private and intimate details as well as banking or other data signatures and credentials – are regularly shared with companies, institutions, data brokers, and other third parties. Often, there’s no clear responsible party or agency for safeguarding the PII (Personally Identifiable Information) the systems harvest nor have robust legislative, legal, or regulatory mechanisms or AI data governance guidelines yet been broadly adopted to mitigate material and human risks or to provide equitable avenues for redress for harms caused.

We are, in my view, very far from overcoming the challenges of ensuring that the objectives and purposes of computational systems are compatible with and supportive of human values, goals, and preferences. To bad actors and criminal enterprises, this is good news.

Fraudsters are using AI-based techniques that traditional systems can no longer adequately detect or repel. Generative AI offers seemingly endless potential to magnify the nature and scope of fraud against financial institutions and their customers.

The availability of new generative AI tools is democratizing fraud. The dark web boasts a thriving cottage industry that sells scamming software to bad actors who can now easily and cheaply make deepfake videos and audios, and fictitious documents. AI’s automation capabilities are transforming the efficiency of criminal operations. The emergence of fully autonomous AI could usher a new era in organized crime, enabling malevolent actors to execute complex orchestrated attacks by identifying multiple vulnerable targets, hijacking their systems, and stealing valuable data from unsuspecting victims at scale, to launch massive money-laundering or trafficking operations, or exploit socio-political events for corrupt aims.

A 2024 report from Deloitte’s Center for Financial Services predicts that by 2027, GenAI could enable fraud losses in the United States to reach US$40 billion, up from US$12.3 billion in 2023, a compound annual growth rate of 32%. In the last three years, deepfake attacks increased a staggering 2,137%, representing 1 in 15 cases of all fraud attempts detected [8].

Hostile nation-states and mercenary adversaries aiming to disrupt elections or destabilize geopolitical affairs and transglobal commerce are likewise gaming AI to spread propaganda or push false or inflammatory information. Because chatbot answers depend on and reflect the data fed into them, the responses they give users can be manipulated by infecting the datasets with intentionally skewed or false information.

In these situations, AI systems are weaponized in ingenious and creatively malevolent but still fundamentally conventional ways: as a malicious tool. The system’s vulnerabilities are deliberately exploited to force the system to do what the malicious actors want it to; what it was supposed to do be damned. Sysadmin restrictions and other safeguards or countermeasures are supposed to protect against such nefarious acts, although wily actors understand how to defeat these by identifying blind spots and weaknesses.

But adoption or willful misuse of cutting-edge technologies for the knowing commission of nefarious or unlawful activities is, as alluded to earlier, entering an altogether different dimension. Frontier AI can now so effectively deceive us that trust in consensual agreement of validated truth and reality is being functionally eradicated. Advancements in AI blur the line between reality and fabrication, challenging our trust not just in digital content but in reality itself, including knowing or caring who is human or not.

“The point of modern propaganda isn’t only to misinform [but] to annihilate truth.”
– Garry Kasparov

We have willingly allowed a powerful but still largely unproven technology to be integrated into nearly every strata of society, commerce, and human affairs [9].

There is a growing body of social science research demonstrating that many AI-based
systems and services are adversely impacting children’s psycho-social development and
hamstringing kids’ abilities to learn and think critically, even as companies market and sell AI tools and services into the education and mental health marketplace. Technologies that appeared (or were hoped) to be solutions to the adolescent and young adult mental health crisis following the Covid pandemic, are now being understood as causal to, not reparative of, increases in isolation, loneliness, anxiety, depression, self-harm, and suicidality.

Advances in GenAI’s capabilities have also given rise to issues involving what MIT sociologist Sherry Turkle discusses as the “perils of pretend empathy and artificial intimacy.” [10] Not merely over-the-horizon conceptual challenges, these are currently iterated in real-world commercial ventures. There is already a robust profit-driven market sector producing sex-bots – sophisticated life-like artificial sex partners – and death-bots (also referred to as griefbots) – avatars that recreate the appearance, speech, and primary personality features of a deceased person.

There are now chatbot “therapists” and a wave of other related mass-market applications purporting the capability of diagnosing and responding to depression, anxiety, relationship travails, and various other mental health issues. Other chatbot services can generate human-like text responses based on a user’s customization to provide “realistic conversations” with AI-generated personas. The technology is presented to encourage – manipulate may be the more accurate word – us into believing that we can have relationships with it, and it with us — as depicted in the films “Ex Machina” (2014) and “Her” (2013) — in ways functionally and emotionally equivalent, possibly even superior, to those with other people.

All of which leads me to suggesting that we can plausibly consider AI as an imposter – meaning, the assumption of a false or disguised identity to deliberately deceive – a perspective adjacent to the philosopher and cognitive scientist Daniel C. Dennett’s caution against the problem of “using AI to generate counterfeit people.”[11]

A deep dive into the conceptual details and psycho-social complexities of what I call techno-imposturousness will need to wait. I only want to quickly spotlight here the bilateral, relational nature of this issue. Fraud, as I pointed to at the beginning, pivots on relationships. But now, we are entering uncharted territory where the fraudster is not a person but a computational system.

On closer examination, the culprit, the true agent and instrument of deception, is more properly understood to be the technology industry and, in particular, a relatively small group of individuals — the principal innovators, investors, tech and business leaders, and others in Silicon Valley whose personal beliefs, ideologies, and psychological proclivities in pursuit of a techno-utopian vision of the future [12] — who are guiding concept, design, research and development, operationalization, and commercialization. These tech evangelists promise magical solutions just as medieval alchemists believed the Philosopher’s Stone could transform base metals into gold.

Trust cannot be broken unless it is first given. AI’s high-velocity ascent is to a large degree driven by our collective acceptance of its anthropomorphized presentation, strategically enabled by the relentless delivery of savvy marketing propaganda masquerading as validated science. We have been given to accept that machine-generated simulacra of something human-like is not only good enough but absolutely equivalent. Imposturousness cannot succeed, no matter how brilliantly it’s executed, unless its victims agree to mistake or deny appearance for actuality. Society has been groomed to enable what may come to be understood as the most far-reaching and consequential fraud in history.

How does this happen? “People,” Professor Dennett says, “have a natural inclination to treat anything that seems to talk sensibly with us as a person … almost impossible to resist.” [13]

Referencing Dutch historian Johan Huizinga’s idea of the “magic circle”[14] – the space
in which the normal rules and reality of the world are suspended and replaced by the artificial reality of a game world – Edward Castronova, a professor of media at Indiana University Bloomington known for his work on the economies of synthetic worlds, suggests that “the membrane between synthetic worlds and daily life is porous … [and that] people are crossing it all the time in both directions, carrying their behavioral assumptions and attitudes with them.”[15]

In a recent article elaborating key distinctions between human and computational thought [16], I suggested that the project to engineer intelligent machines has always been linked to an idealized fantasy of pure cognitive decision-making. We are, it would seem, enthralled by the wish for a perfect brain-in-a-vat unburdened by the needs and challenges of the physical body and devoid of affect, irrationality, fallibility, vulnerability, pain, memory, or other vicissitudes of the human condition. Ambivalence and hostility toward our physical and emotional selves—the drive to eradicate or replace ourselves with an “improved” version—is woven into the history of civilization. Synthetic super-intelligent entities along with humanoid robots, cyborgs, and other human-like surrogates have long been an appealing leitmotif in sci-fi as a ‘solution’ to the ‘problem’ of being human. Once the stuff only of writers’ and filmmakers’ imaginations, they are now commonplace in the real world.

But what have we done in being able to actualize that? Magical thinking and willful blindness — mental devices in service of attempting to stamp out something in reality that feels overwhelming – conjoin. In “Remembering, Repeating, and Working-Through,” published in 1914 [17] (and elaborated in his 1920 book “Beyond the Pleasure Principle”), Freud identified our propensity for making and remaking the same mistakes over and over again. We are infinitely resourceful in recreating old problems in new ways, thinking that we’re solving them without realizing we’re not only preserving the original but often also creating a new one.

Techno-imposturousness isn’t equivalent to a Trojan horse, a boiler room, Nigerian prince, or charlatan’s grift. We are drowning in an ocean of fakery, a digitally conjured facsimile of water in a virtual world populated with computer-generated avatars. We have apparently decided and declared that all of that is real. Or approximately real enough. It is not. But the consequences are unavoidable and irrevocable. Being defrauded by our own invention is entirely self-generated and self-inflicted. The problem is not technology. It is us.

References

1. Stein, Alexander. Innovations and Strategic Applications in the Psychology of Fraud, 2023 ICC FraudNet Global Report — Fraud and Asset Recovery in an Unstable World.

2. Global Catastrophic Risk Assessment. Homeland Security Operational Analysis Center. RAND Corporation, 2024. https://www.rand.org/pubs/research_reports/RRA2981-1.html. (accessed 1 July 2025)

3. Stein, Alexander. Pitfalls of Outsourcing Self-Awareness to AI: What Leaders Need to Know. Forbes, Jan 6, 2019. https://www.forbes.com/sites/alexanderstein/2019/01/06/the-pitfalls-of-outsourcing-self-awareness-to-ai-heres-what-leaders-need-to-know/#673115e2318d (accessed 1 July 2025)

4. Turing, Alan. Computing Machinery and Intelligence. Mind: A Quarterly Journal of Philosophy and Psychology, October 1950

5. Weizenbaum, J. ELIZA: A Computer Program for the Study of Natural Language Communication Between Man
and Machine. Computational Linguistics 9(1): 36-45 January 1966

6. Op. cit.

7. Jones, C.R., & Bergen, B.K. (2024). People cannot distinguish GPT-4 from a human in a Turing test. ArXiv, arXiv:2405.08007

8. Lalchand, Satish; Srinivas, Val; Maggiore, Brendan; Henderson, Joshua. Generative AI is expected to magnify the risk of deepfakes and other fraud in banking. (29 May 2024). Deloitte Center for Financial Services https://www2.deloitte.com/us/en/insights/industry/financial-services/financial-services-industrypredictions/2024/deepfake-banking-fraud-risk-on-the-rise.html

9. Stein, Alexander. What AI Can and Can’t Do and How Psychoanalysis Can Help. The CAI Report, Issue 1 | January 2025 https://apsa.org/what-ai-can-and-cant-do/

10. Turkle, Sherry. 2024. “Who Do We Become When We Talk to Machines?” An MIT Exploration of Generative AI, March. https://doi.org/10.21428/e4baedd9.caa10d84.

11. Dennett, Daniel C. The Problem with Counterfeit People. The Atlantic, May 16, 2023. https://www.theatlantic.com/technology/archive/2023/05/problem-counterfeit-people/674075/
(accessed 1 July 2025)

12. A set of ideologies known as TESCREALism, an acronym which denotes “transhumanism, Extropianism, singularitarianism, (modern) cosmism, Rationalism, Effective Altruism, and
longtermism” per Gebru, Timnit and Torres, Émile P. The TESCREAL bundle: Eugenics and the promise of utopia through artificial general intelligence. First Monday, Volume 29, Number 4 – 1 April 2024 DOI: https://doi.org/10.5210/fm.v29i4.13636

13. Dennett, Daniel C. Op. cit.

14. Huizinga, Johan. Homo Ludens: A Study of the Play-Element in Culture, 1938

15 .Castronova, Edward. Synthetic Worlds: The Business and Culture of Online Games. University of Chicago Press, 2005

16. Stein, Alexander. Computation Is Not Mentation: Why embodiment and lived experience matter. The American Psychoanalyst, March 20, 2025 https://americanpsychoanalyst.substack.com/p/computation-is-not-mentation or https://tinyurl.com/mscdzsjw

17. Freud, S. Remembering, Repeating and Working-Through (Further Recommendations on the Technique of Psycho-Analysis II) in The Standard Edition of the Complete Psychological Works of Sigmund Freud,12():145-156, 1914

 

<><><><><><><>

* This article was originally published in the ICC FraudNet 2025 Global Annual Report: “The State of Fraud and Asset Recovery: Timeless Crimes, Modern Approaches” Dr Dominic Thomas-James, Editor.