Recent Insights on AI in Research

In addressing these concerns, Messeri and Crockett, in their March 7th article for Nature Magazine — "Artificial intelligence and illusions of understanding in scientific research," caution that while AI can enhance productivity and objectivity, it might also exploit our cognitive biases, leading to misconceptions about our understanding. This could hinder innovation and increase the likelihood of errors in scientific work.

Messeri and Crockett delineate four distinct perceptions of AI within the scientific arena: (1) Oracle; (2) Surrogate; (3) Quant; (4) Arbiter.

The Illusion of Understanding

The graphic accompanying their argument visually demonstrates how AI can limit scientific comprehension. The "Illusion of Explanatory Depth" underscores the danger of AI presenting seemingly objective answers without clarifying the underlying processes, thereby obscuring true understanding. Additionally, the "Illusion of Exploratory Breadth" reveals how the necessity for AI-compatible data could restrict the very questions scientists pursue, biasing research toward quantifiable phenomena. These limitations, coupled with the "Illusion of Objectivity" — the misguided belief that AI inherently yields unbiased results — reinforce the 'ontological tether' linking researchers to AI.

Including AI in scientific communities poses a significant threat to the entire scientific enterprise. The allure of AI is rooted in its perceived capacity to transcend human limitations in the quest for objectivity, a quality foundational to scientific trust. However, integrating AI into this trust framework could disrupt the delicate balance within knowledge communities.

Scientists often envision AI as 'superhuman' collaborators capable of overcoming human limitations, especially in terms of objectivity and quantitative analysis. This emphasis on AI's ability to deliver straightforward, quantifiable explanations resonates with our cognitive tendencies, making these tools seem exceptionally reliable while masking the potential for illusory understanding. Such illusions can undermine the nuanced, qualitative assessments critical to scientific rigor, as AI's tendency toward reductive, quantitative outputs threatens to replace genuine comprehension of complex phenomena.

Video Insert: Exploring AI's Myths, Risks, and Opportunities

The Dangers of Scientific Monoculture

The dangers of monoculture, as tragically illustrated by the Dust Bowl in the 1930s, extend far beyond agriculture. A scientific community overly reliant on a singular approach or technology risks similar devastation.

In agriculture, monoculture refers to a farming practice where only one crop species is cultivated at a time. While this method is efficient, we know how it devastated the American Plains. Single-crop farming depletes essential soil nutrients without the regenerative benefits of crop rotation, rendering them vulnerable to pests and diseases. Analogously, a scientific community that overly emphasizes a single approach, such as the widespread use of AI, risks neglecting alternative methodologies and becoming blind to potential pitfalls.

AI Tools and the Growth of Monoculture

AI tools can foster scientific monocultures by: (1) favoring questions and methods best suited for AI, thereby narrowing the scope of inquiry, and (2) prioritizing certain perspectives that may silence marginalized voices. For instance, research heavily dependent on AI-generated data is likely to overlook areas where data is less quantifiable or where marginalized communities are underrepresented in the datasets utilized. This constrained focus not only increases the risk of errors and biases in scientific findings but also perpetuates a misleading sense of understanding.

This relentless pursuit of an idealized AI-driven objectivity reinforces the ontological tether. When AI, seen as oracles and arbiters, is viewed as superior to human-led research due to its alleged ability to transcend human subjectivity, the quest for detached, unbiased evaluators in science becomes perilous. It reduces scientific inquiry to a purely 'objective' process and naively assumes that removing all traces of subjectivity invariably leads to superior science.

The pursuit of objectivity through AI neglects the vital role that human diversity plays in scientific advancement. Cognitive diversity ensures a broader range of inquiries, problem-solving approaches, and fresh insights — all crucial for scientific progress. For example, a physician's direct experience with patients may inspire a line of questioning that a data scientist focused solely on statistical models might overlook, let alone an AI trained on abstracted datasets.

Is Another Science Possible?

In her book 'Another Science is Possible: A Manifesto for Slow Science,' Isabelle Stengers presents compelling arguments that align closely with the concerns raised about AI's integration into scientific processes. Stengers critiques the trend towards "fast science," which prioritizes efficiency, productivity, and quantifiable outcomes. Her critique extends to a broader examination of how contemporary science, influenced by neoliberal ideologies and market forces, increasingly favors research agendas that promise immediate economic returns over more speculative or foundational inquiries.

In the book's introduction, Stengers argues that scientists are becoming increasingly isolated from society, creating a 'systematic distancing' where scientific institutions, the state, and private industry converge. This isolation results in a vacuum filled with individuals she calls 'connoisseurs,' who can grasp scientific work and its societal implications. Stengers advocates for a science that produces not only specialists but also connoisseurs. She draws parallels with fields like music, sports, or software, where creators must consider how their work will be perceived and utilized by the public, rather than simply presenting it as incontrovertible fact.

Stengers' framework emphasizes a scientist's essential duty to engage with an informed public. This resonates with the significance of active public participation and accountability in the conceptions of responsibility put forth by Emerson and Dewey. However, the incorporation of AI tools threatens to create the opposite dynamic, deepening the 'systematic distancing' she warns against.

As AI assumes authoritative roles as Oracles, Surrogates, Quants, and Arbiters, it exacerbates the divide between scientists and the public. The risk lies in a public that becomes increasingly unable to critically evaluate research co-produced by AI and researchers. This renders the public as passive recipients of scientific claims about 'progress,' further alienating them from influencing what constitutes meaningful scientific endeavors. Ultimately, this disengagement erodes the societal foundations upon which science relies for support, legitimacy, and even inspiration for future breakthroughs.

Stengers initiated a three-year experiment at her university where students analyzed past scientific outcomes. Through this analysis, they uncovered subtle ways scientists sometimes dismiss as 'non-scientific or ideological' factors that others deem important. The students recognized that scientific situations are characterized by uncertainty, intertwined with a web of facts and values — decisions made by scientists to intentionally overlook elements that fall outside predefined parameters. This selectivity is now being incorporated into the logic of AI models. As AI collaborates with scientists, there’s a danger of exacerbating these biases exponentially.

The Myth of the Ideal Scientist

The ideal of the scientist possessing the 'right stuff' — a disposition aligned with the 'spirit of science' and devoid of supposedly unscientific traits like 'emotion' and 'whimsy' — is a long-standing concern. Robert Boyle, often hailed as the founder of Modern Chemistry, advocated for an ethos of 'spiritual chastity' centered on modesty and disciplined reasoning. This ideal of the detached, rational scientist still subtly influences our expectations. Could this explain the misplaced faith in AI as a means to achieve objectivity? If this connection resonates, my piece "When Did We Lose The Right To Be Imperfect?" examines this shift in our culture more broadly, beyond just the sciences.

In this mindset, the 'big questions' regarding the purpose of our research and its implications for what we deem a good life are quickly dismissed as irrelevant. This results in an impoverished ontology, where one form of faith replaces another. Now, embodying the ethos of a scientist necessitates unwavering belief that what scientific inquiries deem irrelevant truly do not matter. It is a faith that, as Stengers states, "defines itself against doubt."

The Danger of Oversimplification

Bracketing specific aspects of reality that do not align with a particular research question — temporarily setting aside certain factors to focus on a problem — is a crucial part of scientific inquiry. However, this definition of 'proper' scientific objectivity represents a normative claim. It is a choice cloaked in the illusion of pure, unbiased truth, detached from our human emotions and subjectivities. As Stengers articulates, this approach necessitates the 'refusal of the Big Question that might lure opinions, which is "always wrong." AI poses a risk of amplifying this danger. It can create the illusion that a computationally perfect, bracketed answer represents the entire reality, obscuring the significance of broader questions and the necessity for diverse viewpoints.

The laboratory, whether a physical space or a conceptual one, is now interwoven with modern productive forces. This relentless emphasis on efficiency, speed, and competitive advantage creates fertile ground for unchecked AI integration. AI, promising further optimization, will exacerbate a scientific landscape driven by instrumental knowledge production rather than genuine inquiry and deeper understanding.

Any element that could distance the researcher from this instrumental mindset is deemed irrelevant, a 'waste of time,' or a source of doubt. Doubt lingers like a fog, a field of anxiety, where the slightest contemplation, a "what if," could disrupt this relentless pursuit of quantifiable outcomes. This disruption introduces a subtle yet potentially destructive wobble in the sphere of instrumental rationality. Is embracing the weird a risk, or is this wobble where true wisdom resides?

Video Insert: The Role of Social Sciences in AI Research

Embracing the Weird

This brings me back to the crucial role of the weird in science and our existence as modern individuals. The hosts of Weird Studies describe the weird as follows:

"The Weird is that which resists any settled explanation or frame of reference. It encompasses the bulging file labeled 'other/misc.' in our mental cabinets, filled with supernatural entities, magical synchronicities, and occult practices. It also surfaces when a piece of art disrupts our habitual perceptions, causing ordinary elements to become uncanny."

The weird can be seen as anything that lies beyond the threshold of what we can easily accept from our world and what we cannot.

I'm not advocating for the abolition of science; quite the opposite! Just observe the incredible inventions surrounding us. We are no longer subject to the unpredictable whims of a toothache that escalates into a fatal abscess. We have established a society capable of meeting the basic needs of every human being, a task that occupied the majority of our not-so-distant ancestors.

Yet, within this world, a sense of something being off lingers. Surrounded by all these innovations and explanations for what was once mysterious, there exists a profound emptiness, a glaring neon sign flashing incessantly, 'So What?'

Miles Davis' 'So What' is significant to our discussion. Before this song and album, jazz had settled into a routine, adhering to a specific form with some stylistic variations. Miles himself was a pioneer of 'the way,' characterized by a set of ii-V-I chord changes and various alterations of that basic structure, creating movement for melodies and changes for soloists to 'blow' over, generating tension and release. Then Miles decided to linger on one chord for 16 measures. No changes, just D-Dorian for 16 measures, then Eb-Dorian for another 16, and back again. Quite unconventional.

To suggest that this song and album revolutionized music would be an understatement. Kind of Blue is the best-selling jazz album in music history. But what transpired here? Beyond the initial shock of its deceptive simplicity, it provided a depth of expression that resonated profoundly with listeners. This embodies a relationship between the public and experts, where 'connoisseurs' engage with the specialists in a socially meaningful symbiosis that transforms the weird into reality. Just as Stengers advocates in her book Another Science is Possible, science requires a similar relationship with the public.

Rather than solely emphasizing efficiency, scientists should be encouraged to embrace the weird — the uncertain, the doubtful, and the lingering questions at the periphery, allowing for a bit of wobble. These are the spaces where transformative discoveries lie. To cultivate a healthy scientific ethos, we must bridge the widening gap between scientists and the public. A flourishing scientific ethos necessitates not only scientists pushing boundaries but also an engaged public — connoisseurs who can actively participate in evaluating and discussing the uncertainties and outcomes of scientific endeavors.

As we weave AI tools into the fabric of scientific inquiry, let us harness their capabilities to illuminate new patterns and possibilities. However, we must not forget the ultimate force driving scientific progress — the subjective 'why' in all its manifest weird forms.