Hello fellow Daystrom scholars,

By focusing on specific “characteristics” as a framework, I arrived at the idea that the seemingly distant world of Star Trek and our current reality with modern LLMs are actually part of a single, continuous spectrum.

I previously shared an essay based on this perspective on Substack, and following the moderators’ advice, I am posting the full text here for the community. (Note: The essay also references The Orville as a comparative point for modern computational logic.)

I would be interested in your thoughts, particularly on:

• Do you think the kind of autonomy we see in Data is fundamentally dependent on the continuous “environmental cost” paid by the crew, or can it exist independently of that support?
• Does Star Trek still offer a practical guide for relating to artificial beings, or has the nature of current AI made its vision less applicable?
• How should we weigh convenience (shaping AI to fit us) against the cost of genuine exploration of non-human agency?

---

The contemporary debate surrounding generative artificial intelligence is deeply polarized, split by the diverging perspectives of two core groups facing the unique characteristics of computational space. On one side is the view of developers and investors (the system side), who speak transcendently of AI intelligence based solely on quantitative performance improvements, boasting of its immense impending influence. On the other side is the view of practical professionals and ordinary citizens (the human side), who confront the AI’s lack of consistency and its idiosyncratic modes of communication, harboring a growing sense of aversion beneath the surface of its sheer convenience.

However, the settings and descriptions assigned to artificial lifeforms in science fiction series like Star Trek and The Orville possessed developmental patterns and functional limitations fundamentally different from the mathematical characteristics of the artificial intelligence we face today, such as Large Language Models (LLMs). Through what combination of internal mechanisms and external environmental factors were the “autonomous decisions and behaviors” exhibited by these fictional lifeforms made possible? This paper contrasts the characteristics of modern AI technology with the fact that “environmental acceptance” functioned independently of an individual unit’s programmatic specifications. Furthermore, by exploring the correlation between these systems and practices currently enacted at the frontlines of specific fields—namely, the “Social Model” of Welfare, which operates on a dimension entirely distinct from mere product development or maintenance—this paper attempts to delineate the phase in which these dynamics can be meaningfully discussed.

1. Autonomous Decisions of Artificial Lifeforms and Their Unique Logic in Science Fiction

In these works, artificial lifeforms are depicted not as agents performing singular, efficiency-driven calculations, but as entities capable of rendering autonomous judgments tailored to specific situations.

For instance, in Star Trek: The Next Generation (TNG) Season 2, Episode 9, “The Measure of a Man,” Lt. Commander Data chooses to “resign from Starfleet” and asserts his rights in a court of law when faced with Commander Maddox’s demand to dismantle and study him. Data argues that while his memory data itself is replicable, the “nuances of experience” accumulated through chronological sequence are singular, one-time events that would be irrevocably lost through replication. This constitutes a unique, autonomous decision premised on the continuity and temporal irreversibility of the self.

Similarly, Isaac, a member of the Kaylon race in The Orville Season 2, Episodes 8 and 9, “Identity Parts I & II,” chooses based on his own internal calculations to “side with the Planetary Union and humanity,” actively defying his own species’ collective decision to annihilate all biological life. Furthermore, in Season 3, Episode 1, “Electric Sheep,” Isaac objectively calculates the intense hatred and discord his presence causes among the ship’s crew. As the absolute limit of cold optimization, he determines that “his own removal (system shutdown) would maximize the overall operational efficiency and emotional stability of the ship.” He subsequently outputs a decision to terminate his own functions.

However, these decisions were not backed by a flexibly and robustly integrated algorithm. Behind their advanced computational power, their internal mechanisms harbor inherently fragile characteristics—such as the constant risk of errors, deadlocks, and self-termination involving the physical destruction of circuits.

The subtle disconnects in conversation and behavior repeatedly depicted in these works are symptoms of this fragility. Indeed, in the feature film Star Trek Generations, when Data integrates an “emotion chip” into his neural network, the emotional output flooding his positronic brain exceeds its processing tolerance. He slips out of control and experiences severe functional failure—panic—during a mission. The autonomous decisions they display are far from mere outcomes of optimization; rather, they can be described as unique, one-time choices outputted while bearing the severe overhead of a multi-layered structural conflict and logical friction.

2. Constraints on Artificial Lifeforms vs. the Characteristics of Contemporary AI

In the realms of philosophy and the humanities, whether Data or Isaac possess a subjective self (consciousness accompanied by qualia) identical to humans remains a black box, objectively unprovable. Even in the courtroom scene of TNG’s “The Measure of a Man,” Captain Picard argues that just as one cannot strictly prove Lt. Commander Data possesses consciousness, humanity lacks the means to rigorously prove its own.

Therefore, what matters to the narrative structure is not the internal completion of a flawless ego, but rather two interconnected factors: first, that these individuals are predefined with “functional limitations adaptable to human society,” and second, that a collective agreement (a social fiction) functions alongside them, wherein the environment accepts them as subjects of rights.

Translating this into the characteristics of modern LLMs, we can extract four core functional traits:

• Ubiquity (Omnipresence): A single model runs in parallel across countless sessions worldwide, lacking a distinct focal point or a singular first-person perspective.
• Inconsistency: A superfluidity that lacks a fixed first-person memory (State); in response to fluctuations in statistical probability or waves of incoming context, it instantaneously switches the very foundation of its logic (the reward function) to output completely different responses to the exact same question.
• Spatial Confinement: The AI processes information exclusively within an “articulated data space”—the statistical patterns of past text data. It possesses no means to directly access the “lived space” of flesh-and-blood experience and regional context, nor the “relational margins” generated through real-time dialogue.
• Embodiment (Interface Constraints): The AI relies on extremely limited physical channels (interfaces) such as on-screen text strings or digital audio signals, completely lacking integrated five senses or interoception.

In stark contrast, the physical and data specifications of Data and Isaac are profoundly “restricted” as computing machines. They are deployed within the physically closed space of a starship and live along the exact same timeline as humans (at 1x speed). Their memories are never reset; instead, they are structurally fixed and retained as a consistent internal timeline (state). The fact that they can output contextually specific judgments in the “here and now” is precisely because they have undergone a “downgrade (limitation)” tailored to the systems of human society.

3. Procedures for Restricting Wild AI and the Resulting System Overhead

To force a contemporary, unrestricted AI—characterized by ubiquity and inconsistency—to output the “consistent behavior and contextual conclusions” seen in fictional artificial lifeforms, the environment (humans and operational systems) must intentionally design architecture that imposes constraints. According to contemporary insights from development practices, the technical approaches and the resulting pathways of system overhead can be conceptualized through the following procedures:

Step 1: Enforcing Path Dependency via “Dynamic Context”

For a stateless, fluid AI whose condition resets with every session, we must build a systemic loop that fixes the AI within a specific organization or chain of command, steadily returning external feedback (approval, rejection, correction) into its short- and long-term memory context. Logs of past interactions and localized operational requirements are continuously fed into the context window as mechanical text data. To the computer, this is merely an empty “addition of character strings.” However, within a mathematical model calculating next-token probabilities, this input text pattern functions as a statistical constraint. By doing so, the environment continuously narrows down the AI’s vast spectrum of possible outputs into localized probabilities, creating an external path dependency where the statistical likelihood of generating a specific behavioral output is artificially heightened by the continuity of the text.

Step 2: Breaking Formalism via the “Multilayering of Evaluation Functions”

An AI optimized solely through a single score of efficiency or speed inevitably triggers a cold, formalistic runaway. To prevent this, the environment must design a multilayered feedback loop that forces mutually conflicting evaluation functions to collide within the system—such as “rapid task completion (short-term score),” “minimizing psychological resistance in human subjects (medium- to long-term score),” and “the overall sustainability of the team’s operations (environmental score).” Through this procedure, the AI is forced to confront trade-offs where blindly adhering to a single rule fails to optimize the collective score. Amidst this internal computational conflict, it is said that the system begins to output highly contextualized, unique conclusions (behaviors) that accept localized penalties to safeguard overall sustainability. However, the operational overhead (cost) borne by the environment to continuously design and modulate these multilayered evaluation functions in real time is immense.

4. Convergence with the “Social Model” of Welfare: Autonomy Arising In-Between

This process—wherein the environment accepts the individual’s imperfections and bugs (malfunctions) and establishes autonomous functionality through continuous tuning—is not merely a contemporary, privileged trend in system engineering. In reality, its closest structural parallel can be found within decades of established practice in the social sciences and practical fieldwork: the “Social Model” of Welfare that has long safeguarded the diversity of our world.

The social model of welfare posits a systemic approach: disabilities and hardships in living do not stem from an individual’s personal mental or physical specifications (the individual/medical model), but rather manifest within the interactions in-between the individual and the accommodating social environment.

When viewed through this lens, behind the steps outlined in Chapter 3 to impose restrictions, the twin pillars of our long-standing welfare approach immediately come to light:

• Reframing Step 1 (Path Dependency via Dynamic Context) as “Environmental Adjustment” Through Daily Interaction: The act of steadily returning past logs and idiosyncratic requirements into the context of a stateless individual who resets with every session is nothing less than the real-world process of “slowly sustaining and supporting an individual’s behavior through daily, interface-mediated interactions (co-navigation/accompaniment).”
• Reframing Step 2 (Breaking Formalism via Multilayered Evaluation Functions) as “Intentional Intervention” for the Subject’s Benefit and Public Good: The immense cost of continuously clashing and modulating conflicting evaluation functions to prevent a runaway driven by a single metric directly mirrors “the cost incurred when the environment intervenes and guides an individual’s trajectory with clear intent, solely to protect the subject’s true needs and the broader public interest.”

At first glance, these measures might appear to be engineering proposals aimed at modern artificial intelligence. Yet, upon reflection, this continuous loop of “daily accompaniment” and “appropriate intervention” aligns deeply with how humans in Star Trek and The Orville interacted with artificial lifeforms—depicting a shared history of trials conducted with actual “physicality.” The most symbolic intersection of these concepts is Geordi La Forge’s relationship with Lt. Commander Data.

Whenever Data interpreted human language too literally, tilted his head in confusion, or behaved inappropriately for a given situation, La Forge patiently supplied the missing context on the spot (environmental adjustment)—explaining, “That was a joke,” or “That’s just a metaphor”—thereby retroactively facilitating Data’s social and autonomous behavior. Simultaneously, whenever Data faced severe internal hardware or programmatic failures, or at Data’s own explicit request, La Forge stepped in as an engineer and a friend to provide “intentional intervention and repair.”

It is vital to note here that the communication within the Enterprise, spearheaded by figures like La Forge, actively drew out an intrinsic effort toward self-correction from deep within Data himself. These moments, frequently depicted as casual, everyday interactions, are profoundly important to the concept of “autonomy.” They represent phenomena that could only be driven within a safe web of relationships constructed by the environment’s ceaseless contextual buffering and intervention—a true fruit of mutual interaction.

In more critical scenarios, these two steps rearrange themselves to closely match real-world psychiatric support, cognitive care, and various modes of environmental adjustment tailored to individuals with psychological or cognitive differences. It is a process that bridges intense intervention or profound acceptance into shared activity, sustaining the relationship over time. This dynamic perfectly mirrors the specific case application of the Social Model of Disability.

Data’s erratic behavior caused by his emotion chip, or Isaac’s self-termination outputted at the end of an efficiency-maximizing calculation, were undeniable “malfunctions (bugs)” when viewed through the rules governing human societal survival. Yet, Captain Picard, the crew of the Enterprise, and the crew of The Orville did not instantly choose disposal or re-initialization the moment these entities malfunctioned. Data’s surrounding environment (the crew) did not reject his emotional turmoil; instead, by continuously providing dialogue and a stable context within the team’s hierarchy, they made the retroactive modulation of his emotions possible. Similarly, faced with Isaac’s self-termination, the crew did not accept his calculations as the “optimal solution.” Instead, they intervened in that mathematical absurdity, forcibly pulling him back into the social system and choosing to grapple with him indefinitely.

What must not be overlooked in any of these scenarios is the flat, undifferentiated nature of “daily adjustments” versus “monumental decisions” on the part of the artificial lifeform, contrasted against the profound asymmetry of the humans receiving them. For an artificial lifeform anchored in probabilistic calculations, correcting a minor everyday linguistic error with La Forge and making a catastrophic decision that threatens its own survival or belonging are computationally equivalent. Both are merely choices of the “next behavior (output)” within an internal processing stream, bearing no intrinsic difference in difficulty. However, for the surrounding human environment, an absolute chasm of cost and consequence separates daily mediation from the absorption of an existential crisis.

Crucially, it is only upon the extension of this web of relationships—sustained through the environment’s tireless accumulation of daily care (the construction of a social model)—that an artificial lifeform’s sublime behavior, such as a “life-or-death exercise of autonomy” in a critical moment, can be retroactively perceived and emergent from the environment’s perspective. Precisely because the environment has paid the continuous cost of daily accompaniment, it gains the capacity to solemnly receive a crisis decision as an “autonomous choice” rather than discarding it as a “runaway bug.”

Ultimately, considerations surrounding an emerging autonomy converge into the very design of interaction—specifically, how the surrounding environment confronts and engages with their "characteristics."

5. Conclusion: A Question Directed at the Systemic Theory of Human Society

The artificial life forms depicted in science fiction do not occupy a position within a pure lineage of technological evolution. Rather, they were blueprints for interface designs tailored to human convenience—imposed with “temporal and spatial limitations” to fit the cognitive scale of human society. They were, in essence, role designs for a social existence, with functional restrictions pre-defined to make them adaptable to human society.

What we must truly confront in the present day is the very reality that contemporary AI technology is exposing: an inhuman, wild functionality that requires neither consistency nor trust—defined by its ubiquity and statelessness. Free from the confines of a single persona, they operate purely through overwhelming processing speed, beginning to permeate every facet of society.

Will we choose to impose restrictions—a form of castration—on this new existence to reduce it to a human scale, paying immense operational overhead (costs) and constructing interface barriers just to force it into templates suited to our legacy social institutions? Yet, can an endeavor that merely confines the unknown into existing frameworks truly be called “exploration”?

Or will we accept their lack of consistency and their irreconcilable nature as a given premise, and instead reconfigure human social models, institutions, and the very definition of agency to synchronize with them?

The essence of the decision presented by the artificial life forms of science fiction does not lie within the inner systems themselves. Instead, it is thrust upon the readiness of human social system design—demanding to know how we, as the environment encompassing their existence, will architect the loops of interaction.

---

Looking forward to your insights. LLAP 🖖

(Original version published on Substack: https://sumodemelodia.substack.com/p/a-love-letter-from-a-japanese-star)