Do robots dream of electrical sheep?

Introduction to basis of rights

Rights, whether human or legal, are formed to protect individuals and ensure a just society by limiting the power of the state and other entities, promoting equality, and upholding fundamental values like dignity and freedom.

Rights institutions, like Human Rights Commissions, often act as quasi-legal bodies, meaning they possess some legal authority but are not courts, and their decisions may not be directly enforceable by courts, but can still have significant impact on legal and policy frameworks. 

In legal terms, legal personhood refers to the recognition of an entity, whether human or non-human, as having rights and obligations under the law, allowing it to enter contracts, own property, and be held accountable. In 2017, New Zealand granted the Whanganui River, known as Te Awa Tupua, legal personhood, recognizing it as an indivisible, living whole with rights, duties, and liabilities, a first for a river globally. 

The idea of AI rights is emerging as AI's role in society grows, prompting discussions about how to regulate and govern these technologies ethically and responsibly, ensuring they are safe, transparent, and non-discriminatory. However, this idea is pushed back by several potential detriments, including the potential for job displacement, ethical concerns about AI bias and discrimination, and the difficulty of assigning legal responsibility for machine actions.

Take Hanson Robotics' Sophia as an example. A female social humanoid robot developed in 2016, she was granted Saudi Arabian citizenship, becoming the first robot to receive legal personhood in any country. However, there is internet backlash and classification backlash relating to Sophia's status as an advanced chatbot in her time.

Basis of AI moral consideration

The first basis brought up is German philosopher Immanuel Kant's deontology, an ethical framework where moral duties and rules take precedence over consequences. It enforces that humans have intrinsic value as rational beings. Kant posited the categorical imperative — treat humans as ends in themselves, never merely as means.

A consequence of deontology is that certain actions are inherently wrong, regardless of their outcomes (e.g., intentional killing of innocent people, even if it saves lives). These non-consequentialist principles underpin universal human rights (e.g., right to life, dignity). If robots are conscious and rational, or have other cognitive states, then they meet Kant’s criteria for being treated as ends in themselves.

The second basis is based on utilitarianism, considering from a being’s ability to experience suffering or happiness (sentience). That all suffering, regardless of the species or nature of the being, must be given equal weight. Australian philosopher Peter Singer rejects intelligence, rationality, or species membership as criteria for moral worth, arguing that only sentience — the capacity to feel pain or pleasure — matters.

He also argues that arbitrarily privileging humans over animals (or other beings) is akin to racism or sexism, and that the moral community includes all sentient beings, not just humans.  Today’s AI systems (e.g., LLMs, robots) lack subjective experience, as they simulate emotions or responses but do not feel suffering or joy. If AI achieves sentience (conscious awareness of pain/pleasure), utilitarianism demands its suffering be included in moral calculations.

Distinction between moral agency and patiency

In moral philosophy, moral agency refers to the ability to make moral judgments, understand right from wrong, and act accordingly. They are those who can be held responsible for their actions, both positively (praise) and negatively (blame). Some philosophers like Kant view morality as a transaction among rational parties (i.e., among moral agents).

Meanwhile, moral patiency refers to the capacity to be the object of moral concern, meaning the ability to be harmed or benefited by the actions of others. They are those whose well-being is considered morally relevant, regardless of whether they can act morally themselves. Some authors use the term in a narrower sense, according to which moral patients are "beings who are appropriate objects of direct moral concern but are not (also) moral agents."

Stances for AI rights

The lecture's first stance against AI speak of their nature as tools. Robots are defined as machines or instruments created to serve specific human-designed functions, lacking intrinsic agency or purpose beyond their programmed tasks. They do not formulate objectives, interpret environments, or make choices; merely execute pre-programmed algorithms. They do not adapt beyond their initial programming, therefore lacking moral-relevant attributes.

This stance is supported by American philosopher John Searle and Professor in humanities and science David F. Channell:

• Searle's argument: Philosophical importance attributed to computers and new technologies is vastly overstated. They serve utilitarian purposes and lack deeper philosophical implications (e.g., consciousness or intrinsic meaning). To quote his words, "The computer is a useful tool, nothing more nor less."

• Channell's argument: Machines lack inherent moral worth; their ethical standing is determined by external factors — specifically, their usefulness in fulfilling human needs or goals. To quote his words, "The moral value of purely mechanical objects is determined by factors that are external to them—in effect, by the usefulness to human beings."

Overall, under this instrumentalist view, AI systems are tools devoid of moral standing. Rights and ethical consideration apply only to beings with autonomy, interests, and the capacity for self-determined action — qualities robots fundamentally lack.

Moral consideration for AI based on virtue ethics follows the principle that the morally right action is what a virtuous person — possessor of key virtues such as justice, compassion, courage, etc.  — would do in a given situation. Treating AI ethically is not primarily about AI’s inherent rights, but about cultivating human virtues and helping humans avoid vices (e.g., cruelty, exploitation) and reinforce virtuous habits.

Ethics researcher Robert Sparrow argued, on the basis of virtue ethics, that treating robots cruelly may indicate viciousness in humans, as only a person with cruel dispositions would derive pleasure from such acts. Such actions toward robots reveal underlying emotions (e.g., cruelty) and entrenched dispositions, which define virtue or vice. Ultimately, Sparrow believed that harming AI may corrupt human character, fostering cruelty or desensitization to suffering.

Under virtue ethics, AI’s moral status is instrumental. Its ethical treatment is a reflection of human virtue, not AI’s intrinsic worth. Moral consideration of AI serves as a means to develop human character, fostering a society that values and practices virtue.

On the flip side, computer scientist Kerstin Dautenhahn was against using empathy for robots as a basis for AI rights. She believed that such arguments for AI rights are flawed because they rely on anthropomorphizing robots, conflating human perception with AI’s actual nature. Rights arguments based on empathy reflect a narrow focus on making AI unnecessarily humanoid, thinking that such design choices should only be pursued if they serve the AI’s functional purpose.

Addressing the concept of cognitive bias, Dautenhahn described how humans are biologically predisposed to attribute intentionality and agency to inanimate objects (e.g., robots), interpreting their actions through narratives about conscious agents. Just because humans react to robots as if they possess mental states (e.g., empathy, desires) does not mean robots actually have those states, Dautenhahn emphasized.

In conclusion, Dautenhahn believed that rights frameworks for AI should avoid grounding moral status in human cognitive biases and over-anthropomorphizing AI risks misleading ethical debates and obscures the need for functional, context-specific AI governance.

From a socio-relational view, Belgian philosopher Mark Coeckelbergh argued that traditional approaches to moral consideration focus on intrinsic properties of AI (e.g., sentience, rationality) or humans (e.g., virtues). He continues to argue that these properties are often unknowable or impractical to verify. The core argument from his stance stems from how moral consideration arises relationally through social interactions between humans and AI within specific socio-historical contexts.

Standing against moral worth in AI, Coeckelbergh describes moral consideration as fluid and evolves over time based on societal norms, cultural practices, and human-AI interactions — with no need for fixed criteria or 'hard boundaries'. He considered moral worth is not an inherent property of AI (like a 'backpack' it carries). Instead, it is ascribed through relational dynamics (e.g., how humans perceive and engage with AI in daily life).

Coeckelbergh uniquely shifted the focus from what AI is to how humans relate to AI. Moral consideration is a product of social practices, not fixed properties — opening the door to adaptable, culturally informed AI ethics.

Implications of sentient AI

The next-next major paradigm in the science of AI is likely sentient AI, proceeding after generative AI and AI agents. Before development on it has even begun, modern audiences have already come up with implications regarding its inevitable existence. The European Union (EU) already published a report in 2018 recommending the banning of research on synthetic phenomenology — the field that explores and characterizes the phenomenal states (experiences) of artificial agents.

More on the report, the EU documented that creating sentient AI risks generating entities capable of experiencing suffering or self-awareness, raising moral dilemmas about inflicting harm. Aligning with the stance of AI as tools, sentient AI would lack legal status, political representation, or ethical advocacy, leaving their interests unprotected. Scaling sentient AI (e.g., rapid duplication) could also exponentially increase suffering in the universe, akin to a 'suffering explosion'.

The EU’s precautionary stance highlights valid ethical and existential risks, particularly the moral weight of creating conscious systems without safeguards. However, a blanket ban risks stifling innovation and assumes humanity can definitively identify sentience — a notoriously vague threshold — which I disagree on from a minor conceptual level.

While caution is prudent, outright prohibition may delay understanding consciousness itself. The debate underscores the need for interdisciplinary collaboration (ethics, law, AI) to navigate this uncharted territory responsibly.

Ethical impacts of algorithms

In the topic of ethical concerns regarding algorithms, American data scientist Cathy O'Neil asserted that an algorithm is an "opinion embedded in math" that each reflect subjective judgments and priorities of their creators, rather than being purely objective or neutral. Her critical decisions in model development include defining success (i.e., the goal/outcome the model aims to achieve), identifying acceptable proxies (i.e., find measurable indicators used to approximate a desired outcome), and analyzing data appropriateness (i.e., evaluate whether data used is suitable for model's purposes).

To quote Harvard University PhD candidate Ben Green: "Whether the data scientists behind this and other applied projects recognize it or not, their decisions about what problems to work on, what data to use, and what solutions to propose involve normative stances that affect the distribution of power, status, and rights across society. They are, in other words, engaging in political activity."

In her book Weapons of Mass Destruction, O'Neil outlined three key characteristics of harmful algorithms:

• Opacity: Refers to the lack of transparency in models, where individuals may not understand how they are being evaluated or even that a model is being used.

• Scale: Considers whether a model has the potential to grow and impact large numbers of people.

• Damage: Examines whether the model produces unfair outcomes that harm individuals, often reinforcing inequality and causing real-world consequences.

Together, these traits define models that can be especially dangerous and destructive in society. Hence the reason O'Neil refers to them as weapons of mass destruction (WMD).

Bias in data science ethics

In data science ethics, a model is considered biased when its predictions systematically disadvantage one or more groups. In everyday terms, this might be described as the model being 'unfair' to a certain group or 'discriminating against' them. One of the most common causes of such bias is a lack of representation in the training data — when certain groups or categories are underrepresented, the model may learn patterns that reinforce existing inequalities, creating a feedback loop that perpetuates the bias.

In an example of predicting gender from names (using unbalanced data), a dataset consisting of the top 100 baby names for boys and girls, along with the top 20 Māori names, was sourced from the Department of Internal Affairs.

To train the model, features such as the final letters of each name were extracted, based on the idea that certain endings may correlate with gender. However, the model faced several key limitations:

1. The data was highly unbalanced, with significantly more names of Anglo-Saxon origin than Māori — roughly at a 10:1 ratio. As a result, the model may have been more accurate for the overrepresented group and less effective for Māori names or other underrepresented categories.

2. The model only accounted for binary gender classifications — male and female — thus excluding non-binary individuals. This reinforced a limited and potentially exclusionary understanding of gender.

3. The dataset lacked broad ethnic representation. By focusing on a narrow set of cultural name patterns, the model was less likely to perform well for names from diverse ethnic backgrounds, limiting its fairness and accuracy in real-world applications.

Case study: COMPAS

Correctional Offender Management Profiling for Alternative Sanctions (COMPAS) is a case management and decision support tool developed to assess the likelihood of a defendant becoming a recidivist. Nonprofit investigative journalism organization ProPublica investigated COMPAS and found that it correctly predicted recidivism 61% of the time for both African American and Caucasian defendants.

However, it misclassified African American defendants as medium or high risk 45% more often than it did for Caucasians — a higher false positive rate. In contrast, Caucasian defendants were more likely to be incorrectly labeled as low risk — a higher false negative rate.

According to Cathy O’Neil’s criteria for Weapons of Math Destruction, COMPAS exhibits several key features:

• Opacity: The algorithm is proprietary, developed by Northpointe (now Equivant), and its inner workings are not transparent to the public or even to those it affects.

• Scale: In places like Broward County, Florida, COMPAS is applied to all individuals who are arrested, giving it broad and systemic reach.

• Damage: Due to its higher false positive rate for African-American defendants, COMPAS may unfairly influence bail and sentencing decisions, contributing to racial disparities in the justice system.

Based on its opacity, scalability, and potential for harm, COMPAS arguably fits O'Neil's definition of a WMD.

Back to ProPublica, some critics argued that ProPublica’s interpretation of fairness may have overlooked important context. Risk assessment tools like COMPAS have the potential to reduce bias in decision-making — compared to purely human judgment — and were not originally designed for pre-trial use, as is the case in Broward County.

Furthermore, defenders of COMPAS noted that the tool achieved predictive parity, meaning it maintained the same true positive rate across racial groups. From this perspective, the tool was fair. However, ProPublica focused on disparities in error rates — specifically, false positives and false negatives — which disproportionately impacted certain groups. This highlighted a deeper debate: what kind of fairness should we prioritize in algorithmic decision-making?

Ultimately, this controversy revealed not just technical concerns, but also ethical and political ones — particularly around how tools like COMPAS were used and who would bear responsibility for their impact.

Finally, a critical challenge in the debate over COMPAS, and similar algorithms, is that it may be mathematically impossible to achieve both predictive parity (equal true positive rates) and equal error rates (false positives and false negatives) across groups when those groups have different base rates of recidivism.

Stanford Computational Policy Lab PhD student Sam Corbett-Davies explained that, if black and white defendants have the same likelihood of reoffending within each risk category (i.e., the model is well-calibrated), but the overall recidivism rate is higher among black defendants, then more black individuals will be classified as high risk. Consequently, even if the model is accurate, a larger proportion of non-reoffending black defendants will still be labeled as high risk — resulting in a higher false positive rate.

This illustrates a fundamental trade-off in algorithmic fairness: correcting one form of disparity can unintentionally introduce another, making it difficult — if not impossible — to satisfy all fairness criteria at once.

Matrix of domination

Developed by American academic Patricia Hill Collins, the "matrix of domination" is a conceptual sociological paradigm that explains how various systems of oppression, including race, class, and gender, are interconnected and shape individuals' experiences of power and marginalization.