In recent years, we have witnessed experiments bordering on science fiction in the chatbot training field. Still, it is essential to understand their mechanisms and limits to avoid being fooled when new technologies imitate people. How important is language in defining ourselves? What is the limit not to be crossed in creating artificial intelligence modeled on real individuals?
Questions that are centuries old and so new that they sound like science fiction intersect in addressing the questions posed by artificial intelligence technologies: We are no longer faced with objects distinguishable from living beings but with something that tends to increasingly resemble us, often designed precisely to deceive us about its very synthetic nature.
One of the fascinating aspects of IT law and the protection of personal data derives from the fact that, in the digitalized world in which we all – willingly or unwillingly – are immersed, everyone leaves around them a sort of trail of information which, combined, they can help recreate a more or less in-depth representation of ourselves.
All with disturbing consequences, to say the least, which go far beyond the simple ones of chatbots programmed for the various online scams and which can often be easily “unmasked.” For some years now, artificial intelligence has begun to be created capable of imitating (and, often, replacing) the conversation with a well-defined individual.
The idea of chatting with a deceased boyfriend (or friend) is very reminiscent of an episode of Black Mirror (“Be Right Back”), the popular sci-fi/dystopian series by Netflix, which, after all, is not such sci-fi: this episode is one of the ideas that a few years ago led Eugenia Kuyda, co-founder of Luka, an artificial intelligence startup, to program a chatbot using the messages exchanged with Roman Mazurenko, who died in an accident, to erect a sort of “digital monument” to his friend and allow people who knew him to continue conversing with a very faithful replica of him.
Kuyda’s initiative was mostly welcomed (even by the deceased’s mother), so much so that the entrepreneur then launched herself into a new project: Replika, a bot that, by chatting with a person, learns to imitate them and, slowly slowly, it becomes an avatar of the person himself, a sort of digital assistant in his image and likeness.
Today, Replika has been downloaded by millions of people and is used to “improve mental health” (at least, according to what the email that arrives after registering on the app says); according to its description found on Google Play, It is clear, however, that these technological developments require more in-depth reflections beyond the simple emotional reactions to the possibility of grieving the death of a person by exchanging a last conversation with software that imitates them. Language and personality.
We find ourselves in a highly delicate field where ethical and moral considerations encounter legal and existential problems regarding understanding human identity and how it expresses itself in the digital world. Finally, it is necessary to remember that the moment we rely on a person who existed and is incapable, by force of circumstances, of giving consent to the creation of a chatbot modeled in his image, we are invading a private sphere.
Creating A Chatbot And Imitating Human Speech
Let’s start from the basics: In recent years, science has made enormous progress in programming artificial intelligence systems using machine learning mechanisms. The field that interests us for our discussion is, in particular, the training of chatbots, i.e., computer programs aimed at having conversations with human beings credibly and efficiently.
Chatbots can be used in various sectors, and it is not sure that they are malicious or that their fictitious nature is always hidden; on the contrary, examples of chatbots can be found in consumer assistance services as the first intervention to which, in the event of customer dissatisfaction, a human operator follows, while even those who are more sophisticated in imitating people in many cases immediately make it clear that they are not real individuals, but are no less efficient for this (Kuyda’s Roman himself fell into this category)
The first step in “training” a bot to talk to people is ensuring the program understands and responds to them using the same language. “Natural Language Processing” ( NLP) is the discipline that deals with the computerized processing of natural language, which for computers is particularly complex because it also contains innuendos and ambiguities; often, in fact, to disprove the simplest chatbots, you just need to use sarcasm.
Viola Bachini and Maurizio Tesconi, in their book “Fake People – Stories of Social Bots and Digital Liars,” explain how programmers today often combine two different ways to teach a machine to speak: on the one hand, they provide the algorithm with a series of linguistic rules (more traditional method); on the other hand, at the same time, they use machine learning techniques, feeding the algorithm an enormous amount of examples so that it learns by itself how to respond to stimuli.
Thanks to the computational power of today’s computers, it has become possible to use neural network systems to create deep connections – deep learning. However, if we aim to train a chatbot to imitate a person, it is not enough for this to provide sensible answers to our questions.
It becomes necessary to study psychology and cognitive mechanisms to understand what distinguishes us from machines and fill, at least apparently and as far as possible, the gaps in artificial intelligence. On this point, Nobel Prize winner Daniel Kahneman’s theory on slow and fast thoughts is interesting. According to Kahneman, we can imagine human thought as if it were divided into two systems, which he calls 1 and 2. The first is the intuitive one, which generates quick ideas and makes us make decisions without almost realizing it.
At the same time, the second is the rational one, which governs conscious, deliberative, educable, and, therefore, inevitably, slow thought. Kahneman’s theory, according to Bachini and Tesconi, allows us to understand “one of the most interesting scientific and technological challenges of the moment: teaching machines to be intuitive, to think with the system 1” (“Fake people – stories of social bots and liars digital,” p. 69).
You try to do that with systems machine learning but be careful. As long as the processing of algorithms remains only and purely rational, it will be possible to distinguish even the most sophisticated ones from real people. According to Bachini and Tesconi, what is necessary to recreate human intelligence is to program machines so intuitive as to make mistakes – with all the consequences, even harmful, that derive from it, given that artificial intelligence is often used in particularly sensitive contexts.
Language As The Creator Of A Digital Personality In The Chatbot
In the last century, the philosophy of language has assumed its systematic nature and autonomy concerning the ontological and gnoseological discourses in which it had been inserted since ancient times. Great thinkers in this field brought new life to the study of logic, from which computer science took its first steps. Since its origins, there has been a close connection between analyzing how we describe the outside world and the creation of digital dematerialized realities. After all, the very basis of software is code sequences.
In the case of Kuyda’s chatbot, however, an attempt was made to teach artificial intelligence the language that belonged to a natural person who has now disappeared. In a certain sense, all of Roman Mazurenko’s personality was reduced to the messages he exchanged with friends and relatives, trying to deduce from them, through machine learning, how the man would respond in a new conversation.
One could argue for a long time about whether or not one can understand their nature from a person’s language using probabilistic methods. Still, one point of the story remains firm and undeniable: We will always, inevitably, be faced with a partial representation of the man. Data, however many it may be, will never be able to tell us everything.
The implications, the unsaid, and the intentions are all implicit elements that cannot be deduced from a line of text, not to mention that people are constantly changing. At the same time, in this way, the software crystallizes a version of a particular historical moment and situation – which is not said to be the best possible.
As in the case of Professor Hiroshi Ishiguro, who uses his studies on robotics and artificial intelligence to understand the human soul fully (as if robots could show us what we are starting from what they lack), also in this case of chatbots, there is the risk of searching in the artificial responses of the software for a clue about elements of the “recreated” person that had escaped us.
Of particular impact, in this sense, are the statements that Mazurenko’s mother released to Casey Newton for “The Verge”: the woman claims that there were many things she did not know about her son, but chatting with Kuyda’s chatbot she I feel like I get to know him better as if he were still here. However, it would be essential to understand how much of what the woman reads was said by her son and how much is the result of the processing of messages by the machine learning algorithm– in short, there is a risk of making a person say something, without being sure what they would have thought if they were still alive.
The Right To Be Forgotten And The Digital Identity Of The Deceased
Eugenia Kuyda’s Black Mirror idea also raises some perplexities from the point of view of protecting the privacy and identity of a deceased person whose friends or relatives try to recreate a “simulacrum” through a chatbot. From a legal point of view, the issue is particularly murky.
In Europe, the Personal Data Protection Regulation (GDPR) does not apply to the data of deceased persons, an area whose regulation is left to the discretion of the Member States. In Indian, Legislative Decree 101/2018 introduced art. 2- tendencies, entitled “Rights relating to deceased persons.”
The regulation concerns the exercise of the rights provided by the GDPR on the data of the deceased by those who “have an interest of their own, or act to protect the interested party, as their representative, or for family reasons worthy of protection.” It is, therefore, not easy to understand how to protect the deceased’s identity against behaviors carried out by these categories of people. This happened in the case of Roman Mazurenko, whose data necessary for creating the chatbot were provided to the startup Luka by friends and relatives.
In India, there may be those who, in the interest of the deceased, exercise the right to have their data deleted to prevent their use in the programming of artificial intelligence; however, in the event of an agreement between all the people still alive, it will probably be possible to exploit the data left by the deceased person to teach a chatbot to imitate their language and the little bit of personality that can be deduced from such information.
Conclusions
If until a few years ago, the problems in this sector mainly were linked to the identification of malicious bots and attempted scams and computer crimes, today the frontiers of new technologies are moving further and further away, risking leading us to invade the sphere privacy of individuals in an attempt to “insert” their identity into a virtual mechanism. The individuals themselves could give a possible limit, perhaps leaving a “digital will” to regulate the processing of data concerning them when they can no longer exercise any rights independently.
Read Also: ThinkPhone By Motorola: Born For Business, Ideal For Everyone
