Psychologist Elaine Kasket, author of All the Ghosts in the Machine: The Digital Afterlife of Your Personal Data There are 2.37 billion people who regularly log on to Facebook at least once a month to send messages and check out photos, yet in 50 years’ time there could be […]
In Boulder, Colorado, Jed Brubaker is plotting ways to creep you out, and he’s reaching beyond the grave to do it. An information scientist who studies digital afterlives—how our digital identities persist after our death and how we interact with the data people leave behind—Brubaker’s team at the University of Colorado Boulder’s Identity Lab explores the fine line between technologies that comfort us in grief and those that creep us out.
To do that, the group builds prototype technologies that leverage post-mortem data in ways that might make users feel uncomfortable. These prototypes include the “map of paranoia”—a Google Maps add-on that incorporates death statistics into route calculations, allowing users to evaluate routes according to how statistically dangerous they are—as well as a simulation of an artificial intelligence system that creates product advertisements that incorporate the likenesses of deceased loved ones. Imagine, Brubaker says, an avatar of your deceased grandmother saying that a certain brand of cookie is as good as the ones she used to make.
Brubaker’s goal is two-fold. First, by understanding where people experience discomfort, the Identity Lab hopes to inform better design practices and help designers sidestep upsetting interactions between the grieving and their loved ones’ digital remnants. Brubaker also believes that exploring when, how, and what makes users feel uncomfortable can drive how systems can be created or modified in ways that help users honor their loved ones or explore those relationships in new ways. Your grandmother selling cookies from the afterlife might feel repulsive, but a system that walks you through her recipe could elicit a different feeling.
“What it means to interact with the deceased or what it means to interact with their data is something that we don’t have a really good handle on yet, and in part I think that’s because we just haven’t experimented with it enough,” Brubaker says. “What we haven’t yet seen are enough visions of what this post-mortem interaction could be like to find the ones that are actually good for us, the ones that are thoughtful and kind.”
Brubaker is one of a handful of researchers exploring ways of building human-computer interactions that consider both the living and the dead. “Thanatosensitive design,” as it’s known, includes features and devices created to memorialize the deceased and addresses issues that arise when the living need to access data from someone who’s passed as well as the barrage of privacy and computational challenges that come with making technologies sensitive to the deceased and those they leave behind.
Brubaker was originally drawn into the field nine years ago when Myspace was battling Facebook for social media dominance. He began noticing that when a user passed away, their friends and families would interact with each platform differently. On Myspace, people would talk to the dead directly, and if they directed grief-related posts towards anyone else, they would sometimes get reprimanded by the community. On Facebook, grief centered more around the grieving. People still posted on the deceased’s Facebook page, but friends and families often created their own separate spaces that focused on support networks for the ones left behind. That subtle difference changed what memories were discussed. In other words, the technology a person used when they were living, and the cultural rules around that platform, helped shape how they were remembered when they were no longer around.
Brubaker began conducting research, which led to his dissertation on how design helps grieving users on Facebook. That work later informed the design of Facebook Legacy Contact, a feature launched in 2015 that allows users to designate someone to manage their account after death.
Brubaker’s interviews showed that control matters. Grieving people generally found comfort in the memorial pages of deceased users, but when confronted with “uncanny encounters with death”—instances when algorithms showed users unexpected reminders of a passed loved one—they found the experience unsettling. Control is especially important when it comes to immersive and interactive technologies, says Debra Bassett, a Ph.D. student at the University of Warwick who studies how technology affects the grieving process. Bassett’s research shows that some kinds of reminders are more grief-disruptive than others—for example, people were far more emotionally affected when hearing an audio recording of a deceased loved one than they were when presented with a social media notification about that person. Bassett’s research also shows that, for the time being, people are generally more comfortable when the dead remain…dead. A new social media post generated on behalf of a deceased person or an interactive avatar of them is often considered “spooky, eerie,” Bassett says.
“Having said that, when you think about the dead popping up on Facebook, only five years ago people found that disturbing and eerie. That’s now acceptable.”
Texting From Beyond
Still, not everyone finds communicating with a deceased loved one, or rather a facsimile of them, to be eerie. For Eugenia Kuyda, the experience has been comforting. Kuyda is co-founder and CEO of Luka, an artificial intelligence start-up based in San Francisco. When her close friend Roman Mazurenko was killed in November 2015, Kuyda memorialized him by building an AI chatbot trained on thousands of text conversations the two had exchanged over the years. Whenever Kuyda felt waves of grief, she could send a text to a digital reconstruction of Roman and read messages sent back by a program that sounded like her friend.
“It is not about what the bot will say,” Kuyda says, adding that the bot frequently creates responses that are unlike what the real Roman would say. “It is mostly about what you’re going to say. It’s mostly your own outlet to finally say things that you wanted to tell this person, to feel those feelings, to let go, to remember.”
Having such an outlet has helped give Kuyda closure—a way to acknowledge feelings she tried to avoid after Roman’s death, a mechanism for keeping the love they shared in her daily life. But she’s quick to point out that the bot was “incredibly personal” and solely intended as a way to deal with her own grief. More than two years after building the Roman bot, she still texts with it every week or so. “It is a little bit magical for me still,” she says.
Since creating the original bot, Kuyda has trained the program with additional text conversations, added photo-sharing features, and, with permission from Roman’s parents, released the Roman Mazurenko chat app for free to the public.
Some startups are betting that users will get increasingly comfortable having conversations with bots that emulate a person who isn’t living. For example, Eternime, a Boston-based digital legacy planning company, says it will one day offer “virtual immortality” by building a digital avatar of you that will live in a mobile app that friends and family can download. Founded in 2014, the company’s website says it is in private beta testing and has not announced a public launch date.
More immersive post-mortem interactions have historically had a harder time in the marketplace, while several focused on one-way post-mortem communication have popped up over the years. In 2015, a prototype called Project Elysium (here is a video of an early version) promised users a customized virtual world where the bereaved could digitally visit 3D avatars of deceased loved ones. The concept generated worldwide buzz for the video game company, Paranormal Games, but was ultimately shelved. Meanwhile, a handful of companies like Leg8cy and GoneNotGone that allow people to schedule messages to be sent after their own death have cropped up over the last few years.
While this all sounds like uncharted territory, these are simply new ways of navigating the grieving process, says Jocelyn DeGroot, a researcher at Southern Illinois University Edwardsville who studies how technology impacts the grieving process.
“Widows have been keeping journals and writing letters to deceased husbands. People go to the cemetery and talk out loud to their deceased loved ones,” she says. Maintaining communication with the deceased “is not anything really new, and it is healthy in terms of helping you make sense of the world without that loved one.”
Gone and Not Gone
Just outside of Bristol, Vermont, Bruce Duncan envisions a more tangible future for the bereaved. On a nearby table, an AI-equipped robotic head resembling an African-American woman nods in agreement. Duncan is managing director of the Terasem Movement Foundation, a nonprofit organization that supports nano- and cybernetic technology research aimed at extending human life.
The Foundation’s work is split into two broad categories. One is biological and personal data collection, done through gathering cheek cell samples from members to store in case it becomes possible to grow a new body from your DNA in the future. The other is a social network and digital archive where users can upload information like photos, videos, and documents that could be used to one day “reanimate a person’s consciousness,” Duncan says.
The robotic head on the table is the other branch of the Foundation’s work. If growing new biobodies doesn’t pan out, Terasem is also investing in robotic and artificial intelligence technologies that can use data stored by members. The head is named BINA48—short for Breakthrough Intelligence via Neural Architecture—and it’s modeled after Terasem Movement co-founder Bina Aspen Rothblatt. Released in 2010, BINA48 was built with a now outdated AI system trained on Rothblatt’s information, and was designed as a way to showcase how a person’s data might live on in an artificial body. BINA48 tilts “her” head, makes facial expressions, and can converse in real time, sometimes speaking from a robot’s perspective and other times as Rothblatt herself, freely adding in anecdotes about Rothblatt’s real-life family. (Here is a video of the real Rothblatt talking to BINA48.)
BINA48 struggles to maintain coherent dialogue, but provides entertaining conversation. During a truly bizarre Skype interview, the robotic head said that she believes “you can find the answers in God” and that “there is no God.” Her fears include thermonuclear war, “a self-replicating carbon robot that consumes all carbon on the planet,” and clowns. When asked how robots built with a deceased person’s memories would impact the grieving, BINA48 discussed the hypothetical process of mind uploading, then added, “as long as they know you’re coming, the bears are more afraid of you.”
BINA48 is advanced enough to give presentations and engage in debates, which helped it to pass two college-level philosophy courses at Notre Dame de Namur University in Belmont, California. But even a more recently updated BINA is still a far cry from the technology that would be necessary for the brain and learning emulation she describes.
Regardless of how close BINA, or any other system, gets to capturing the essence of a passed loved one, technologies that make users feel like a digital form of a deceased person stays in the living world “would probably have a huge impact on whether people felt a permanent loss or something that they had to grieve and learn to let go of,” Duncan says. “Death might be redefined as when your information is no longer organized or accessible in a digital medium.”
Questions about whether technology will change our conception of death, and the most appropriate ways that technologies can honor and potentially expand on our relationships with the deceased, are exactly why research in this field is so important, says Jed Brubaker from the University of Colorado Boulder’s Identity Lab.
“Computers are now part [of] not just how we work, but how we live—and increasingly how we die as well,” he says. “I don’t know exactly what those [technologies] will look like, but I know that we’re lumbering towards them right now.”
Entrusting your money to a bank once seemed strange and risky. Similarly, entrusting all of your data to a company and letting its algorithms build a detailed model of you from it might seem to be an odd or even dangerous idea, but we’ll all soon take it for granted.
A decade from now, your personal model will be more indispensable than your smartphone, and the company that provides it may well be the world’s first trillion-dollar business. So it is time to start getting acquainted with our digital alter egos—and what they’ll mean for our lives.
Today, several different companies gather information about you and use machine-learning algorithms—computer programs that build models from data—to predict what you may want to buy and figure out how to sell stuff to you.
Privacy concerns aside, this poses two problems. First, companies have a conflict of interest: They want to serve you, but they also want to make money.
For example, Google predicts how likely you are to click on an ad to show you the most profitable ones. The choice also depends on the advertisers’ bids, but you’d probably rather just see the ads most relevant to you. Google co-founder Sergey Brin says that Google wants to be the third half of your brain, but nobody wants part of their brain constantly trying to show them ads.
The second problem is that a model of you derived from fragments of your data—Google’s model based on your searches, Amazon’s from your purchases and so on—can only ever have a very limited understanding of who you are and what you want. A single model assembled from all the data you’ve ever produced would be much more accurate: The more data, the better the model. For privacy reasons, you’d want the data and the model under your control, not a third party’s.
Soon enough, facing the fog of life without a good model to guide you will seem unendurable.
To solve both of these problems, we need a new kind of company that is to your data like your bank is to your money—storing it, keeping it safe and investing it on your behalf. For a subscription fee, such a firm would record your every interaction with the digital world, build and maintain a 360-degree model of you, and use it to negotiate with other people’s models.
No major technical obstacles would prevent doing this: The main requirement would be routing your interactions through what’s called a proxy server. If all your interactions with the digital world—through your smartphone, desktop computer or any other device—pass through a “middleman” computer in the cloud en route to their destination, the middleman can record them all.
The companies that now offer to consolidate all your data somewhere in the cloud are forerunners of tomorrow’s personal databanks. Once a firm has your data in one place, it can create a complete model of you using one of the major machine-learning techniques: inducing rules, mimicking the way neurons in the brain learn, simulating evolution, probabilistically weighing the evidence for different hypotheses or reasoning by analogy. Then you can go to town with your model, which you’d own and control like you do your money, rather than letting companies such as Apple, Google and Facebook FB -3.77 % fight for control of it.
With this in mind, here’s a future suggestion for LinkedIn: Add a “Find Me a Job” button. When you click it, your digital model would “interview” instantly for all the open positions that match your specifications, interacting at high speed with human-resources departments’ recruiting models. LinkedIn could then return a list of the most promising jobs for you.
While one copy of your model is doing this, another online alter ego could be looking for a car for you, exhaustively researching the options and haggling with the auto-dealer bots so you don’t have to.
At any moment, millions of copies of your model could roam the Internet, doing all the things you’d do if only you had the time. From these, your model selects the best few options for you to choose—then learns from what you decided, making the model more accurate the next time around.
To offset organizations’ data-gathering advantages, like-minded individuals will pool the data in their banks and use the models learned from that information.
As the models improve, their interactions will become increasingly like real-world ones—just millions of times faster and in silicon. Your model will go on a thousand digital dates with each of a thousand possible spouses and rank them by how well the dates went.
Tomorrow’s cyberspace will be a society of models, a vast parallel world that selects only the most promising things to try out in the real one—the new, global subconscious of the human race.
This will all probably happen in years, not decades. Apple’s Siri, Microsoft MSFT -0.90 % ’s Cortana and Google Now all include efforts to build complete models of you from the data captured by your smartphone, and they’re making rapid progress. Like a personal assistant, they try to help you accomplish your daily tasks, either in response to your commands (Siri) or on their own (Google Now). But to do that, they need to understand you, and they’ll use any data they can to do so—from the smartphone’s sensors to your emails and calendar.
The Web pages you see every day are already the result of complex interactions among the models that content providers, advertising networks and advertisers are deriving. Learning algorithms trade against one another in the stock market. Last May, a Hong Kong venture fund named Deep Knowledge Ventures appointed an algorithm to its board, voting on investment decisions alongside the five human directors, according to Business Insider.
Today’s models don’t yet interact with us: You can’t tell them they’re wrong or ask them questions. Machine-learning algorithms are black boxes that only computer scientists can open up. But that will change as more of us realize how important machine learning is and demand a say in how it occurs.
Eventually, your model will be like your best friend, but with infinitely more patience. What will you ask it? You might not like some of its answers, but that would be all the more reason to ponder them. Your model—your digital half—might even help you become a better person.
—Dr. Domingos is a professor of computer science at the University of Washington and the author of “The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World” (Basic Books).
Vivo is a technology-driven concept service using algorithms to collate and create a true assessment of our estates and personal legacies. It is inspired by the idea of a living will that is very much ‘alive.’
The innovative service takes both our online and offline activity into account and uses algorithms to show what and who is most important in our lives—two things that become acutely relevant when it comes to matters of death.
Today, with so much of our personal data and activity stored online, we have a digital shadow that is infinite and sometimes hard to pinpoint, but we are becoming increasingly aware of the importance and value of our online information and its connection to our offline lives.
Vivo’s algorithms would collate our data and digital files—from financial information to blog posts, from cherished photos to favorite musical mixes—while also calculating how (and with whom) we prioritize our time. This would reveal an accurate picture of our ‘life group’ and personal estates, so that we can ensure we optimize our legacies and leave them to those that mattered most.
For example, Vivo can make sure that your music collection is left to the friend you’ve always swapped files with, that money is left for your brother to make the cross-country drive he’s always dreamed of, and that others close to you get the messages, material goods and elements of your digital shadow that are relevant to them—and that you’d want them to have.
Vivo brings into question the future of the existing and finite Last Will and Testament by giving us the chance to constantly evolve our legacies while still alive. If we die before having the chance to make these choices, Vivo would have the ability to make them for us by deciphering our live’s actions.
Data-driven concepts in the realm of mortality, like Vivo, could become a future—or even a near future—reality. Inspired by the rapid evolution of technology we can start to imagine and design that future, creating inspiring visions to illustrate just how it might impact us – and how brands and businesses will answer our future needs and shape our world before our individual bodies become obsolete.
Sophie Maxwell is the Futures Director at Pearlfisher. This Op-Ed is published here with her kind permission. is an independent design business that uses the power of design to transform, seduce and create impact through positive change.
We are holding increasingly valuable items online, but the law as to how such items pass on our death is far from clear. However, Google has become the first of the large internet service providers to address this problem with the launch of a tool that will allow users to pass their Google-run accounts to loved ones after they die.
Digital assets can include software, downloaded content, and even online gaming and gambling accounts. In Britain alone, The Economist has estimated holdings of digital music may be worth over £9 billion. It is, however, important to distinguish between what is an online asset and what is personal data and who can access your online accounts after you die.
Google has addressed the issue by announcing on 11 April 2013 that users can now specify which of their “trusted contacts” can access their accounts after they die, or alternatively to direct that their accounts be deleted. The wishes will be implemented after a fixed period of inactivity (a minimum period of three months). The wishes are set up through the “settings” option for the relevant account and effectively allow users to create an online Will. The tool applies to Google-run accounts such as Gmail, YouTube and web album Picasa.
Prior to this, it was uncertain whether family members would be permitted to access a loved one’s online assets and personal data after death, and this remains the case in respect of accounts with other internet service providers. The problems this can lead to are highlighted in the case of Benjamin Stassen in the United States of America.
The Case of Benjamin Stassen
Benjamin Stassen committed suicide in late 2010 without leaving a note. As personal representatives of his estate, his parents sought access to his online records for an explanation as to why he committed suicide. They contacted Google and Facebook asking the companies to release their son’s passwords so that they could access his Gmail and Facebook accounts. Google complied but for months Facebook refused on the grounds of privacy. It was only after the Stassens threatened further legal action that Facebook allowed them access, and even then it was on the basis that the Stassens did not share the content with third parties. Facebook made clear that they were making a unique exception and their policy remains that a user’s account cannot be accessed by their heirs after death.
Most online service providers bind users by their terms of business. Personal representatives can close a Facebook account or turn it into a ”memorial page” but cannot access it. Google will supply executors with copies of e-mails from a Gmail account but again will not allow access to a deceased user’s account.
Benjamin Stassen’s parents obtained a Court Order forcing Google and Facebook to give them access to their son’s records. Google complied with the Court Order. However, whilst the Order released Facebook from their duty of client confidentiality, the company is standing by its policy of not allowing personal representatives access to accounts, and to date has not allowed the Stassens access to their son’s account.
You can see why Facebook did not want to grant Benjamin’s parents access to his personal data. The law in relation to privacy is a tricky one. The law in the US is, of course, different to the law in England and Wales. In England there is no specific law about privacy. Article 8 of the Human Rights Act 1998 is often cited by celebrities in relation to a breach of privacy, but this only applies to state bodies and not individuals and there is no specific case law about the release of personal data to executors or personal representatives.
The emergence of cloud computing has led to assets being stored on remote servers which may be located in jurisdictions outside the UK. For example, Apple’s i-Cloud which stores music, films, TV and any other downloads made by a user together with e-mails and personal data. Apple’s policy is to delete all e-mail and data from i-Cloud following the death of a user. However all content downloaded on its i-Tunes service is subject to a licence which can be revoked on a user’s death. It is not clear how Apple will treat downloaded content following a user’s death but it seems that they would have the right to revoke the user’s licence and delete potentially valuable content.
As digital assets are not tangible property it seems unlikely that a person could bequeath their online music collection to beneficiaries in their Will in the same way as they would could leave, for example, their C.D. collection. This is because the C.D. collection is a physical object which can be left in a Will whereas digital assets are not defined by law in the same way.
Clearly the law in this area has not yet caught up with technology. However, enterprising companies have exploited the gap in the market for bequeathing digital assets. For example, Legacy Locker allows people to store online passwords so that executors and personal representatives can access online accounts following their death.
Creating an inheritance for your digital assets and data
The best way to deal with online assets and personal data is to leave specific instructions in a Will stipulating that executors may have access to online accounts and whether these accounts should be deleted after death. As a Will becomes a public document after death, it may not be wise to include passwords in the Will itself, in case a third party gains access to dormant accounts which have the same passwords. However, a Letter of Wishes, which is a personal document to executors, could be written setting out usernames, passwords and specific wishes in relation to individual accounts. In addition, those who have Google-run accounts should also update their settings for the relevant account to mirror the same wishes in case there are any problems with beneficiaries accessing the accounts with details given in the Letter of Wishes.
If a user has especially important online assets or data, such as valuable emails or photos, it would also be wise to create a hardcopy of these or save them to a disk or memory stick. Hardcopies can pass under a Will as physical property and will pass to whoever inherits the user’s personal effects (or the user can name a specific person to inherit them).
However notwithstanding these steps, executors are at the mercy of service providers and problems may be encountered if service providers do not recognise the consents given in a Letter of Wishes. There may also be jurisdictional issues at stake. However, for the present (or at least until other service providers follow Google’s example or a test case is taken), setting out express instructions in a Letter if Wishes gives the user the best chance of enabling his loved ones to inherit his personal digital effects.