söndag 21 januari 2018

SVT:s Idévärlden om AI-risk

Veckans avsnitt av SVT:s Idévärlden behandlar risker med framtida utveckling inom artificiell intelligens (AI), och kan ses i SVT Play. AI-risk var ett framträdande tema vid det gästforskarprogrm om Existential risk to humanity som jag var med och drev vid Chalmers och Göteborgs universitet i höstas (något som t.ex. framgick av flera av föredragen vid vår publika workshop i september), och därför är det kanske inte helt ologiskt att samtliga tre gäster i SVT:programmet hämtades bland deltagarna i vårt gästforskarprogram: Anders Sandberg, Dorna Behdadi och Karim Jebari. Det blev ett riktigt bra avsnitt av Idévärlden, något som i alla fall för mig var ytterst väntat, i ljuset inte bara av de utmärkta interna diskussioner vi hade i gästforskarprogrammet, utan också av de publika paneldiskussioner jag de senaste åren deltagit i med var och en av tre: Anders, Dorna och Karim. Även programledaren Daniel Sjölin gjorde ett bra jobb.

onsdag 17 januari 2018

Two excellent contemporary writers

Two of my favorite contemporary writers, operating however in very different genres, are Ted Chiang and Scott Alexander:
  • Ted Chiang is a science fiction writer specializing in short stories. When I read his collection Stories of Your Life and Others I said to myself "wow, this guy is almost better than Greg Egan" (but let me withhold final judgement on that comparison). The book opens with Tower of Babylon, which explores a beautiful alternative cosmology more in line with what people believed in ancient times, and continues with Understand, which, albeit lacking somewhat in realism, gives what is probably the best account I've read on what it might be like to attain superintelligence - an impossible topic, yet important in view of possible future transhumanistic developments. Among the other stories in the book is the title one, Story of Your Life, which was later adapted to the Hollywood movie Arrival; I recommend both reading the story and seeing the movie (the plots diverge somewhat in interesting respects) and then listening to the podcast Very Bad Wizards discussing them.

  • Scott Alexander blogs about science, philosophy, future technologies and related topics. He often penetrates quite deeply into his chosen topic, and his posts are often longish to very long. Several of his blog posts have influenced me significantly, such as...
Excellence is writing may however be more or less genre-specific; I suspect that most good authors of university-level mathematics textbooks suck as poets, and vice versa. Another example is that when last month Ted Chiang tried his luck at writing an essay on AI futurology with the title Silicon Valley is Turning into its Own Worst Fear, his brilliance does not shine through. The gist of Chiang's argument is that superintelligent AI and capitalism are similar in that they both relentlessly optimize for something that is not entirely well-aligned with human well-being, and that since superintelligent AI does not at present exist while capitalism does, only capitalism poses a real danger. This last conclusion is a non sequitur.

And now, finally... get ready for my excuse for discussing Chiang and Alexander in the same blog post! Scott Alexander's blog post Maybe the Real Superintelligent AI is Extremely Smart Computers from earlier this week is a masterful exposition of the errors in Chiang's arguments. When I first saw Chiang's essay, I saw mostly the same errors that Alexander saw, but would never have been able to explain them quite as pedagogically as he does. Do read it (Alexander's blog post, that is), as I have nothing to add to it.

lördag 13 januari 2018

Science Magazine on existential risk

I recommend reading the news article on existential risk research featured in Science Magazine this week. Several prominent researchers are interviewed, including physicist Max Tegmark, who should by now be well known to regular readers of this blog. Like philosopher Nick Bostrom, whose work (including his excellent 2014 book Superintelligence) is discussed in the article, he emphasizes a possible future breakthrough in artificial intelligence (AI) as one of the main existential risks to humanity. Tegmark explains, as Bostrom has done before him and as I have done in an earlier blog post, that the trial-and-error method that has served humanity so well throughout history is insufficient in the presence of catastrophic risks of this magnitude:
    Scientists have an obligation to be involved, says Tegmark, because the risks are unlike any the world has faced before. Every time new technologies emerged in the past, he points out, humanity waited until their risks were apparent before learning to curtail them. Fire killed people and destroyed cities, so humans invented fire extinguishers and flame retardants. With automobiles came traffic deaths—and then seat belts and airbags. "Humanity's strategy is to learn from mistakes," Tegmark says. "When the end of the world is at stake, that is a terrible strategy."
The alternative to trial-and-error is to raise the level of foresight, and partly for this reason it is excellent that existential risk research in general and AI safety work in particular gets the kind of media exposure that the Science Magazine article exemplifies.

On the other hand, this line of research is controversial in some circuits, whence today's media logic dictates that its adversaries are heard. Recently, cognitive scientist Steven Pinker has become the perhaps most visible such adversary, and he gets to have his say in the Science Magazine article. Unfortunately, he seems to have nothing to offer beyond recycling the catchy oneliners he used when I met him face to face at the EU Parliament in Brussels in October last year - oneliners whose hollowness I later exposed in my blog post The AI meeting in Brussels last week and at greater length in my paper Remarks on artificial intelligence and rational optimism. Pinker's poor performance in these discussions gives the impression (which I will not contradict) that proponents of the position "Let's not worry about apocalyptic AI risk!" do not have good arguments for it. The impression is reinforced by how even leading AI researchers like Yann LeCun, when trying to defend that position, choose to revert to arguments on about the same level as those employed by Pinker. To me, that adds to the evidence that apocalyptic AI risk does merit taking seriously. Readers who agree with me on this and want to learn more can for instance start by reading my aforementioned paper, which offers a gentle introduction and suggestions for further reading.

fredag 5 januari 2018

Emma Frans årets folkbildare!

Kom ihåg vad jag skrev här på bloggen redan i juli 2014, långt innan Emma Frans nådde den nivå av berömmelse hon har idag:
    Emma Frans är ofta först bland svenskspråkiga skribenter med spännande nyheter om beteendevetenskap och annan forksning, och om någon vill utnämna Emmas selektion till Sveriges bästa vetenskapsblogg just nu så har jag inget att invända.
Just hennes blogg har varit halvt om halvt vilande det senaste året, men Emma arbetar desto mer med folkbildning via en rad andra kanaler, och idag har Föreningen Vetenskap och Folkbildning (VoF) tillkännagivit att hon utsetts till Årets folkbildare 2017. I skrivande stund är deras webbplats svår att nå på grund av den överlastning av trafik som tillkännagivandet givit upphov till, men på föreningens ordförande Peter Olaussons blogg Faktoider hittar vi följande uttalande:
    Föreningen Vetenskap och Folkbildning (VoF) har utsett Emma Frans, doktor i medicinsk epidemiologi, till Årets folkbildare 2017. Emma Frans tilldelas utmärkelsen för sin förmåga att på ett pedagogiskt och humoristiskt sätt sprida kunskap och förklara myter och missförstånd kring vetenskap.

    – Emma Frans har fått särskilt mycket uppmärksamhet det senaste året och det förtjänar hon. Hon är den första folkbildaren som började sin verksamhet på sociala medier och det är fortfarande där hon har sin starkaste plattform. Sociala medier har visat sig vara vår tids främsta kanal för förvillande – där behövs källkritik och folkbildning mer än någon annanstans, säger Peter Olausson, ordförande för Vetenskap och Folkbildning.

    – Men det är inte tillräckligt att behärska en kanal. Emma Frans är verksam på Twitter, och en stor tidning, och nu även i bokform. Det är så en modern folkbildare behöver arbeta: I olika kanaler med olika möjligheter och framför allt olika målgrupper.

    [...]

    Doktor Emma Frans gav under 2017 ut boken Larmrapporten - Att skilja vetenskap från trams, på Volante förlag. I boken redogör Frans på ett pedagogisk sätt för hur vi som individer kan navigera genom den djungel av information som florerar runt omkring oss dagligen. Med hjälp av lustiga anekdoter tagna ur vardagslivet beskrivs begrepp som placeboeffekten och cherry picking. Även avdelningar rörande källkritik, informationsinhämtning och statistik avhandlas med hjälp av tydliga exempel, ofta hämtade från verkligheten.

    Genom att Emma Frans uttrycker sig lättbegripligt och ofta med en stor portion humor, utan att tumma på det vetenskapliga förhållningssättet, är hon en god folkbildare i den digitala tidsåldern.

Jag instämmer i allt detta, och kan tillägga att jag funnit Emmas bok så bra, med dess närmast unika kombination av lättillgänglighet och vederhäftighet, att jag avser använda den på den kurs inom lärarutbildningen på Göteborgs universitet som jag är engagerad i.

Grattis Emma!

lördag 23 december 2017

Do not obey in advance

About the recent public outcry about a 1984-style ban on the terms vulnerable, entitlement, diversity, transgender, fetus, evidence-based, and science-based at the United States Centers for Disease Control and Prevention (CDC). It turns out, according to a Slate article a couple of days ago, that the ban came not from the Trump administration, but from within. From the Slate article:
    Anonymous sources at the Department of Health and Human Services told the National Review’s Yuval Levin this week that any language changes did not originate with political appointees, but instead came from career CDC officials who were strategizing how best to frame their upcoming budget request to Congress. What we’re seeing, his interviews suggest, is not a top-down effort to stamp out certain public-health initiatives, like those that aim to help the LGTBQ community, but, in fact, the opposite: a bottom-up attempt by lifers in the agency to reframe (and thus preserve) the very work they suspect may be in the greatest danger.
This does not make the case any less worrisome. It is a good example of what Timothy Snyder calls obedience in advance. The Trump administration didn't have to tell the CDC to suppress talk about diversity and transgender, because managers at the CDC are sufficiently sensitive to figure out for themselves what the politicians in Washington want. Their reasons, as explained in the Slate quote above, may be respectable, but still, obeying an authoritarian-leraning regime in advance in this way is a dangerous step. The first lesson in Snyder's important book On Tyranny: Twenty Lessons from the Twentieth Century is entitled "Do not obey in advance", and here is his three-sentence summary of that lesson:
    Most of the power of authoritarianism is freely given. In times like these, individuals think ahead about what a more repressive government will want, and then offer themselves without being asked. A citizen who adapts in this way is teaching power what it can do.
Other lessons in Snyder's book with direct relevance to the CDC case are Lesson 5 ("Remember professional ethics") and Lesson 9 ("Be kind to our language").

I do not often reread books, but I've read Snyder's On Tyranny three times in 2017. My goal is to learn his lessons so well that when the time comes, my default reactions will be in line with his lessons.

(Note to my Swedish readers: Snyder's book is explicitly about present-day United States, but his lessons are important here too. We are not immune to authoritarianism. Do not let yourselves be fooled for a second into thinking that Grotesco's satirical account of the evening of our parliamentary election day in 2022 is the slightest bit implausible.)

fredag 22 december 2017

Three papers on AI futurology

Here are links to three papers on various aspects of artificial intelligence (AI) futurology that I've finished in the past six months, arranged in order of increasing technical detail (i.e., the easiest-to-read paper first, but none of them is anywhere near the really technical math papers I've written over the years).
  • O. Häggström: Remarks on artificial intelligence and rational optimism, accepted for publication in a volume dedicated to the STOA meeting of October 19.

    Introduction. The future of artificial intelligence (AI) and its impact on humanity is an important topic. It was treated in a panel discussion hosted by the EU Parliament’s STOA (Science and Technology Options Assessment) committee in Brussels on October 19, 2017. Steven Pinker served as the meeting’s main speaker, with Peter Bentley, Miles Brundage, Thomas Metzinger and myself as additional panelists; see the video at [STOA]. This essay is based on my preparations for that event, together with some reflections (partly recycled from my blog post [H17]) on what was said by other panelists at the meeting.

  • O. Häggström: Aspects of mind uploading, submitted for publication.

    Abstract. Mind uploading is the hypothetical future technology of transferring human minds to computer hardware using whole-brain emulation. After a brief review of the technological prospects for mind uploading, a range of philosophical and ethical aspects of the technology are reviewed. These include questions about whether uploads will have consciousness and whether uploading will preserve personal identity, as well as what impact on society a working uploading technology is likely to have and whether these impacts are desirable. The issue of whether we ought to move forwards towards uploading technology remains as unclear as ever.

  • O. Häggström: Strategies for an unfriendly oracle AI with reset button, in Artificial Intelligence Safety and Security (ed. Roman Yampolskiy), CRC Press, to appear.

    Abstract. Developing a superintelligent AI might be very dangerous if it turns out to be unfriendly, in the sense of having goals and values that are not well-aligned with human values. One well-known idea for how to handle such danger is to keep the AI boxed in and unable to influence the world outside the box other than through a narrow and carefully controlled channel, until it has been deemed safe. Here we consider the special case, proposed by Toby Ord, of an oracle AI with reset button: an AI whose only available action is to answer yes/no questions from us and which is reset after every answer. Is there a way for the AI under such circumstances to smuggle out a dangerous message that might help it escape the box or otherwise influence the world for its unfriendly purposes? Some strategies are discussed, along with possible countermeasures by human safety administrators. In principle it may be doable for the AI, but whether it can be done in practice remains unclear, and depends on subtle issues concerning how the AI can conceal that it is giving us dishonest answers.

fredag 15 december 2017

Litet efterspel till panelen i Bryssel

Sent i lördags eftermiddag (den 9 december) ökade plötsligt trafiken hit till bloggen kraftigt. Den kom mestadels från USA, och landade till större delen på min bloggpost The AI meeting in Brussels last week från förrförra månaden, så till den grad att bloggposten inom loppet av 24 timmar klättrade från typ ingenstans till tredje plats på all-time-high-listan över denna bloggs mest lästa inlägg (slagen endast av de gamla bloggposterna Quickologisk sannolikhetskalkyl och Om statistisk signifikans, epigenetik och de norrbottniska farmödrarna). Givetvis blev jag nyfiken på vad som kunde ha orsakat denna trafikökning, och fann snabbt en Facebookupdatering av den framstående AI-forskaren Yann LeCun vid New York University. LeCun länkar till min Bryssel-bloggpost, och har ett imponerande antal Facebook-följare, vilket förklarar trafikökningen.

Precis som sin AI-forskarkollega Peter Bentley och kognitionsforskaren Steven Pinker - vilka båda deltog i Bryssel-panelen - anser LeCun att alla farhågor om en eventuell framtida AI-apokalyps är obefogade. Hans imponerande meriter som AI-forskare väcker såklart förhoppningar (hos den som läser hans Facebookuppdatering) om att han skall presentera väsentligt bättre argument för denna ståndpunkt än dem som Pinker och Bentley levererade i Bryssel - förhoppningar som dock genast kommer på skam. LeCuns retoriska huvudnummer är följande bisarra jämförelse:
    [F]ear mongering now about possible Terminator scenarios is a bit like saying in the mid 19th century that the automobile will destroy humanity because, although we might someday figure out how to build internal combustion engines, we have no idea how to build brakes and safety belts, and we should be very, very worried.
Vad som gör jämförelsen bisarr är att (åtminstone 1900-talets och dagens) bilar totalt saknar de självreproducerande och rekursivt självförbättrande egenskaper som en tillräckligt intelligent framtida AI enligt många bedömare kan väntas få, vilka är grunden för de potentiellt katstrofala intelligensexplosionsscenarier som diskuteras av bland andra Yudkowsky, Bostrom och Tegmark (liksom i min egen bok Here Be Dragons). Till skillnad mot i AI-scenarierna finns inget rimligt bilscenario där vår oförmåga att bygga fungerande bromsar skulle ta kål på mänskligheten (allt som skulle hända om det inte gick att få till en fungerande bromsteknologi vore att ingen skulle vilja köra bil och att biltillverkningen upphörde). Vad som krävs för att LeCuns jämförelse skall få minsta relevans är att han påvisar att fenomenet med en rekursivt självförbättrande AI inte kan bli verklighet, men han redovisar inte tillstymmelse till sådant argument.

LeCuns Facebookuppdatering bjuder också på en direkt självmotsägelse: han hävdar dels att "we have no idea of the basic principles of a purported human-level AI", dels att...
    [t]he emergence of human-level AI will not be a singular event (as in many Hollywood scenarios). It will be progressive over many many years. I'd love to believe that there is a single principle and recipe for human-level AI (it would make my research program a lot easier). But the reality is always more complicated. Even if there is a small number of simple principles, it will take decades of work to actually reduce it to practice.
Här frågar sig naturligtvis den vakne läsaren: om nu LeCun har rätt i sitt första påstående, hur i hela glödheta h-e kan han då ha den kunskap han gör anspråk på i det andra? Det kan naturligtvis hända att han har rätt i att utvecklingen kommer att gå långsamt, men hans dogmatiska tvärsäkerhet är (i avsaknad av solida argument för att backa upp ståndpunkten) direkt omdömeslös.

Vad som gör LeCuns Facebookuppdatering den 9 december ännu mer beklämmande är att dess text är kopierad från en kommentar han skrev i en annan Facebooktråd redan den 30 oktober. Uppenbarligen tyckte han, trots att han haft mer än en månad på sig att begrunda saken, att hans slagfärdigheter var av tillräckligt värde för att förtjäna ytterligare spridning.