Science and Technology

One small part of the recent (June 2015) CFAR workshop caused a significant improvement in how I interact with people. I’ve become more spontaneous about interacting with people.

For several years I’ve suspected that I ought to learn how to do improv-style exercises, but standard improv classes felt ineffective. I’ve since figured out that their implied obligation for me to come up with something to say caused some sort of negative association with attempts at spontaneity when I failed to think of anything to say. That negative reaction was a large obstacle to learning new habits.

Deeply ingrained habits seem to cause some part of my subconscious mind that searches for ideas or generates words to decide that it can’t come up with anything worthy of conscious attention. That leaves me in a state that I roughly describe as a blank mind (i.e. either no verbal content at the conscious level, or I generate not-very-useful meta-thoughts reacting to the lack of appropriate words).

Since I much more frequently regret failing to say something than I regret mistakenly saying something hastily that I should have known not to say, it seems like I’ve got one or more subconscious filters that has consistently erred in being too cautious about generating speech. I tried introspecting for ways to simply tell that filter to be less cautious, but I accomplished nothing that way.

I also tried paying attention to signs that I’d filtered something out (pauses in my flow of words seem to be reliable indicators) in hopes that I could sometimes identify the discarded thoughts. I hoped to reward myself for noticing the ideas as the filter started to discard them, and train the filter to learn that I value conscious access to those ideas. Yet I never seem to detect those ideas, so that strategy failed.

What finally worked was that I practiced informal versions of improv exercises in which I rewarded myself [*] for saying silly things (alone or in a practice session with Robert) without putting myself in a situation where I felt an immediate obligation to say anything unusual.

In a few weeks I could tell that I was more confident in social contexts and more able to come up with things to say.

I feel less introverted, in the sense that a given amount of conversation tires me less than it used to. Blogging also seems to require a bit less energy.

I feel somewhat less anxiety (and relatedly, less distraction from background noise), maybe due to my increased social confidence.

I may have become slightly more creative in a variety of contexts.

I hypothesize that the filtering module was rather attached to a feeling of identity along the lines of “Peter is a person who is cautious about what he says” long after the consciously accessible parts of my mind decided I should weaken that identity. Actually trying out a different identity was more important to altering some beliefs that were deeply buried in my subconscious than was conscious choice about what to believe.

I wonder what other subconscious attachments to an identity are constraining me?

Something still seems missing from my social interactions: I still tend to feel passive and become just a spectator. That seems like a promising candidate for an area where I ought to alter some subconscious beliefs. But I find it harder to focus on a comfortable vision for an alternative identity: aiming to be a leader in a group conversation feels uncomfortable in a way that aiming to be spontaneous/creative never felt.

Thanks to John Salvatier and Anna Salamon for the advice that helped me accomplish this.

[*] – I only know how to do very weak self-rewards (telling myself to be happy), but that was all I needed.

Book review: Artificial Superintelligence: A Futuristic Approach, by Roman V. Yampolskiy.

This strange book has some entertainment value, and might even enlighten you a bit about the risks of AI. It presents many ideas, with occasional attempts to distinguish the important ones from the jokes.

I had hoped for an analysis that reflected a strong understanding of which software approaches were most likely to work. Yampolskiy knows something about computer science, but doesn’t strike me as someone with experience at writing useful code. His claim that “to increase their speed [AIs] will attempt to minimize the size of their source code” sounds like a misconception that wouldn’t occur to an experienced programmer. And his chapter “How to Prove You Invented Superintelligence So No One Else Can Steal It” seems like a cute game that someone might play with if he cared more about passing a theoretical computer science class than about, say, making money on the stock market, or making sure the superintelligence didn’t destroy the world.

I’m still puzzling over some of his novel suggestions for reducing AI risks. How would “convincing robots to worship humans as gods” differ from the proposed Friendly AI? Would such robots notice (and resolve in possibly undesirable ways) contradictions in their models of human nature?

Other suggestions are easy to reject, such as hoping AIs will need us for our psychokinetic abilities (abilities that Yampolskiy says are shown by peer-reviewed experiments associated with the Global Consciousness Project).

The style is also weird. Some chapters were previously published as separate papers, and weren’t adapted to fit together. It was annoying to occasionally see sentences that seemed identical to ones in a prior chapter.

The author even has strange ideas about what needs footnoting. E.g. when discussing the physical limits to intelligence, he cites (Einstein 1905).

Only read this if you’ve read other authors on this subject first.

I use Beeminder occasionally. The site’s emails normally suffice to bug me into accomplishing whatever I’ve committed to doing. But I only use it for a few tasks for which my motivation is marginal. Most of the times that I consider using Beeminder, I either figure out how to motivate myself properly, or (more often) decide that my goal isn’t important.

The real value of Beeminder is that if I want to compel future-me to do something, I can’t give up by using the excuse that future-me is lazy or unreliable. Instead, I find myself wondering why I’m unwilling to risk $X to make myself likely to complete the task. That typically causes me to notice legitimate doubts about how highly I value the result.

Book review: The Charisma Myth: How Anyone Can Master the Art and Science of Personal Magnetism, by Olivia Fox Cabane.

This book provides clear and well-organized instructions on how to become more charismatic.

It does not make the process sound easy. My experience with some of her suggestions (gratitude journalling and meditation) seems typical of her ideas – they took a good deal of attention, and probably caused gradual improvements in my life, but the effects were subtle enough to leave lots of uncertainty about how effective they were.

Many parts of the book talk as if more charisma is clearly better, but occasionally she talks about downsides such as being convincing even when you’re wrong. The chapter that distinguishes four types of charisma (focus, kindness, visionary, and authority) helped me clarify what I want and don’t want from charisma. Yet I still feel a good deal of conflict about how much charisma I want, due to doubts about whether I can separate the good from the bad. I’ve had some bad experiences in with feeling and sounding confident about investments in specific stocks has caused me to lose money by holding those stocks too long. I don’t think I can increase my visionary or authority charisma without repeating that kind of mistake unless I can somehow avoid talking about investments when I turn on those types of charisma.

I’ve been trying the exercises that are designed to boost self-compassion, but my doubts about the effort required for good charisma and about the desirability of being charismatic have limited the energy I’m willing to put into it.

Book review: Value-Focused Thinking: A Path to Creative Decisionmaking, by Ralph L. Keeney.

This book argues for focusing on values (goals/objectives) when making decisions, as opposed to the more usual alternative-focused decisionmaking.

The basic idea seems good. Alternative-focused thinking draws our attention away from our values and discourages us from creatively generating new possibilities to choose from. It tends to have us frame decisions as responses to problems, which leads us to associate decisions with undesirable emotions, when we could view decisions as opportunities.

A good deal of the book describes examples of good decisionmaking, but those rarely provide insight into how to avoid common mistakes or to do unusually well.

Occasionally the book switches to some dull math, without clear explanations of what benefit the rigor provides.

The book also includes good descriptions of how to measure the things that matter, but How to Measure Anything by Douglas Hubbard does that much better.

I recently got Bose QuietComfort 15 Acoustic Noise Cancelling Headphones.

I had previously tried passive earplugs and headphones that claimed 30 dB noise reduction, and got little value out of them.

The noise cancelling headphones suppress a good deal more train (BART) noise, enough that I’m now able to read nonfiction while on the train.

It won’t help with the situations where noise bothers me most (multiple conversations nearby) because it mainly eliminates predictable noises. It makes speech sound more distant without affecting the speech volume a lot. But reducing the cost of train and plane travel is valuable enough that I feel foolish about not having tried them earlier.

Book review: The Depths: The Evolutionary Origins of the Depression Epidemic, by Johnathan Rottenberg.

This book presents a clear explanation of why the basic outlines of depression look like an evolutionary adaptation to problems such as famine or humiliation. But he ignores many features that still puzzle me. Evolution seems unlikely to select for suicide. Why does loss of a child cause depression rather than some higher-energy negative emotion? What influences the breadth of learned helplessness?

He claims depression has been increasing over the last generation or so, but the evidence he presents can easily be explained by increased willingness to admit to and diagnose depression. He has at least one idea why it’s increasing (increased pressure to be happy), but I can come up with ideas that have the opposite effect (e.g. increased ease of finding a group where one can fit in).

Much of the book has little to do with the origins of depression, and is dominated by descriptions of and anecdotes about how depression works.

He spends a fair amount of time talking about the frequently overlooked late stages of depression recovery, where antidepressants aren’t much use and people can easily fall back into depression.

The book includes a bit of self-help advice to use positive psychology, and to not rely on drugs for much more than an initial nudge in the right direction.

Book review: Superintelligence: Paths, Dangers, Strategies, by Nick Bostrom.

This book is substantially more thoughtful than previous books on AGI risk, and substantially better organized than the previous thoughtful writings on the subject.

Bostrom’s discussion of AGI takeoff speed is disappointingly philosophical. Many sources (most recently CFAR) have told me to rely on the outside view to forecast how long something will take. We’ve got lots of weak evidence about the nature of intelligence, how it evolved, and about how various kinds of software improve, providing data for an outside view. Bostrom assigns a vague but implausibly high probability to AI going from human-equivalent to more powerful than humanity as a whole in days, with little thought of this kind of empirical check.

I’ll discuss this more in a separate post which is more about the general AI foom debate than about this book.

Bostrom’s discussion of how takeoff speed influences the chance of a winner-take-all scenario makes it clear that disagreements over takeoff speed are pretty much the only cause of my disagreement with him over the likelihood of a winner-take-all outcome. Other writers aren’t this clear about this. I suspect those who assign substantial probability to a winner-take-all outcome if takeoff is slow will wish he’d analyzed this in more detail.

I’m less optimistic than Bostrom about monitoring AGI progress. He says “it would not be too difficult to identify most capable individuals with a long-standing interest in [AGI] research”. AGI might require enough expertise for that to be true, but if AGI surprises me by only needing modest new insights, I’m concerned by the precedent of Tim Berners-Lee creating a global hypertext system while barely being noticed by the “leading” researchers in that field. Also, the large number of people who mistakenly think they’ve been making progress on AGI may obscure the competent ones.

He seems confused about the long-term trends in AI researcher beliefs about the risks: “The pioneers of artificial intelligence … mostly did not contemplate the possibility of greater-than-human AI” seems implausible; it’s much more likely they expected it but were either overconfident about it producing good results or fatalistic about preventing bad results (“If we’re lucky, they might decide to keep us as pets” – Marvin Minsky, LIFE Nov 20, 1970).

The best parts of the book clarify many issues related to ensuring that an AGI does what we want.

He catalogs more approaches to controlling AGI than I had previously considered, including tripwires, oracles, and genies, and clearly explains many limits to what they can accomplish.

He briefly mentions the risk that the operator of an oracle AI would misuse it for her personal advantage. Why should we have less concern about the designers of other types of AGI giving them goals that favor the designers?

If an oracle AI can’t produce a result that humans can analyze well enough to decide (without trusting the AI) that it’s safe, why would we expect other approaches (e.g. humans writing the equivalent seed AI directly) to be more feasible?

He covers a wide range of ways we can imagine handling AI goals, including strange ideas such as telling an AGI to use the motivations of superintelligences created by other civilizations

He does a very good job of discussing what values we should and shouldn’t install in an AGI: the best decision theory plus a “do what I mean” dynamic, but not a complete morality.

I’m somewhat concerned by his use of “final goal” without careful explanation. People who anthropomorphise goals are likely to confuse at least the first few references to “final goal” as if it worked like a human goal, i.e. something that the AI might want to modify if it conflicted with other goals.

It’s not clear how much of these chapters depend on a winner-take-all scenario. I get the impression that Bostrom doubts we can do much about the risks associated with scenarios where multiple AGIs become superhuman. This seems strange to me. I want people who write about AGI risks to devote more attention to whether we can influence whether multiple AGIs become a singleton, and how they treat lesser intelligences. Designing AGI to reflect values we want seems almost as desirable in scenarios with multiple AGIs as in the winner-take-all scenario (I’m unsure what Bostrom thinks about that). In a world with many AGIs with unfriendly values, what can humans do to bargain for a habitable niche?

He has a chapter on worlds dominated by whole brain emulations (WBE), probably inspired by Robin Hanson’s writings but with more focus on evaluating risks than on predicting the most probable outcomes. Since it looks like we should still expect an em-dominated world to be replaced at some point by AGI(s) that are designed more cleanly and able to self-improve faster, this isn’t really an alternative to the scenarios discussed in the rest of the book.

He treats starting with “familiar and human-like motivations” (in an augmentation route) as an advantage. Judging from our experience with humans who take over large countries, a human-derived intelligence that conquered the world wouldn’t be safe or friendly, although it would be closer to my goals than a smiley-face maximizer. The main advantage I see in a human-derived superintelligence would be a lower risk of it self-improving fast enough for the frontrunner advantage to be large. But that also means it’s more likely to be eclipsed by a design more amenable to self-improvement.

I’m suspicious of the implication (figure 13) that the risks of WBE will be comparable to AGI risks.

  • Is that mainly due to “neuromorphic AI” risks? Bostrom’s description of neuromorphic AI is vague, but my intuition is that human intelligence isn’t flexible enough to easily get the intelligence part of WBE without getting something moderately close to human behavior.
  • Is the risk of uploaded chimp(s) important? I have some concerns there, but Bostrom doesn’t mention it.
  • How about the risks of competitive pressures driving out human traits (discussed more fully/verbosely at Slate Star Codex)? If WBE and AGI happen close enough together in time that we can plausibly influence which comes first, I don’t expect the time between the two to be long enough for that competition to have large effects.
  • The risk that many humans won’t have enough resources to survive? That’s scary, but wouldn’t cause the astronomical waste of extinction.

Also, I don’t accept his assertion that AGI before WBE eliminates the risks of WBE. Some scenarios with multiple independently designed AGIs forming a weakly coordinated singleton (which I consider more likely than Bostrom does) appear to leave the last two risks in that list unresolved.

This books represents progress toward clear thinking about AGI risks, but much more work still needs to be done.

Book review: Our Mathematical Universe: My Quest for the Ultimate Nature of Reality, by Max Tegmark.

His most important claim is the radical Platonist view that all well-defined mathematical structures exist, therefore most physics is the study of which of those we inhabit. His arguments are more tempting than any others I’ve seen for this view, but I’m left with plenty of doubt.

He points to ways that we can imagine this hypothesis being testable, such as via the fine-tuning of fundamental constants. But he doesn’t provide a good reason to think that those tests will distinguish his hypothesis from other popular approaches, as it’s easy to imagine that we’ll never find situations where they make different predictions.

The most valuable parts of the book involve the claim that the multiverse is spatially infinite. He mostly talks as if that’s likely to be true, but his explanations caused me to lower my probability estimate for that claim.

He gets that infinity by claiming that inflation continues in places for infinite time, and then claiming there are reference frames for which that infinite time is located in a spatial rather than a time direction. I have a vague intuition why that second step might be right (but I’m fairly sure he left something important out of the explanation).

For the infinite time part, I’m stuck with relying on argument from authority, without much evidence that the relevant authorities have much confidence in the claim.

Toward the end of the book he mentions reasons to doubt infinities in physics theories – it’s easy to find examples where we model substances such as air as infinitely divisible, when we know that at some levels of detail atomic theory is more accurate. The eternal inflation theory depends on an infinitely expandable space which we can easily imagine is only an approximation. Plus, when physicists explicitly ask whether the universe will last forever, they don’t seem very confident. I’m also tempted to say that the measure problem (i.e. the absence of a way to say some events are more likely than others if they all happen an infinite number of times) is a reason to doubt infinities, but I don’t have much confidence that reality obeys my desire for it to be comprehensible.

I’m disappointed by his claim that we can get good evidence that we’re not Boltzmann brains. He wants us to test our memories, because if I am a Boltzmann brain I’ll probably have a bunch of absurd memories. But suppose I remember having done that test in the past few minutes. The Boltzmann brain hypothesis suggests it’s much more likely for me to have randomly acquired the memory of having passed the test than for me to actually be have done the test. Maybe there’s a way to turn Tegmark’s argument into something rigorous, but it isn’t obvious.

He gives a surprising argument that the differences between the Everett and Copenhagen interpretations of quantum mechanics don’t matter much, because unrelated reasons involving multiverses lead us to expect results comparable to the Everett interpretation even if the Copenhagen interpretation is correct.

It’s a bit hard to figure out what the book’s target audience is – he hides the few equations he uses in footnotes to make it look easy for laymen to follow, but he also discusses hard concepts such as universes with more than one time dimension with little attempt to prepare laymen for them.

The first few chapters are intended for readers with little knowledge of physics. One theme is a historical trend which he mostly describes as expanding our estimate of how big reality is. But the evidence he provides only tells us that the lower bounds that people give keep increasing. Looking at the upper bound (typically infinity) makes that trend look less interesting.

The book has many interesting digressions such as a description of how to build Douglas Adams’ infinite improbability drive.