3 comments on “Ontological Crises

  1. I am unclear on some details of your scenario:

    – Did the lawgiver (programmer) first program the AI (or, finally lose control of it) at a time when humans had already expanded to another galaxy? (It’s not realistic that such a programmer would fail to address relativity, but I’ll ignore that.)

    – When you say that your scenario should assume

    A version [of the “person-affecting view”] which says only persons who exist are morally relevant, and “exist” only refers to the present time

    do you mean the (constant) present time when the programmer made this definition and/or lost control, or the (varying) present time at which the AI is applying this law?

    (In the latter case, even in a Newtonian framework, the programmer has set things up so that human lives keep coming into and going out of moral relevance (when they are created or destroyed), but I guess they consider that a feature. I guess you mean that, since it makes your scenario difficult in simpler cases than if you didn’t mean that.)

    ==

    If the definition of “what is morally relevant” can change with the velocity of some object (some instance of an AI, or some part of it doing some thinking), then that object ought to never change its velocity, since bad effects will come from perhaps-intentional changes to what it considers morally relevant.

    (Even worse, what if the AI uses a distributed thinking process, and different parts of that process have different velocities?)

    So any ethics in which a velocity change of the thinker would lead to a change in what it considers morally relevant, is at best a very bad idea, and at worse nonsensical.

    The concept of “what’s in my future lightcone” doesn’t change with my velocity. It is also roughly equivalent to “what my present or future actions might possibly affect”, which is a natural limit on what it might be useful for me to consider morally relevant.

    The concept of “what already exists” does change with my velocity, in a relativistic view, but not in a Newtonian view. So *we* might agree it’s nonsensical to use it in defining ethics, but your hypothetical programmer would not have thought that, and thus might have used that concept.

    If they did use that concept, but if the AI interprets what they “said” with the knowledge that they had a Newtonian view (which seems inevitable, if what they “said” is expressed in some ontology that is based on that view), then clearly what they meant by “what already exists” should be interpreted in a Newtonian way (and in whichever frame of reference that original lawgiver would have considered as “the absolute frame of reference”).

    (That removes some of the problem, but not all, eg if spacetime is significantly not flat, or if new observations suggest that what the original lawgiver would have considered “the absolute frame of reference” (when he made that law) to be different than he actually thought then. And guessing what frame of reference he originally treated as absolute might be itself ambiguous.)

    This means I disagree with you that “Option REF seems to be what the AI would choose if it were just trying to find the criteria that most resembled the criteria that the programmer intended.” I think option UTC is that.

    Your statement that you are “confused as to how difficult it is to find a reference frame that doesn’t yield weird results. E.g. I can imagine that many reference frames eventually enter black holes, …” doesn’t make complete sense to me — a reference frame is a not “worldline”, so I’m not sure what it means for it to “enter a black hole”. I think any reference frame (in special relativity or in a Newtonian view) is a mapping from *all* points of spacetime (or at least, as many of them as it is able to make sense for) into a tuple of space and time coordinates. So black holes are more likely to be localized defects in a reference frame, than something encompassing its entire future. A reference frame’s overall extent “in the present” would only be limited by large-scale curvature of space.

    ==

    A lot depends on how this original law (given to the AI) was expressed, and on the more general question of how the AI is supposed to interpret an expressed law. (I realize that your main motivation for raising this scenario might be to help shed light on those issues.)

    A more general question: if the AI stops understanding how to apply its law correctly (which might happen if the universe turns out to have a completely unanticipated structure — this scenario is a special case), should it continue to exert its full power towards applying some law which seems like a close approximation, or should it just stop exerting its power towards any goal? (Or, does it have, from the start, a succession of more and more general/vague laws, so that it can continue to use whichever most specific law still makes sense to it?)

    ==

    You say you “don’t approve of the person-affecting view”. It seems to me your thoughts on this depend on a lot on what you imagine motivates that view. Probably you need to express those more clearly — or perhaps more usefully, express some related view that you *do* approve of (and therefore, we can hope, understand better).

    If your goal is specifically a case study of “how should an AI reinterpret a nonsensical law”, then to decide what you want it to do, you at least have to understand (hypothetically in your scenario) the worldview and motivations of the lawgiver.

    ==

    I think that any expression of an ethical law L (intended to guide behavior), within an ontology which sees the world as having type X, might as well take this general form:

    “the world is roughly like X; find a world like X which explains your sensory inputs and which you think is affected by your actions; then, within that world, use law L (which is expressed in terms of that worldtype X) to guide your actions.”

    (By “find a world like X” I really mean “find a way of interpreting all your inputs and actions as if they came from and affected a world like X”.)

    In that framework, your problem reduces to how we express L and X, and how we find (and keep re-finding, as we learn more) a world like X in the “real world” (the world affecting the AI’s inputs and affected by its actions).

    It seems to me your problem is mainly concerned with how to find a simpler world as a point of view in which to see and affect a more complex one. So the original lawgiver has to decide on some general strategy for doing that, or at least some criterion by which to judge such strategies. That will be needed for far more mundane issues than this one, since every L we normally express uses high-level concepts (eg “humans”) which need to be “found” in much lower-level sensory input and actions.

  2. >- Did the lawgiver (programmer) first program the AI (or, finally lose control of it) at a time when humans had already expanded to another galaxy?

    No, I’m assuming the AI is programmed in our near future, and becomes powerful for a long time.

    > the (varying) present time at which the AI is applying this law?

    Yes.

    >This means I disagree with you that “Option REF seems to be what the AI would choose if it were just trying to find the criteria that most resembled the criteria that the programmer intended.” I think option UTC is that.

    I guess I shouldn’t be surprised at disagreement, given that part of my point is that the programmer was confused in a way that I’m uncertain how to fix.

    >a reference frame is a not “worldline”, so I’m not sure what it means for it to “enter a black hole”. I think any reference frame (in special relativity or in a Newtonian view) is a mapping from *all* points of spacetime (or at least, as many of them as it is able to make sense for) into a tuple of space and time coordinates. So black holes are more likely to be localized defects in a reference frame, than something encompassing its entire future. A reference frame’s overall extent “in the present” would only be limited by large-scale curvature of space.

    I meant a standard that doesn’t try to force spacetime to be flat. But I’m unsure how to express precisely what I mean, so maybe I’m confused.

    >A lot depends on how this original law (given to the AI) was expressed, and on the more general question of how the AI is supposed to interpret an expressed law. (I realize that your main motivation for raising this scenario might be to help shed light on those issues.)

    Exactly.

    >You say you “don’t approve of the person-affecting view”. It seems to me your thoughts on this depend on a lot on what you imagine motivates that view. Probably you need to express those more clearly — or perhaps more usefully, express some related view that you *do* approve of

    I want to say that it’s generally better for a person to exist than to not exist; advocates of the person-affecting view want avoid that rule.
    Some advocates of the person-affecting view also seem to want to avoid needing to account for potentially astronomical numbers of distant future people. I treat them as morally relevant (probably with some time-discounting). Some of this is related to disagreements about the Repugnant Conclusion.

  3. Pingback: Rational Feed – deluks917

Comments are closed.