Eyes-Closed Safety Kernels: Safety of Autonomous Systems Under Loss of Observability

Forrest Laine (UC Berkeley); Chih-Yuan Chiu (UC Berkeley); Claire Tomlin (UC Berkeley)


A framework is presented for handling a potential loss of observability of a dynamical system in a provably safe way. Inspired by the fragility of data-driven perception systems used by autonomous vehicles, we formulate the problem that arises when a sensing modality fails or is found to be untrustworthy during autonomous operation. We cast this problem as a differen- tial game played between the dynamical system being controlled and the external system factor(s) for which observations are lost. The game is a zero-sum Stackelberg game in which the controlled system (leader) is trying to find a trajectory which maximizes a function representing the safety of the system, and the unobserved factor (follower) is trying to minimize the same function. The set of winning initial configurations of this game for the controlled system represents the set of all states in which safety can be maintained with respect to the external factor, even if observability of that factor is lost. This is the set we refer to as the Eyes-Closed Safety Kernel. In practical use, the policy defined by the winning strategy of the controlled system is only needed to be executed whenever observability of the external system is lost or the system deviates from the Eyes-Closed Safety Kernel due to other, non-safety oriented control schemes. We present a means for solving this game offline, such that the resulting winning strategy can be used for computationally efficient, provably-safe, online control when needed. The solution approach presented is based on representing the game using the solutions of two Hamilton-Jacobi partial differential equations. We illustrate the applicability of our framework by working through a realistic example in which an autonomous car must avoid a dynamic obstacle despite potentially losing observability.

Live Paper Discussion Information

Start Time End Time
07/16 15:00 UTC 07/16 17:00 UTC

Virtual Conference Presentation

Paper Reviews

Review 2

I have several concerns focused around wording, implementation, and references to prior work: 1. The approach proposed in the paper is implemented using the Level Set Toolbox. Does the approximation generated by the Level Set Toolbox generate a conservative approximation to the true 1-super level set? If the approximations generated in the paper are conservative, the authors should mention that property. If not, then I am not sure why the the proposed method deals with the potential loss of observability in a provably safe manner. A clarification should be made in the revision to ensure that readers appreciate the significance of the result. 2. There are others who've proposed to plan while treating potentially observed or only partially observed models using forward and then backwards reachability. Though these methods do not treat the problem as a zero-sum, differential game (as is done in this paper), they are probably worth referencing as motivation for this approach. For instance: Ahn, Heejin, Karl Berntorp, and Stefano Di Cairano. "Reachability-based Decision Making for City Driving." 2018 Annual American Control Conference (ACC). IEEE, 2018 3. Have the authors considered how to apply this to real-world or specifically more realistic vehicle models? Some discussion of that is probably merited given the difficulty of applying some of these reachability based approaches. Again, to appreciate the significance of the approach, such a discussion would be helpful.

Review 3

The paper is overall well-written and easy to follow. Minor comments: 1- It seems that you should use "Lemma" instead of "Theorem" for Theorem1 &2, as they are direct results of already published work. 2- Please give more details and the limitations regarding your assumption: "the external system can observe the internal system at all times". In the case of two autonomous cars, it is possible that both systems cannot observe each other. 3- In Section III, as you are only considering the 2D case, the cars are rather disks and not spheres. 4- The analysis in Section III. A. "Notes on Computation" is interesting, can you provide more details regarding your statement: "Nevertheless, with clever representation of the systems, many interesting problems more complicated than the one presented here can be solved using this framework."