Safety with Agency: Human-Centered Safety Filter with Application to AI-Assisted Motorsports
Journal:
arXiv
Published Date:
Apr 16, 2025
Abstract
We propose a human-centered safety filter (HCSF) for shared autonomy that
significantly enhances system safety without compromising human agency. Our
HCSF is built on a neural safety value function, which we first learn scalably
through black-box interactions and then use at deployment to enforce a novel
state-action control barrier function (Q-CBF) safety constraint. Since this
Q-CBF safety filter does not require any knowledge of the system dynamics for
both synthesis and runtime safety monitoring and intervention, our method
applies readily to complex, black-box shared autonomy systems. Notably, our
HCSF's CBF-based interventions modify the human's actions minimally and
smoothly, avoiding the abrupt, last-moment corrections delivered by many
conventional safety filters. We validate our approach in a comprehensive
in-person user study using Assetto Corsa-a high-fidelity car racing simulator
with black-box dynamics-to assess robustness in "driving on the edge"
scenarios. We compare both trajectory data and drivers' perceptions of our HCSF
assistance against unassisted driving and a conventional safety filter.
Experimental results show that 1) compared to having no assistance, our HCSF
improves both safety and user satisfaction without compromising human agency or
comfort, and 2) relative to a conventional safety filter, our proposed HCSF
boosts human agency, comfort, and satisfaction while maintaining robustness.
