Energy-Based Detection of Adverse Weather Effects in LiDAR Data

Autonomous vehicles rely on LiDAR sensors to perceive the environment. Adverse weather conditions like rain, snow, and fog negatively affect these sensors, reducing their reliability by introducing unwanted noise in the measurements.

In this work, we tackle this problem by proposing a novel approach for detecting adverse weather effects in LiDAR data. We reformulate this problem as an outlier detection task and use an energy-based framework to detect outliers in point clouds. More specifically, our method learns to associate low energy scores with inlier points and high energy scores with outliers allowing for robust detection of adverse weather effects.

In extensive experiments, we show that our method performs better in adverse weather detection and has higher robustness to unseen weather effects than previous state-of-the-art methods. Furthermore, we show how our method can be used to perform simultaneous outlier detection and semantic segmentation.

Finally, to help expand the research field of LiDAR perception in adverse weather, we release the SemanticSpray dataset, which contains labeled vehicle spray data in highway-like scenarios.

Given an input point cloud \(\boldsymbol{p}\), we aim to detect if a point \(\boldsymbol{x}_i \in \boldsymbol{p}\) is caused by adverse weather. We reframe the problem as an outlier detection task and use the proposed point energy score to detect outlier points. During training, we minimize the loss function \(\ell_\text{total}\), which results in the model \(f(\boldsymbol{p})\) associating a low energy score with inlier points and a high energy score with outliers. This creates an energy gap between the two categories, which can be used to select a classification threshold \(\tau\). The top-right plot shows an example of the energy gap between inlier and outlier points on the test set of the WADS dataset when training \(f(\boldsymbol{p})\) on snowy conditions (WADS training set). During inference, points are classified as outliers (red points in the bottom-right plot) if their energy score is greater than the threshold \(\tau\).