Abstract: Point clouds are a fundamental representation for robotic perception tasks such as localization, mapping, and object pose estimation. However, LiDAR-acquired point clouds are inherently sparse and non-uniform, providing incomplete observations of the underlying geometry. Such sparsity and non-uniformity hinder reliable geometric reasoning, leading to degraded performance in downstream perception tasks. To mitigate these issues, prior work has attempted to compensate for the sparsity and non-uniformity of point clouds by estimating point cloud geometry. However, in the absence of an explicit model of point cloud geometry, existing approaches have predominantly relied on either hand-crafted statistics of local point distributions or end-to-end supervised deep learning, which often suffer from limited scalability or require large amounts of accurately labeled training data. To address these challenges, we explicitly model and estimate point cloud geometry under a principled mathematical formulation. Theoretically, we represent the point cloud geometry as a statistical manifold induced by a family of Gaussian distributions that captures the local geometry of each point. Building on this formulation, we design a probabilistic model that predicts per-point local geometry in the form of a Gaussian distribution. Practically, we introduce a deep neural network to instantiate the estimation of these Gaussian distributions, and term the resulting estimator as Point-to-Ellipsoid (POLI). By consistently estimating point-wise local geometry across diverse point clouds, POLI learns a mapping between point cloud observations and the statistical manifolds that represent their underlying geometry. Importantly, this mapping is learned in a self-supervised manner, removing the reliance on labeled data while maintaining strong geometric inductive biases. The resulting representation integrates seamlessly into existing robotic perception pipelines without requiring architectural modifications. Extensive experiments demonstrate that the proposed theory and practice enable accurate and robust estimation of point cloud geometry and consistently improve performance across a wide range of robotic perception tasks.