Box Embeddings: A Fresh Look at Geometric Learning

Geometric embeddings have been in the spotlight lately for their natural ability to represent relationships using containment. One popular method is box embeddings, where objects are depicted as n-dimensional hyperrectangles. These boxes are special because they can be intersected easily and their volume can be calculated with a snap. This makes them perfect for representing probability distributions.

But there's a catch. This method also brings up a problem called local identifiability. This means that small changes in parameters can lead to the same results, making it tough for models to learn effectively. Previous solutions used an approximation of Gaussian convolution over the box parameters, but this increased the sparsity of the gradient. In this work, the researchers decided to model the box parameters using min and max Gumbel distributions. Why? Because these distributions keep the space closed under intersection. Calculating the expected intersection volume with these parameters involves all of them, and experiments show that this approach greatly improves learning.

Actions