An Embedding Framework for Consistent Polyhedral Surrogates

Published in Neural Information Processing Systems (NeurIPS) 2019, 2019

Recommended citation: Jessica Finocchiaro, Rafael Frongillo, and Bo Waggoner. (2019). "An Embedding Framework for Consistent Polyhedral Surrogates" https://arxiv.org/abs/1907.07330

Abstract: We formalize and study the natural approach of designing convex surrogate loss functions via embeddings for problems such as classification or ranking. In this approach, one embeds each of the finitely many predictions (e.g. classes) as a point in Rd, assigns the original loss values to these points, and convexifies the loss in between to obtain a surrogate. We prove that this approach is equivalent, in a strong sense, to working with polyhedral (piecewise linear convex) losses. Moreover, given any polyhedral loss L, we give a construction of a link function through which L is a consistent surrogate for the loss it embeds. We go on to illustrate the power of this embedding framework with succinct proofs of consistency or inconsistency of various polyhedral surrogates in the literature.

Download paper here