We propose to derive deviation measures through the Minkowski gauge of a given set of acceptable positions. We show that, given a suitable acceptance set, any positive homogeneous deviation measure can be accommodated in our framework. In doing so, we provide a new interpretation for such measures, namely, that they quantify how much one must shrink or deleverage a financial position for it to become acceptable. In particular, the Minkowski Deviation of a set which is convex, translation insensitive, and radially bounded at non-constants, is a generalized deviation measure in the sense of [R. T. Rockafellar, S. Uryasev and M. Zabarankin, Generalized deviations in risk analysis, Finance Stoch. 10 2006, 1, 51–74]. Furthermore, we explore the converse relations from properties of a Minkowski Deviation to its sub-level sets, introducing the notion of acceptance sets for deviations. Hence, we fill a gap existing in the literature, namely the lack of a well-defined concept of acceptance sets for deviation measures. Dual characterizations in terms of polar sets and support functionals are provided.