SOC-ICNN: From Polyhedral to Conic Geometry for Learning Convex Surrogate Functions
TLDR
SOC-ICNN generalizes Input Convex Neural Networks from LP to SOCP, enhancing representational capacity with smooth curvature while maintaining efficiency.
Key contributions
- Generalizes ICNNs from Linear Programming to Second-Order Cone Programming.
- Introduces native smooth curvature, overcoming piecewise-linear limitations.
- Strictly expands representational capacity without increasing asymptotic complexity.
- Demonstrates improved function approximation and competitive downstream decision quality.
Why it matters
Classical ICNNs are limited to piecewise-linear functions, restricting their utility. SOC-ICNN overcomes this by using SOCP, allowing for smooth, more expressive convex functions. This advancement is crucial for tasks requiring accurate convex surrogates, improving optimization and decision-making.
Original Abstract
Classical ReLU-based Input Convex Neural Networks (ICNNs) are equivalent to the optimal value functions of Linear Programming (LP). This intrinsic structural equivalence restricts their representational capacity to piecewise-linear polyhedral functions. To overcome this representational bottleneck, we propose the SOC-ICNN, an architecture that generalizes the underlying optimization class from LP to Second-Order Cone Programming (SOCP). By explicitly injecting positive semi-definite curvature and Euclidean norm-based conic primitives, our formulation introduces native smooth curvature into the representation while preserving a rigorous optimization-theoretic interpretation. We formally prove that SOC-ICNNs strictly expand the representational space of ReLU-ICNNs without increasing the asymptotic order of forward-pass complexity. Extensive experiments demonstrate that SOC-ICNN substantially improves function approximation, while delivering competitive downstream decision quality. The code is available at https://github.com/Kanyooo/SOC-ICNN.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.