Exact ReLU realization of tensor-product refinement iterates
TLDR
This paper shows that 2D tensor-product refinement iterates can be exactly realized by ReLU networks with fixed width and depth O(n).
Key contributions
- Presents the first 2D extension of exact ReLU realization theory for refinement cascades.
- Shows that iterates of 2D tensor-product dyadic refinement operators have exact ReLU realizations.
- Demonstrates these ReLU networks have fixed width and depth proportional to the iteration number O(n).
- Leverages a 1D loop-controller framework, transporting residual dynamics to products of polygonal loops.
Why it matters
This work is the first to extend exact ReLU realization theory for refinement cascades to 2D, a significant advance for neural network-based signal processing. It provides a concrete method for precisely realizing these complex operators, opening new possibilities for designing accurate neural networks.
Original Abstract
We study scalar dyadic refinement operators on R^2 of the form (Vf)(x,y) = sum_{(j,k) in Z^2} c_{j,k} f(2x-j, 2y-k), where only finitely many mask coefficients c_{j,k} are nonzero. Under a fixed support-window hypothesis, we prove that for every compactly supported continuous piecewise linear seed g:R^2->R, the iterates V^n g admit exact ReLU realizations of fixed width and depth O(n). This gives a first genuinely two-dimensional extension of the exact realization theory for refinement cascades. Using the one-dimensional exact loop-controller framework, the proof transports the tensor-product residual dynamics exactly on the product of two polygonal loops and reduces the remaining seam ambiguity to a final readout and selector step. The matrix cascade is then handled by a fixed-depth recursive block, and general compactly supported continuous piecewise linear seeds are reduced to a finite decomposition together with exact clamped gluing on the support window. This identifies the tensor-product dyadic case as a natural first multivariate instance of the loop-controller method for refinement iterates.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.