Random Cloud: Finding Minimal Neural Architectures Without Training
TLDR
Random Cloud is a training-free method for neural architecture search that finds minimal network topologies via stochastic exploration and structural reduction.
Key contributions
- Introduces Random Cloud, a training-free neural architecture search method.
- Discovers minimal network topologies via stochastic exploration and progressive reduction.
- Outperforms pruning baselines on 6/7 datasets, achieving 87% parameter reduction.
- Significantly faster, avoiding full network training (0.67-0.94x cost of full training).
Why it matters
This paper introduces a novel, efficient way to find compact neural networks. By avoiding full training cycles, it significantly reduces computational cost and time. This approach could accelerate the development and deployment of efficient AI models.
Original Abstract
I propose the \emph{Random Cloud} method, a training-free approach to neural architecture search that discovers minimal feedforward network topologies through stochastic exploration and progressive structural reduction. Unlike post-training pruning methods that require a full train-prune-retrain cycle, this method evaluates randomly initialized networks without backpropagation, progressively reduces their topology, and only trains the best minimal candidate at the end. I evaluate on 7 classification benchmarks against magnitude pruning and random pruning baselines. The Random Cloud matches or outperforms both baselines in 6 of 7 datasets, achieving statistically significant improvements on Sonar ($+4.9$pp accuracy, $p{=}0.017$ vs magnitude pruning) with 87\% parameter reduction. Crucially, the method is faster than both pruning baselines in 4 of 5 datasets (0.67--0.94$\times$ the cost of full training), since it avoids training the full-size network entirely.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.