Semi-Supervised Neural Architecture Search
- Renqian Luo (University of Science and Technology of China) email@example.com
- Xu Tan (Microsoft Research) firstname.lastname@example.org
- Rui Wang (Microsoft Research) email@example.com
- Tao Qin (Microsoft Research) firstname.lastname@example.org
- Enhong Chen (University of Science and Technology of China) email@example.com
- Tie-Yan Liu (Microsoft Research) firstname.lastname@example.org
Neural architecture search (NAS) relies on a good controller to generate better architectures or predict the accuracy of given architectures. However, training the controller requires both abundant and high-quality pairs of architectures and their accuracy, while it is costly to evaluate an architecture and obtain its accuracy. In this paper, we propose SemiNAS, a semi-supervised NAS approach that leverages numerous unlabeled architectures (without evaluation and thus nearly no cost) to improve the controller. Specifically, SemiNAS 1) trains an initial controller with a small set of architecture-accuracy data pairs; 2) uses the trained controller to predict the accuracy of large amount of architectures~(without evaluation); and 3) adds the generated data pairs to the original data to further improve the controller. SemiNAS has two advantages: 1) It reduces the computational cost under the same accuracy guarantee. 2) It achieves higher accuracy under the same computational cost. On NASBench-101 benchmark dataset, it discovers a top 0.01% architecture after evaluating roughly 300 architectures, with only 1/7 computational cost compared with regularized evolution and gradient-based methods. On ImageNet, it achieves a state-of-the-art top-1 error rate of 23.5% (under the mobile setting) using 4 GPU-days for search. We further apply it to LJSpeech text to speech task and it achieves 97% intelligibility rate in the low-resource setting and 15% test error rate in the robustness setting, with 9%, 7% improvements over the baseline respectively.
All of the audio samples use Griffin-Lim as vocoder.
*The good it tried to do took active shape in the establishment of temporary refuges - at hoxton for males and in the hackney road for females
*But shall also tend more effectually to preserve the health
*Hired a room for the night and morning which he and a large party of friends occupied before and during the execution
*Every link in that great human chain is shaken along the whole lengthened line has the motion jarred and each in turn sees
*This phial he had managed to retain in his possession in spite of the frequent searches to which he was subjected in newgate
*No time was lost in carrying out the dread ceremony but it was not completed without some of the officials turning sick and the moment it was over
*No other employee has been found who saw oswald enter that morning
*Three months prior to his regularly scheduled separation date ostensibly to care for his mother who had been injured in an accident at her work
*Oswald’s activities with regard to cuba raise serious questions as to how much he might have been motivated in the assassination
*Lawson also in the lead car did not scan any buildings since an important part of his job was to look backward at the president’s car
*Christmas is coming.
*Nineteen twenty is when we are unique together until we realise we are all the same.
Our Related Works
Neural Architecture Optimization
FastSpeech: Fast, Robust and Controllable Text to Speech
FastSpeech 2: Fast and High-Quality End-to-End Text to Speech
LRSpeech: Extremely Low-Resource Speech Synthesis and Recognition
Almost Unsupervised Text to Speech and Automatic Speech Recognition