Lei Huang (黄 雷)

Associate Professor

State Key Laboratory of Software Development Environment

Institute of Artifical Intelligence, Beihang University, Beijing, China


Office: Room B1009, New Main Building, Beihang University, Haidian, Beijing

Email: huanglei36060520@gmail.com


[Research Interests] | [Publication] | [Professional Activities] |[Education] | [Awards and Honors] | [Source Code]| [Miscellaneous] | [CV]

Lei Huang received his BSc and PhD degrees under supervision of Prof. Wei Li, respectively in 2010 and 2018, at the School of Computer Science and Engineering, Beihang University, China. From 2015 to 2016, he visited the Vision and Learning Lab, University of Michigan, Ann Arbor, as a joint PhD student supervised by Prof. Jia Deng. During 2018 to 2020, he was a research scientist in Inception Institute of Artifical Intelligence (IIAI), UAE. His current research mainly focuses on normalization techniques (involving methods, theories and applications) in training DNNs. He also has wide interests in deep learning theory (representation & optimization) and computers vision tasks. He serves as a reviewer for the top conferences and journals such as CVPR, ICML, ICCV, ECCV, NeurIPS, AAAI, JMLR, IJCV, TNNLS, etc.
  • NEWS Sep 2021: One paper is accepted by NeurIPS 2021.
  • NEWS July 2021: Two papers are accepted by ICCV 2021.
  • NEWS March 2021: Two papers are accepted by CVPR 2021.
  • NEWS Feb 2021: I will organize a tutorial named "Normalization Techniques in Deep Learning: Methods, Analyses, and Applications" at CVPR 2021.
  • NEWS Dec 2020: Two papers are accepted by AAAI 2021.
  • NEWS Sep 2020: Our survey on normalization: "Normalization Techniques in Training DNNs: Methodology, Analysis and Application" can be available on arXiv and GitHub .
  • NEWS July 2020: One paper is accepted by ACM Multimedia 2020
  • NEWS July 2020: Two papers (one oral, one poster) are accepted by ECCV 2020
  • NEWS June 2020: One paper is accepted by ICML 2020
  • NEWS Feb 2020: Two papers are accepted by CVPR 2020 (oral)
  • NEWS Jan 2020: One paper is accepted by ECAI 2020
  • NEWS May 2019: I will co-organize The 1st Statistical Deep Learning in Computer Vision workshop in conjunction with ICCV 2019.
  • NEWS March 2019: Two papers are accpeted by CVPR 2019.
  • NEWS September 2018: I co-organized a tutorial named "Normalization Methods for Training Deep Neural Networks: Theory and Practice" at ECCV 2018. The slides can be found here .
  • NEWS Feb 2018: Our paper is accepted by CVPR 2018
  • NEWS November 2017: One paper is accepted by AAAI 2018 (oral)
  • NEWS  Sep 2017: I will atend the Doctoral Symposium at ICIP 2017, on 19 September at China National Convention Center (CNCC), BeiJing. I will present my research about 'Normalization Techniques in Training Deep Neural Networks'
  • NEWS  Jul 2017: One paper accepted by ICCV 2017.
  • Research Interests

    • Machine Learning and Computer Vision: Normalization Techniques and Optimization on Deep Learning, Semi-Supervised Learning, Active Learning, Manifold Learning, Adversarial Examples, Kernel Methods, Image Translation, Human Pose Estimation
    • Multimedia Content Analysis and Image Retrieval: Large Scale Annotation, Hash Based Nearest Neighbor Search

    Selected papers | All paper list

    Rot-Pro: Modeling Transitivity by Projection in Knowledge Graph Embedding.
    Tengwei Song, Jie Luo, Lei Huang.
    NeurIPS 2021
    Towards Defending Multiple Adversarial Perturbations via Gated Batch Normalization
    Aishan Liu, Shiyu Tang, Xianglong Liu, Xinyun Chen, Lei Huang, Zhuozhuo Tu, Dawn Song, Dacheng Tao
    arXiv:2012.01654
    Normalization Techniques in Training DNNs: Methodology, Analysis and Application
    Lei Huang, Jie Qin, Yi Zhou, Fan Zhu, Li Liu, Ling Shao
    arXiv:2009.12836
    Group Whitening: Balancing Learning Efficiency and Representational Capacity
    Lei Huang, Yi Zhou, Li Liu, Fan Zhu, Ling Shao
    CVPR 2021, arXiv:2009.13333
    Layer-wise Conditioning Analysis in Exploring the Learning Dynamics of DNNs
    Lei Huang, Jie Qin, Li Liu, Fan Zhu, Ling Shao
    ECCV 2020 (Oral)
    On the Number of Linear Regions of Convolutional Neural Networks
    Huan Xiong, Lei Huang, Mengyang Yu, Li Liu, Fan Zhu, Ling Shao
    ICML 2020
    (Deep Learning Theory)
    Controllable Orthogonalization in Training DNNs
    Lei Huang, Li Liu, Fan Zhu, Diwen Wan, Zehuan Yuan, Bo Li, Ling Shao
    CVPR 2020 (Oral, Best Paper Nomination )
    An Investigation into the Stochasticity of Batch Whitening
    Lei Huang, Lei Zhao, Yi Zhou, Fan Zhu, Li Liu, Ling Shao
    CVPR 2020 (Oral)
    An Efficient Agreement Mechanism in CapsNets By Pairwise Product
    Lei Zhao, Xiaohui Wang, Lei Huang
    ECAI 2020
    Iterative Normalization: Beyond Standardization towards Efficient Whitening
    Lei Huang, Yi Zhou, Fan Zhu, Li Liu, Ling Shao
    CVPR 2019
    Collaborative Learning of Semi-Supervised Segmentation and Classification for Medical Images
    Yi Zhou, Xiaodong He, Lei Huang, Li Liu, Fan Zhu, Shanshan cui, Ling Shao
    CVPR 2019
    Decorrelated Batch Normalization
    Lei Huang, Dawei Yang, Bo Lang, Jia Deng
    CVPR 2018
    Orthogonal Weight Normalization: Solution to Optimization over Multiple Dependent Stiefel Manifolds in Deep Neural Networks
    Lei Huang, Xianglong Liu, Bo Lang, Admas Wei Yu, Bo Li
    AAAI 2018 (Oral), (arXiv:1709.06079)
    Projection Based Weight Normalization for Deep Neural Networks
    Lei Huang, Xianglong Liu, Bo Lang, Bo Li
    preprint (arXiv:1710.02338)
    Block-normalized Gradient Method: an Empirical Study for Training Deep Nerual Network
    Adams Wei Yu, Lei Huang, Qihang Lin, Ruslan Salakhutdinov, Jaime Carbonell.
    preprint (arXiv:1707.04822)
    Centered Weight Normalization in Accelerating Training of Deep Neural Networks
    Lei Huang, Xianglong Liu, Yang Liu, Bo Lang, Dacheng Tao
    IEEE ICCV, 2017
    DGCNN: Disordered Graph Convolutional Neural Network Based on the Gaussian Mixture Model
    Bo Wu, Yang Liu, Bo Lang, Lei Huang
    Neurocomputing, 2018 (arXiv:1712.03563)
    Query-adaptive hash code ranking for large-scale multi-view visual search
    Xianglong Liu, Lei Huang, Cheng Deng, Bo Lang, Dacheng Tao
    IEEE Transactions on Image Processing (TIP). 2016
    Multi-View Complementary Hash Tables for Nearest Neighbor Search
    Xianglong Liu, Lei Huang, Cheng Deng, Jiwen Lu, Bo Lang
    IEEE ICCV, 2015
    Graph-based Active Semi-supervised Learning: a New Perspective for Relieving Multi-class Annotation Labor
    Lei Huang, Yang Liu, Xianglong Liu, Xindong Wang, Bo Lang
    IEEE ICME, 2014
    Journal extension: A general non-parametric active learning framework for classification on multiple manifolds, Pattern Recognition Letters, 2019
    Efficient Semi-supervised Annotation with Proxy-based Local Consistency Propagation
    Lei Huang, Yang Wang, Xianglong Liu, Bo Lang
    IEEE ICME, 2013
    Journal extension: Online Semi-Supervised Annotation via Proxy-based Local Consistency Propagation, Neurocomputing, 2014

    Talk Slides

    • September 8th, 2018. Normalization Methods for Training Deep Neural Networks: Theory and Practice, ECCV 2018 Tutorial. Munich, Germany. [Slides]

    • August 17th, 2017. Normalization techiniques in deep learning. Multimedia Signal and Intelligent Information Processing Laboratory, Tsinghua University, Beijing. [Slides]

    • November 2016- January 2017. Deep Learning Seminar For graduated students in State Key Laboratory of Software Development Environment, Beihang University, Beijing. [slides1-Introduction] [slides2-MLP] [slides3-CNN] [slides4-RNN]

    • November 1th, 2014. Graph-based active Semi-Supervised Learning: a new perspective for relieving multi-class annotation labor. ICT International Exchange Workshop 2014, Laboratory of Advanced Research B, University of Tsukuba, Japan [Slides]


    Professional Activities

    • Regular Conference Reviewer: NeurIPS2021, ICCV2021, ICML 2021, CVPR 2021, AAAI 2021, WACV 2021, NeurIPS 2020, ACM Multimedia 2020, ECCV2020, IJCAI2020, CVPR2020, AAAI2020, ICCV 2019, CVPR 2019, AAAI 2019, ACM Multimedia 2019

    • Journal Reviewer: JMLR, IJCV, IEEE Transactions on Cybernetics, IEEE Transactions on Neural Networks and Learning Systems, PR.

    • Sub-Reviewer of CVPR 2016, NIPS 2016, IJCAI 2017

    Education

    • Sep. 2015 - Oct. 2016
      • Visiting Ph.D student in Vision & Learning Lab, at the University of Michigan, Ann Arbor
      • Research advisor: Profs. Jia Deng
    • Sep. 2011 - 2018
      • Ph.D student in Computer Science, School of Computer Science and Engineering, Beihang University
      • Advisor: Profs. Wei Li and co-advised by Profs. Bo Lang
    • Sep. 2010 - Sep. 2011
      • Master student in Computer Science, School of Computer Science and Engineering, Beihang University
      • Advisor: Profs. Wei Li and co-advised by Profs. Bo Lang
    • Sep. 2006 - Jun. 2010
      • B.Sc in Computer Science, School of Computer Science and Engineering, Beihang University
      • Thesis advisor: Profs. Wei Li

    Awards and Honors

    • China Scholarship Council (CSC) Scholarship, 2015.

    • The National Scholarship for Ph.D. candidate, 2014.

    • The Guanghua Scholarship, 2013.

    • Outstanding graduate, Beihang University. 2010

    • The Second Class Prize of the 2nd Beihang University Dancing Contest, 2010

    • The Second Class Prize of the 1st Beihang University Dancing Contest, 2008.

    • The Third Class Scholarship for Academic Performance in Beihang University, 2009

    • The Third Class Scholarship for Contest Excellence in Beihang University, 2008

    • The Third Class Prize of the 18th Mathematics Contest for College Students in Beijing, 2007.

    • The Third Class Prize of the 24th National Physics Contest for College Students, 2007.

    • The Second Class Scholarship for Excellent Freshman in Beihang University, 2006.


    Source Code

    • ONI: This project is the implementation of the paper "Controllable Orthogonalization in Training DNNs" (arXiv:2004.00917).

    • StochasticityBW: This project is the implementation of the paper "An Investigation into the Stochasticity of Batch Whitening" (arXiv:2003.12327).

    • IterNorm: This project is the implementation of the paper "Iterative Normalization: Beyond Standardization towards Efficient Whitening" (arXiv:1904.03441).

    • DBN: This project is the Torch implementation of the paper : Decorrelated Batch normalization (arXiv:1804.08450).

    • OWN: This project is the Torch implementation of the paper : orthogonal weight normalization method for solving orthogonality constraints over Steifel manifold in deep neural networks (arXiv:1709.06079).

    • CWN: This project is the Torch implementation of our accepted ICCV 2017 paper: Centered Weight Normalization in Accelerating Training of Deep Neural Networks

    • NormProjection: This project is the Torch implementation of the paper: Projection Based Weight Normalization for Deep Neural Networks (arXiv:1710.02338)

    • Ladder_deepSSL_NP: The reimplementation of Ladder networks with projection based weight normalization. We achieved test errors as 2.52%, 1.06%, and 0.91% on Permunate invariant MNIST dataset with 20, 50, and 100 labeled samples respectively, which is the state-of-the-art results.


    Miscellaneous