Nilotpal Sinha

I am working as a post-doctoral researcher at Computer Vision, Imaging & Machine Intelligence Research Group (CVI2 Lab) in University of Luxembourg. Currently, I am working on designing algorithms for automatically creating AI models (i.e. deep neural network architecture) for edge devices (i.e. devices with low power comsumption such as raspberry pi, NVIDIA Jetson Nano etc.) in space applications.

I finished my PhD in Computer Science from National Yang Ming Chiao Tung University in 2022. I was a part of the COVIS Lab, advised by Prof. Kuan-Wen Chen (陳冠文) during my doctoral studies. My doctoral thesis mainly dealt with designing algorithms that can automatically design the neural networks architecture that gives good performance for a given task. Normally, such algorithms takes long time to give the result and my works was to design such algorithms with reduced search time cost.

I am fueled by my passion for learning new things which has led me to my simple philosophy. "Have I gained any new knowledge today as compared to yesterday?". It has helped me to learn new skills (like programming, writing research papers, hacking things) and pursue seemingly daunting goals like reading 100 books, pursuing PhD and trying to understand quantum mechanics.

Research Interests

Artificial Intelligence, AutoML, Machine Learning, Evolutionary Computation, Neural Architecture Search, Computer Vision, Signal Processing.

Programming Languages

Python, Matlab, C++, C, Assembly language, Machine learning frameworks: pytorch, tensorflow, HTML, Javascript

Language

English, Hindi, Bengali, Bishnupriya Manipuri

Education

  • PhD. in Computer Science (2017 - 2022)

    National Yang Ming Chiao Tung University, Taiwan

     ↪ My doctoral thesis mainly dealt with designing algorithms that can automatically design the architecture of neural networks that gives good performance for a given task. Such algorithms are called Neural Architecture Search (NAS).
     ↪ In general, NAS algorithms are computationally expensive and require hugh amount of time to perform the architecture search.
     ↪ My thesis focued on designing NAS algorithms that require lower computational resources and can perform the architecture search in less time.
     ↪ I used genetic algorithm for searching the Neural Architecture Search Space.
     ↪ For the Neural Architecture evaluation, I used the supernet (or One Shot Model) which reduced the search time from 3000 GPU hours to 10 GPU hours.
     ↪ For more details on the published papers, please check the link.

  • Masters in Electrical Engineering (2015 - 2017)

    National Cheng Kung University, Taiwan

     ↪ In my master's thesis, I designed a signal processing algorithm for processing the 3D heart pulse.
     ↪ The main objective was to classify different types of heart pulse.
     ↪ It was done using the multi-dimensional fourier analysis. For more information, please checkout the paper

  • Bachelor in Electrical Engineering (2009 - 2013)

    National Institute of Technology, Silchar, India

Awards

  • Recipient of 2017 Mediatek Inc. ASEAN South Asia “Taiwan Advanced Study Fellowship”. (link)
  • Recipient of Scholarship for Outstanding New Student Award, National Yang Ming Chiao Tung University, Hsinchu City, Taiwan
  • Recipient of NCKU Distinguished International Student Scholarship, National Cheng Kung University, Tainan, Taiwan

Activities

Work Experience

  • Postdoctoral Researcher (November 2022 - Present )

    CVI2 Lab, University of Luxembourg

     ↪ Develop algorithms for automatically designing Neural Network Architecture (NAS) for edge devices (i.e. low power consuming devices like raspberry pi etc.) for space applications.
     ↪ Published in WACV 2024 and CVPR Workshop 2024. Here, we propose a hardware aware algorithm that performs the neural architecture search for a given task, a specific edge device and the specific hardware cost of that edge device.
    An example application: find the best performing neural network architecture for the classification task on NVIDIA Jetson Nano under the constraint that latency of the architecture has to be less than 2 seconds
     ↪ The effectiveness of the proposed method was shown experimentally on different edge devices. For more details, please checkout the paper or the video.
     ↪ Mentor PhD student in his thesis. This resulted in a full-length paper submission to ECCV 2024 (Under review). The paper deals with pruning channels in a given network (ResNet) using Information Bottleneck Theory.
     ↪ Teaching course on Computer Vison and implementing AI algorithms on edge devices like raspberry pi.

  • Software Engineer Intern (January 2022 - June 2022)

    Microsoft, Taiwan

     ↪ Develop algorithm that automatically designs the deep neural network architectures for multiple tasks.
     ↪ The objective was to find a neural network architecture that gives good performance in multiple tasks (i.e. more than 1 task)

  • Research Assistant in Luo Lab (February 2017 - May 2017)

    National Cheng Kung University, Taiwan

     ↪ Developing a mathematical model for analysing the 3D heart pulse resulting in the paper.

Publications

EdgeAI related papers

1. Nilotpal Sinha, Peyman Rostami, Abd El Rahman Shabayek, Anis Kacem, Djamila Aouada, “Multi-Objective Hardware Aware Neural Architecture Search using Hardware Cost Diversity”, IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshop 2024. Workshop Name: Efficient Deep Learning for Computer Vision. [arXiv]
2. Nilotpal Sinha, Abd El Rahman Shabayek, Anis Kacem, Peyman Rostami, Carl Shneider, Djamila Aouada, “Hardware Aware Evolutionary Neural Architecture Search using Representation Similarity Metric”, IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) 2024. [paper] [Video Link]

Neural Architecture Search related papers

1. Nilotpal Sinha, Kuan-Wen Chen, “Neural Architecture Search using Covariance Matrix Adaptation Evolution Strategy”. Evolutionary Computation (MIT Press) 2023 (Impact Factor: 6.8). [paper] [arXiv] [code]
2. Nilotpal Sinha, Kuan-Wen Chen, “Novelty Driven Evolutionary Neural Architecture Search", ACM SIGEVO Genetic and Evolutionary Computation Conference (GECCO) 2022. [paper] [arXiv] [code] [Video Link]
3. Nilotpal Sinha, Kuan-Wen Chen, “Neural Architecture Search using Progressive Evolution", ACM SIGEVO Genetic and Evolutionary Computation Conference (GECCO) 2022. [paper] [arXiv] [code] [Video Link]
4. Nilotpal Sinha, Kuan-Wen Chen, “Evolving Neural Architecture Using One Shot Model", ACM SIGEVO Genetic and Evolutionary Computation Conference (GECCO) 2021. [paper] [code] [Video Link]

Bio-Medical Signal Processing related paper

1. Bo Peng, Ching-Hsing Luo, Nilotpal Sinha, Cheng-Chi Tai, Xiaohua Xie and Haiqing Xie, “Fourier Series Analysis for Novel Spatiotemporal Pulse Waves: Normal, Taut, and Slippery Pulse Images”, Evidence-based Complementary and Alternative Medicine, Volume 2019 (Impact Factor: 2.06). [paper]

Signal Processing related papers

1. S K Singh, D Kumari, Nilotpal Sinha, A K Goswami, N Sinha “Gravity Search Algorithm hybridized Recursive Least Square method for power system harmonic estimation”, Engineering Science and Technology, an International Journal (Elsevier), DOI- 10.1016/j.jestch.2017.01.006, 2017. (Impact Factor: 5.7). [paper]
2. S. K. Singh, Nilotpal Sinha, A. K. Goswami, N. Sinha, “Robust Estimation of Power System Harmonics Using a Hybrid Firefly Based Recursive Least Square Algorithm”, (SCI Journal), International Journal of Electrical Power and Energy Systems (IJEPES) (Elsevier) Journal, Elsevier, vol. 80, Issue C, Pages 287 – 296, 2016. (Impact Factor: 5.2) [paper]
3. S. K. Singh, Nilotpal Sinha, A. K. Goswami, N. Sinha, “Several Variants of Kalman Filter Algorithm for Power System Harmonic Estimation” (SCI Journal), International Journal of Electrical Power and Energy Systems (IJEPES) (Elsevier), Vol. 78, issue C, Pages 793 - 800, 2016. (Impact Factor: 5.2) [paper]
4. S. K. Singh, Nilotpal Sinha, A. K. Goswami, N. Sinha, “Optimal Estimation of Power System Harmonics Using a Hybrid Firefly Algorithm Based Least Square Method” (SCI Journal), Soft Computing, Springer Journal, pages 1-14, 2015. (Impact Factor: 4.1) [paper]
5. S. K. Singh, Nilotpal Sinha, A. K. Goswami, N. Sinha, “Variable Constraint Based Least Mean Square Algorithm for Power System Harmonic Parameter Estimation” (SCI Journal), International Journal of Electrical Power and Energy Systems (IJEPES) (Elsevier), vol. 73, Pages 218-228, 2015. (Impact Factor: 5.2) [paper]

Projects

Neural Architecture Search

  • The project deals with designing algorithms for automatically searching for the deep neural network architectures for image classification. All the algorithms were implemented using python and deep learning framework PyTorch. Following are the papers on the algorithms designed in the project:
  • Evolving Neural Architecture Using One Shot Model, ACM SIGEVO GECCO 2021.
  • Progressive Evolutionary Neural Architecture Search, ACM SIGEVO GECCO 2022.
  • Novelty Driven Evolutionary Neural Architecture Search, ACM SIGEVO GECCO 2022.
  • Neural Architecture Search using Covariance Matrix Adaptation Evolution Strategy, Evolutionary Computation (MIT Press) 2023.

Drone Go

  • This project deals with controlling the drone (Parrot AR Drone 2.0) to follow an object of particular color using the "Median Flow" tracker from OpenCV.
  • A PID controller is designed for controlling the movements of the drone.
  • A communication architecture was also designed to allow the drone to connect to Facebook and all the speakers present in the local network. Facebook is used to check the status of the drone and to stop the drone flight. The speakers starts an alarm when the drone is in pursuit of the object.
  • The majority of the project was written in C++ with communication to Facebook and speakers in the local network written in python.

Music EA

  • The project is about generating music using evolutionary algorithm. The whole project was implemented using python.
  • The project tries to model the input music file (in midi file format) using three different models i.e. hidden markov model (HMM), autoregressive linear model (ALM) and recurrent neural network (RNN).
  • The parameters of all the models were optimized using genitic algorithm. The optimized models are then used to generate music similar to the input music.

Chess AI

  • In this project, I try to explore various ways of creating an AI agent for playing chess. Here, four different AI agents were created with the goal of beating me in the game of chess.
  • The AI agents were created using (i) pure classical method i.e. Alpha-beta algorithm, (ii) pure deep learning method i.e. reinforcement learning agent using Monte Carlo Tree Search and (iii) hybrid of classical and deep learning methods.
  • Both deep learning frameworks i.e. PyTorch and Keras with tensorflow backend was used in this project.

Journey of Reading 100 Books

A quest that started on March 2019 has reached its first milestone: “reading 100 books”. It started with a simple goal: “Do I know little bit more than what I knew yesterday?” This journey introduced me to topics that were missing from my traditional school education...

Get in touch

Goals

Reading 100 books (✔)

The quest of pursuing knowledge led me to my first pet project, which started on March 2019. It took me 3 years to finish this goal and was very intrumental in forming my present psyche. One of the key insights that I got during this period was that "It is just as difficult to unlearn something as it is to learn something." Read More

Try to learn something new everyday (Life long goal)

  • In the pursuit of knowledge, I came across something very fundamental about the world around us. The information content of the world is unimaginatively large and it will take a life long effort to get aquainted with it. So, it became my life long goal to increase my knowledge every single day.

  • Following this philosophy, my long term goal is to finish reading 1000 books in the next 20 years.
  • Checkout my Goodreads profile to see what I am currently reading.

Living a healthy lifestyle (Life long goal)

  • With growing age, I have grown to appreciate the benefits of exercise, and healthy eating. I have also experimented with intermittent fasting with positive results.

  • This has forced me to create a habbit of going to gym atleast 3 times a week.

Working knowledge of Quantum Computing(✔)

  • It started as a pet project during the summer break of 2020 with the goal of understanding the weirdness of the quantum mechanics and how can we leverage it for computation i.e. "Quantum Computation".

  • My aim is to use this knowledge as a base to launch into another interesting field of combining quantum computation with artificial intelligence.

  • This led me to aquire Bronze diploma and Silver diploma from QWorld. This gave me experience with Qiskit, an open-source SDK for working with quantum computers.

Working knowledge of Blockchain(✔)

This was a fun endeavor to understand the emerging technology of blockchain. I got interested in this topic after I started reading about investment. Though my primary focus was investment using blockchain, I now appreciate the technology after understanding the basics of the blockchain technology. I wonder if the technology will be useful in fields other than replacing the fiat currencies.

Habit of not making a habit (Life long goal)

Humans like to create a habit out of anything possible as it helps us to arrive at a solution to known problem much faster. The solutions might not be optimal and so we should always try to update our habits. But as we grow old, updating a single habit becomes costly in terms of both mental and physical aspects. Since we are designed to create habits, my hope is to try to create a habit of not making a habit.