Seminar: Sparse Training in Supervised and Unsupervised Deep Learning

The talk starts with a quick overview of Prof. Mocanu's research line. Next, he will introduce along this line one of the many challenges (the dense connectivity) which prevents them from having scalable artificial neural networks at both levels, cloud and edge computing. An emerging state-of-the-art possible solution is presented, i.e., sparse-to-sparse training with dynamic sparsity, which reduces the number of connections up to quadratically while persistently improving the performance.

The discussion starts from the first sparse training works on complex Boltzmann machines [Mocanu et al., Machine Learning 2016]  and sparse evolutionary training [Mocanu et al., Nature Communications 2018] in the typical single task (un)supervised learning. Further, it gradually introduces newer approaches in the more challenging contexts of continual or federated learning. Besides the fundamental theoretical novelty, some practical aspects, such as truly sparse implementations and deep learning energy efficiency are also considered and open for discussion.

Decebal Mocanu is Associate Professor in Machine Learning within the Department of Computer Science at the University of Luxembourg; a Guest Faculty Member within the M&CS department at the Eindhoven University of Technology (TU/e); and an alumni member of TU/e Young Academy of Engineering. In 2017, Decebal received his PhD degree from TU/e. During his doctoral studies and after that, Decebal undertook four research visits at the University of Pennsylvania (2014), Julius Maximilians University of Wurzburg (2015), the University of Texas at Austin (2016), and the University of Alberta (2022).

In the long term, Decebal is interested in studying the synergy between artificial intelligence, neuroscience, and network science for the benefits of science and society.

 
About the EuroCC-2 project

EuroCC 2 will work to identify and address the skills gaps in the European High Performance Computing (HPC) ecosystem and coordinate cooperation across Europe to ensure a consistent skills base.

The role of EuroCC 2 is to establish and run a network of more than 30 NCCs across the EuroHPC Participating States. The NCCs act as single points of access in each country between stakeholders and national and EuroHPC systems. They operate on a regional and national level to liaise with local communities, in particular SMEs, map HPC competencies and facilitate access to European HPC resources for users from the private and public sector.

EuroCC 2 delivers training, interacts with industry, develops competence mapping and communication materials and activities, and supports the adoption of HPC services in other related fields, such as quantum computing, artificial intelligence (AI), high performance data analytics (HPDA) to expand the HPC user base.

This project has received funding from the European High-Performance Computing Joint Undertaking (JU) under grant agreement No 101101903. The JU receives support from the European Union’s programme Digital Europe and Germany, Bulgaria, Austria, Croatia, Cyprus, Czech Republic, Denmark, Estonia, Finland, Greece, Hungary, Ireland, Italy, Lithuania, Latvia, Poland, Portugal, Romania, Slovenia, Spain, Sweden, France, Netherlands, Belgium, Luxembourg , Slovakia, Norway, Switzerland, Turkey, the Republic of North Macedonia, Iceland, Montenegro and Serbia.