TensorICE - Tensor Methods for Interpretable, Controllable, and Efficient Multimodal Foundation Models
Foundation Models have emerged as a dominant paradigm in Artificial Intelligence, demonstrating remarkable capabilities across various domains. However, their massive scale and complexity present significant challenges, including overparameterization leading to a hefty computational cost, lack of interpretability, and potential biases. The technological progress of AI makes these challenges more urgent, with financial and societal impacts that are already observed, including exhibiting biased decision making that may disadvantage underrepresented populations, undesired behavior that is not aligned with user intent, while carrying a significant computational and financial cost for training and inference. The TensorICE project aims to mitigate these challenges by developing novel frameworks and network architectures for interpretable, controllable, and efficient multimodal foundation models. This will be achieved by devising tensor-based optimization problems that can efficiently factorize existing models, while proposing new layers that will exhibit desirable properties by construction, while being able to handle data sources of varying geometry.
As proof-of-concept, TensorICE will develop a novel Multimodal Foundation Model for Climate and Earth Observation Data, integrating diverse data sources such as satellites, ground stations, and simulations. By addressing critical weaknesses in current state-of-the-art AI models, TensorICE has the potential to contribute significantly to advancing our understanding and control of large-scale AI models, the development of more reliable, efficient, and trustworthy AI systems for critical inter-disciplinary applications, hence enhancing efforts towards sustainable development of AI.
CyI Principal Investigator: Dr Andreas Demou
Co-Principal Investigator: Adjunct Assoc. Prof. Dr. Mihalis Nicolaou
Publications
- Koromilas, Panagiotis, Andreas D. Demou, James Oldfield, Yannis Panagakis, and Mihalis Nicolaou. "PolySAE: Modeling Feature Interactions in Sparse Autoencoders via Polynomial Decoding." arXiv preprint arXiv:2602.01322 (2026).
- Koromilas, Panagiotis, Theodoros Giannakopoulos, Mihalis A. Nicolaou, and Yannis Panagakis. "Neural Collapse by Design: Learning Class Prototypes on the Hypersphere." arXiv preprint arXiv:2605.20302 (2026).
- Demou, Andreas D., Panagiotis Koromilas, James Oldfield, Yannis Panagakis, and Mihalis Nicolaou. "fmxcoders: Factorized Masked Crosscoders for Cross-Layer Feature Discovery." arXiv preprint arXiv:2605.09438 (2026).
- Plitsis, Manos, Theodoros Kouzelis, Panagiotis Koromilas, Vassilis Katsouros, Mihalis A Nicolaou, and Yannis Panagakis. "Disentangling Local and Global Semantics in Diffusion Models for Image Editing." International Journal of Computer Vision 134, no. 4 (2026): 160.
- Oldfield, James, Shawn Im, Sharon Li, Mihalis Nicolaou, Ioannis Patras, and Grigorios Chrysos. "Towards interpretability without sacrifice: Faithful dense layer decomposition with mixture of decoders."Advances in Neural Information Processing Systems 38 (2026): 144468-144513.
- Triantafyllou, Thanasis, Mihalis A. Nicolaou, and Yannis Panagakis. "Interpretable Music Harmonic Analysis Through Multilinear Mixture of Experts." In ICASSP 2026-2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), (2026) pp. 16082-16086.
Additional Info
- Acronym: TensorICE
- Website: https://github.com/TensorICE
- Center: CaSToRC
- Funding Source: RIF/EXCELLENCE/0524
- CyI Funding: €191.400
- Funding Period: --
- Starting Date: 01/05/2025
- End Date: 30/04/2027
- Coordinator: THE CYPRUS INSTITUTE
-
Partners:
Foreign Research Organizations: IMPERIAL COLLEGE LONDON, ATHENA RC – ARCHIMEDES UNIT
