Can a Bucket of Water Translate? Exploring the Encoding–Decoding Ability of Randomly Initialized Neuro-Symbolic Transformers by Dr. Arghya Pal

Venue: Bhart501

Abstract: There is a growing interest in developing artificial neural networks through the training of large models. But what lies hidden in an overparameterized neural network with random weights? If the distribution is properly scaled, such a network contains a subnetwork that can perform well without ever modifying its weights. The number of possible subnetworks grows combinatorially with the size of the network, and modern neural networks often contain millions or even billions of parameters. Thus, we should expect that even a randomly weighted neural network contains a subnetwork capable of performing well on a given task. The core idea of this talk is to frame the subnetwork-finding problem as a Differentiable Integer Linear Programming (ILP) Solving problem. Unlike existing neuro-symbolic solvers, this talk will introduce an algorithm that does not require a continuous relaxation of semantic constraints. Instead, it allows for a direct, more precise, and efficient integration of neural representations into the ILP formulation. By the end of the talk, we will see that the solver achieves superior performance compared to conventional ILP solvers, neuro-symbolic black-box solvers, and Transformer-based encoders. Furthermore, a deeper analysis reveals that such a solver can significantly enhance the precision, consistency, and faithfulness of the generated explanations. This opens new opportunities for advancing neuro-symbolic architectures toward explainable and transparent deep learning in complex domains.

Bio: Dr. Arghya Pal is a Lecturer at Monash University. He earned his Ph.D. from the Indian Institute of Technology Hyderabad, followed by postdoctoral research appointments at Monash University and Harvard University.  His research focuses on generative models, transfer learning, causal inference, learning under limited supervision, and logical reasoning. He is a recipient of a gold medal for his Master’s degree, the Intel Ph.D. Fellowship (2016–2020), honored as best researcher award twice – one during PhD and other as an Alumni from IIT Hyderabad. He has been honored with the Magna-cum-Laude award from Harvard MRI society and best researcher award from School of IT Monash University.

Dr. Pal has experience in teaching units like Modeling Discrete Optimization Problems, Data Analytics, Deep Learning, Programming Paradigm, Malicious Attack and Dark Sides of AI, and Modeling Data Science Problems at Monash University. He assumed academic positions beyond teaching such as; session chair in IJCNN, Senior Area Chair in AAAI, Reviewer of prestigious venues such as TPAMI, NeurIPS, AAAI, CVPR, ICLR, etc.