Giovani in un'ora - Ciclo di seminari - Terza parte
- Day - Time: 16 October 2024, h.11:00
- Place: Area della Ricerca CNR di Pisa - Room: C-29
Speakers
Referent
Abstract
Gabriele Lagani - "Hebbian learning algorithms for deep neural networks: explorations and outlooks"Abstract: Deep learning systems have achieved outstanding results in various AI tasks. However, such system suffer from a number of limitations, for example in terms of energy and data efficiency. Is it possible to improve upon such limitations by taking inispiration from the human brain? Recent results suggest that this is the case! This talk aims at presenting biologically-grounded Hebbian learning methodologies, showing how they can be harnessed to improve learning efficiency in deep networks. Spiking neural models – which represent a promising direction for more efficient AI -- will also be discussed, highlighting the open challenges towards future developments in this field.
Katherine Elizabeth Abramski - "Investigating biases in humans and LLMs using a multilayer semantic network approach"
Abstract: LLMs have been shown to exhibit many of the same biases that humans possess, some of which can have negative impacts on society, such as harmful stereotypes. For that reason, it is important to gain a better understanding of these biases so we can mitigate their harmful effects. We know that these biases originate in training data, and we can observe them in LLM outputs, but what is the nature of the semantic representations underlying these biases? How is conceptual knowledge in LLMs organized, and how is that knowledge similar and different to that of humans? We aim to answer these questions by modeling conceptual knowledge in both humans and LLMs as multilayer semantic networks. We explore implicit biases within these conceptual knowledge bases by simulating cognitive processes within the networks. Our exploration provides insights about similarities and differences in implicit biases between humans and LLMs.
Cosimo Rulli - "Exploring Expansion in Learned Sparse Retrieval"
Abstract: Large Language Models (LLMs) can project text pieces into numerical vectors, enabling the modeling of document relevance to a query using simple similarity measures. In Learned Sparse Retrieval (LSR), each token is assigned a positive score proportional to its importance in the text, producing sparse, high-dimensional vectors. LSR provides an efficient and effective retrieval solution, with a human-understandable format and the ability to generalize to unseen data. LSR techniques inherently apply query and document expansion, activating relevant tokens in the vocabulary even if they do not appear in the text. We explore the expansion capabilities of LSR systems showing how LSR can match the performance of more resource-intensive approaches, such as multi-vector retrieval systems.