Using LLMs to Study and Predict Innovation Dynamics

By leveraging the growing availability of open-source Large Language Models (LLMs), CREF is automating the analysis of enormous amounts of unstructured information—websites, scientific publications, patents, and code repositories. This allows for both a geographic and semantic mapping of technological capabilities in countries, regions, and cities.

Geographic mapping identifies territorial specialization profiles and future innovation directions with the greatest competitive advantages, offering useful tools for regional policy. Semantic mapping, on the other hand, measures the distance between different technical domains, allowing for the development of models to predict technological innovations based on new combinations (recombinant innovation). A practical application of this approach has already been recognized by the European Patent Office (October 2024).

Over the next three years, this research will be extended to other document sets (e.g., scientific articles) and integrated with models to extract and interpret semantic meanings from the activations of predictive neural networks. These tools will also be used to connect policy guidelines with local capabilities, particularly for environmental policies aimed at a more sustainable society. For example, it will be possible to link patents with European legislation on reducing industrial pollution or to map the incidence of critical raw materials in green technologies to identify risks in supply chains.

Using LLMs to Study Online Social Phenomena

Similar to the techno-scientific analysis, LLMs will be used to gather signals on social dynamics from discussions on social networking platforms. The project will explore the feasibility of these tools for mapping social phenomena like social discontent and inequality.

Theory of Artificial Creativity

Research in this area focuses on the autonomous evolution of AI when faced with unexpected events and non-stationary dynamics. A crucial challenge is Continual Learning—integrating new data without incurring “Catastrophic Forgetting” (the loss of previously acquired knowledge). This is connected to creativity, understood as the exploration of unknown conceptual spaces that are linked to acquired information, drawing inspiration from Stuart Kauffman’s concept of the “Adjacent Possible.”

The Dreaming Learning algorithm is a significant contribution in this area, as it is the first statistical learning system capable of exploring new information contextually linked to its existing knowledge base and correctly describing non-stationary data sources. These features have been validated both theoretically and experimentally. Studies are underway for extensions that leverage the topological properties of dynamic systems implemented by neural networks, aiming to overcome current limitations. Among the practical applications being developed are systems that support human creativity, such as intelligent autonomous agents capable of dynamically evolving their own knowledge base. A major result of Dreaming Learning is its ability to mitigate Model Collapse, the progressive loss of generative capacity in AIs trained on data they themselves have generated, which is particularly relevant for LLMs. This problem is addressed through specific parameters, studied with statistical mechanics approaches. An ongoing case study involves a system for analyzing and anticipating human movements, with applications in dance creativity, using technologies based on Transformers and vector quantization.

Additionally, Lyapunov Learning has been developed, a method that extends the capabilities of Dreaming Learning to multidimensional variables. Based on the principle of the “Edge of Chaos,” this approach brings the autonomous dynamics of neural networks closer to conditions on the edge of deterministic chaos. Already applied to simple systems, it is now being extended to real-world phenomena like climate change and ecosystem collapse.

Socioeconomic Impact of AI – The Labor Market

The project aims to provide a quantitative perspective on the impact of AI on the world of work, examining how it can substitute or complement human skills through three research areas:

• Measuring Occupational Exposure: We will analyze the pitches of AI startups to assess market interest in automating specific professions, comparing these with classic indices. This will identify vulnerable professions, reasons for a lack of automation, and crucial skills for the future.

• Identifying Emerging Skills: We will analyze job advertisements and technological innovations to identify growing and declining skills, developing a methodology to predict fundamental or obsolete abilities, including from code repositories like GitHub.

• Public Opinion and Ethics: We will study the influence of social trust and ethical considerations on the adoption of AI, using data from social networks to connect public perceptions with startup investments.

This interdisciplinary approach integrates market data, occupational analysis, and social perceptions to understand and guide the evolution of work in the AI era.

People’s Values and AI’s Values

With the integration of AI into society, it is essential that these systems reflect local moral values to foster ethical interactions and curb harmful content. The goal is to develop “Responsible Artificial Intelligence” models that are sensitive to ethical, cultural, and social contexts. A central challenge is the effect of AI on online discussions, where “filter bubbles” limit dialogue between different viewpoints.

Recent studies (Brugnoli 2023, 2024) have shown how moral values influence the evolution of opinions in virtual communities, rooting themselves more deeply than political affiliations alone. While different ideologies align with distinct moral configurations, the limited number of “irreducible basic elements” identified by psychological theories of morality (like Moral Foundations Theory, or MFT) suggests common roots across different ideologies. This discovery could be the key to communication that transcends diversity and polarization. In this research area, we will study to what extent these ideological “building blocks” are represented and understood by AI algorithms and how much they influence online discussions, where AI-driven bots are increasingly present.

CREF Researchers

Andrea Tacchella

Angelica Sbardella

Francesco De Cunzo

Sony Csl- Roma Researchers

Alessandro Londei

Vittorio Loreto

Ruggiero Lo Sardo 

FC – International Finance Corporation

Translated.com

Mamacrowd

 STARTS-AIR Project for Art and Science

Aterballetto, for the development of performative artistic interaction technologies between performers and AI systems to support natural creativity.

Linguistics/Centre for Behaviour and Evolution (Christine Cusckley)

CSL-Rome is part of the French project “ScientIA,” which studies the impact of Artificial Intelligence on other scientific disciplines

FC – International Finance Corporation

Translated.com

Mamacrowd

Sony CSL Rome partecipa al progetto STARTS-AIR all’intersezione tra Arte e Scienza.

Aterballetto, per lo sviluppo di tecnologie di interazione artistica performativa tra performer e
sistemi di IA per il supporto alla creatività naturale.

Linguistics/Centre for Behaviour and Evolution (Christine Cusckley)

CSL-Rome è parte del progetto francese “ScientIA” sullo studio dell’impatto dell’Intelligenza Artificiale in altre discipline scientifiche.