Combining LLMs-based Conversational Agents and Ontologies for Open Data Research
Résumé
Open Science has significantly increased the availability of heterogeneous scientific datasets. However, this abundance often results in difficulties identifying data that are truly relevant to a specific user needs, as pertinent resources may be obscured by large volumes of unrelated or poorly described data. This paper proposes an approach that combines semantically enriched metadata and LLM-based agents able to interpret natural language queries, in order to support the retrieval of appropriate datasets. This architecture enables the extraction and refinement of user intent and the generation of intelligible justifications for the retrieved results. To assess the performance of the proposed system, a comparative evaluation across multiple Earth Observation (EO) scenarios has been carried out. Four LLM agents has been evaluated -LLaMA 3.3 70B, Mistral Saba 24B, Deepseek-R1, and Qwen 32B -using metrics such as answer relevancy, contextual precision, recall, and faithfulness. The results, conducted with the Deepeval library associated with the LLaMA 3 8B model, show relatively high scores on answer relevancy and contextual precision, especially for the LLaMA and Deepseek-R1 models, demonstrating the benefits of LLM-based query interpretation and explanation generation.
| Origine | Fichiers produits par l'(les) auteur(s) |
|---|---|
| licence |

