View publication
| Title | Towards a Spatio-Temporal Knowledge Graph Question-Answering Benchmark for Real Estate |
| Authors | Luciana Tanevitch, Aidan Hogan, Diego Torres |
| Publication date | 2025 |
| Abstract | Spatio-temporal knowledge graphs extend traditional knowledge graphs by capturing context with respect to both location and time. The dynamic and multidimensional aspects of these knowledge graphs require specialized methods for representation, integration, and reasoning. Likewise, solving the Question Answering (QA) task over spatio-temporal knowledge graphs involves not only understanding entities and their relationships, but also reasoning over when and where events occur. The Spatio-Temporal Knowledge Graph Question Answering (STKGQA) field remains underexplored. The lack of benchmarks in this field hinders progress by limiting the community's ability to systematically evaluate and compare approaches, identify strengths and weaknesses, and measure improvements over time. In this work, we present the initial strategies for building a STKGQA benchmark in the real-estate domain to assess the feasibility of using large language models (LLMs) to support the creation of synthetic benchmarks for STKGQA. We focus on evaluating how accurately LLMs can translate natural language questions into SPARQL queries compared to manually authored gold-standard queries, and how naturally questions an LLM can generate. Our findings include common error patterns, quality evaluation criteria, and a set of lessons learned that inform future efforts toward scalable and reliable benchmark creation in this domain. |
| Pages | 1-8 |
| Conference name | Proceedings of the International Conference of the Chilean Computer Science Society |
| Publisher | IEEE Computer Society Press (Los Alamitos, CA, USA) |
| Reference URL |
|