Much of Geospatial Web Search Is Beyond Traditional GIS
Ilya Ilyankou, Stefano Cavazzi, James Haworth
TLDR
This paper reveals that geospatial web search is far more prevalent and practically oriented than previously understood, often exceeding traditional GIS capabilities.
Key contributions
- Developed a new method using dense embeddings and SetFit to classify geospatial queries.
- Identified 18% of MS MARCO queries as geospatial, nearly triple prior estimates.
- Found geospatial search is dominated by practical needs like costs, hours, and travel.
- Much of this activity falls outside the scope of traditional GIS and knowledge graphs.
Why it matters
This research significantly redefines the scale and nature of geospatial web search, highlighting a gap between user needs and current GIS/knowledge graph capabilities. It has critical implications for designing hybrid retrieval architectures and benchmarking geographic reasoning in large language models, pushing for more practical, real-time systems.
Original Abstract
Web search queries concern place far more often than existing labelling schemes suggest, yet the landscape of geospatial web search queries - what people ask of place, and how often - remains poorly characterised at scale. We apply dense sentence embeddings, a lightweight SetFit classifier, and density-based clustering to the full MS MARCO corpus of 1.01 million real Bing queries without prior filtering for toponyms or spatial keywords, identifying 181,827 geospatial queries (18.0%), nearly threefold the 6.17% labelled as Location in the original annotations. The resulting taxonomy of 88 query categories reveals that geospatial web search is dominated by transactional and practical lookups: costs and prices alone account for 15.3% of geospatial queries, nearly twice the size of the entire physical geography theme. Much of this activity - costs, opening hours, contact details, weather, travel recommendations - falls outside the scope traditional GIS systems and knowledge graphs are built to serve. The categories vary substantially in the kind of answer they admit, from deterministic lookups answerable from spatial databases or knowledge graphs to evaluative or temporally volatile queries that require generative or real-time systems. We discuss implications for hybrid retrieval architectures and for benchmarks of geographic reasoning in large language models. We openly release the labelled dataset, classifier, and taxonomy.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.