AI Revolutionizes Search and Access at National Archives
The National Archives and Records Administration (NARA) is undergoing a significant technological transformation, moving from struggling with older AI tools to enthusiastically embracing generative artificial intelligence. According to NARA's chief technology officer, Gulam Shakir, previous AI technologies weren't well-suited for managing the agency's immense collection of unstructured, free-form documents. However, the advent of large language models like those powering ChatGPT has proven to be a natural fit for NARA's mission, particularly in drastically improving how users find information within its billions of records, with the aim of delivering the most relevant results on the very first search page.
A key initiative highlighted is an AI-powered semantic search pilot. This advanced search capability goes beyond simple keyword matching, striving to understand the user's underlying intent and the contextual meaning within the documents. This is crucial for navigating NARA's extensive digital holdings, which include hundreds of millions of digitized pages and billions of views across various platforms. Semantic search is expected to simplify the process for the public, researchers, and historians, efficiently tackling the challenges posed by unstructured data. Moreover, it holds the potential to uncover connections between seemingly unrelated records, fostering a deeper understanding of historical events and potentially leading to new discoveries within the archive's vast collection. NARA has chosen Google's Vertex AI platform, utilizing the Gemini model, to power this pilot program.
Beyond enhancing public search, NARA is exploring multiple other AI applications to boost internal efficiency. These include a pilot for an AI-based knowledge chat interface using Amazon Kendra, designed to quickly query information from curated internal knowledge bases spanning various document types. Other projects involve automating the process for handling Freedom of Information Act requests, creating automated tools for generating archival descriptions or metadata (a process previously very labor-intensive), and developing systems to screen and redact sensitive personally identifiable information from digitized records. By exploring these diverse use cases simultaneously, NARA aims to identify the most effective ways artificial intelligence can support its mission and save valuable staff time.
A new startup founded by Pinterest veterans is integrating its AI-powered semantic search technology with Salesforce Commerce Cloud to help online retailers improve search relevancy and personalization.
Discover how Contextual Retrieval enhances RAG systems, reducing retrieval failures by up to 67% and improving AI performance by adding crucial context to data chunks.
Microsoft is preparing an early feature update for Windows 11, expected in April 2025, focusing heavily on new AI capabilities like semantic search and enhanced translation.
Dow Jones is integrating generative AI into its Factiva news database, powered by Google's Gemini, after securing content licensing agreements with over 4,000 news publishers worldwide.
Explore how integrating knowledge graphs and vector databases creates a powerful approach to Retrieval-Augmented Generation (RAG), improving AI accuracy and reliability.
New research uses AI and semantic analysis to reveal how media bias affects both what news events are covered and how topics are discussed.
Explore the development of an AI assistant designed to process vast data into insights while upholding integrity, trust, and transparency.
Learn how Amazon Web Services is enabling businesses to deliver superior search experiences by blending traditional keyword matching with advanced AI-powered semantic understanding and multimodal inputs using Amazon Bedrock and OpenSearch Service.
Discover how advanced search technologies like RAG and vector search are critical for enabling AI, particularly Gen AI, to drive significant business transformation and productivity gains across Australian organisations.