by Researchers from the Institute for Basic Science have made a groundbreaking discovery regarding the memory processing of artificial intelligence (AI) models. They found that the memory consolidation mechanism in AI models, particularly in the Transformer model, closely resembles the memory processing in the hippocampus of the human brain. This finding presents a fresh perspective on memory formation and consolidation in AI systems and sheds light on the development of Artificial General Intelligence (AGI).
Understanding how AI systems learn and retain information is crucial in the quest to develop powerful AI technologies. To delve deeper into this process, the research team focused on memory consolidation, which is the transformation of short-term memories into long-term memories in AI models. They drew inspiration from the human brain’s memory consolidation process, specifically the role of the NMDA receptor in the hippocampus.
The NMDA receptor acts as a “smart door” in the brain, facilitating learning and memory formation. When a brain chemical called glutamate is present, the nerve cell becomes excited. However, a magnesium ion acts as a gatekeeper, blocking the door and only allowing substances to flow into the cell when it steps aside. This process enables the brain to create and store memories.
Interestingly, the researchers discovered that the Transformer model used a similar gatekeeping process to the brain’s NMDA receptor. This led them to explore whether the Transformer’s memory consolidation could be controlled using a mechanism akin to the NMDA receptor’s gating process.
In the animal brain, a low level of magnesium weakens memory function. The researchers found that by mimicking the NMDA receptor, they could enhance long-term memory in the Transformer model. Adjusting the Transformer model’s parameters to reflect the gating action of the NMDA receptor resulted in improved memory performance.
This breakthrough finding suggests that established knowledge in neuroscience can help explain how AI models learn and improve their memory capabilities. By incorporating brain-inspired nonlinearity into AI constructs, researchers can simulate human-like memory consolidation and create more advanced AI systems.
C. Justin Lee, a neuroscientist director at the institute, emphasizes the importance of this research for advancing both AI and neuroscience. The findings allow for a deeper understanding of the brain’s operating principles, enabling the development of more advanced AI systems based on these insights.
CHA Meeyoung, a data scientist in the team, highlights the potential for low-cost, high-performance AI systems that learn and remember information like humans. Unlike large AI models that require immense resources, the human brain operates remarkably with minimal energy. Incorporating brain-inspired mechanisms into AI design opens up possibilities for more efficient and effective AI technologies.
This study sets itself apart by incorporating brain-inspired nonlinearity into AI models, representing a significant leap forward in simulating human-like memory consolidation. The synergy between human cognitive mechanisms and AI design not only holds potential for creating low-cost, high-performance AI systems but also provides valuable insights into the inner workings of the brain through AI models. This research paves the way for further exploration of the convergence between AI and neuroscience, bringing us closer to understanding and replicating human intelligence.
*Note:
1. Source: Coherent Market Insights, Public sources, Desk research
2. We have leveraged AI tools to mine information and compile it
Ravina Pandya, Content Writer, has a strong foothold in the market research industry. She specializes in writing well-researched articles from different industries, including food and beverages, information and technology, healthcare, chemical and materials, etc. With an MBA in E-commerce, she has an expertise in SEO-optimized content that resonates with industry professionals.
