LARGE MODEL AGENTS: A SURVEY ON TECHNICAL EVOLUTION, MULTIMODAL FUSION, AND VERTICAL DOMAIN APPLICATIONS
Keywords:
LLM agent, Multimodal fusion, GLM-4, ReAct, Education, Healthcare, Tool callingAbstract
LLM-driven agent technology has achieved breakthrough progress, marking AI's leap from perceptual to actionable intelligence. This paper reviews four evolution stages: symbolic, reactive, RL, and LLM-driven agents. It analyzes core architecture including ReAct, memory, tool calling, and multimodal perception. For education and healthcare, we design a GLM-4-based multimodal educational assistant and a ReAct-based MedGuard medication assistant, validating effectiveness through experiments. Finally, we analyze six challenges (hallucination, alignment, security, planning, cost, interpretability) and propose directions: world models, RL fine-tuning, verifiable execution, edge deployment, and ethical frameworks.References
[1] Du Zhengxiao, Qian Yujie, Liu Xiao, et al. GLM: General Language Model Pretraining with Autoregressive Blank Infilling. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL 2022). 2022. DOI: 10.18653/v1/2022.acl-long.26.
[2] Wang L, Ma C, Feng X, et al. A Survey on Large Language Model based Autonomous Agents. Frontiers of Computer Science. 2024, 18(6): 1-26. DOI: 10.1007/s11704-024-40231-1.
[3] Brooks R A. A Robust Layered Control System for a Mobile Robot. IEEE Journal of Robotics and Automation. 1986, 2(1): 14-23.
[4] Sutton R S, Barto A G. Reinforcement Learning: An Introduction. 2nd ed. MIT Press, 2018.
[5] Anthropic. Introducing the Model Context Protocol. 2024. Available at: https://www.anthropic.com/news/model-context-protocol.
[6] Hong W, Ding M, Zheng W, et al. CogVideo: Large-scale Pretraining for Text-to-Video Generation. In: Proceedings of the International Conference on Learning Representations (ICLR 2023). 2023.
[7] Yao S, Zhao J, Yu D, et al. ReAct: Synergizing Reasoning and Acting in Language Models. In: Proceedings of the International Conference on Learning Representations (ICLR 2023). 2023.
[8] Kapoor S, Stroebl B, Siegel Z S, et al. AI Agents That Matter. arXiv preprint, 2024. DOI: 10.48550/arXiv.2407.01502.
[9] Ming Li. Applications and Challenges of Multimodal Large Models in Educational Scenarios. China Distance Education, 2025.
[10] Team GLM. ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools. arXiv preprint, 2024. DOI: 10.48550/arXiv.2406.12793.
[11] Hong S, Zhuge M, Che J, et al. MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework. arXiv preprint, 2023. DOI: 10.48550/arXiv.2308.00352.
[12] Park J S, O'Brien C, Cai C J, et al. Generative Agents: Interactive Simulacra of Human Behavior. In: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (ACM CHI 2023). 2023: 1-22. DOI: 10.1145/3586183.3606763.
[13] Cemri M, Pan M Z, Yang S, et al. Why Do Multi-Agent LLM Systems Fail? arXiv preprint, 2025. Available at: https://arxiv.org/abs/2503.13657.
[14] Amodei D. AI Safety in the Age of Agentic Systems. Anthropic, 2024. Available at: https://www.anthropic.com/news/ai-safety-in-the-age-of-agentic-systems
[15] Xi Z, Chen W, Guo X, et al. The Rise and Potential of Large Language Model Based Agents: A Survey. arXiv preprint, 2023. DOI: 10.48550/arXiv.2309.07864.