Smart NPCs using LLAMA on Metaverse

Abstract

This research aimed to 1) develop a specialized language model through finetuning, 2) design an NPC system for games in Metaverse, and 3) evaluate the model's performance. The study developed 3D characters on Unity 2022.3.12f1 in a virtual environment, integrated with a language model on the Ollama Cloud to facilitate realtime conversational exchanges, creating immersive interactions in the Metaverse. Findings include: 1) The specialized language model, LLaMA 3.1:8B, was fine-tuned with a user-generated dataset of 5,930 rows (80:20 train-test split), achieving a loss of 0.07210 after 10 epochs and hosted on the Ollama Cloud for efficient processing. 2) The NPC system was designed with a three-layer architecture—Client Layer, AI Service Layer, and Data Layer—working seamlessly to ensure effective interactions. 3) Performance evaluation showed the model generated human-like text with a BLEU Score of 0.4983 and Perplexity of 1.0987, handled complex questions with ROUGE-1 (0.3072), ROUGE-2 (0.1897), and ROUGE-L (0.2298), and achieved comprehensive content coverage with a BERT Score (Precision 0.6295, Recall 0.7242, F1 0.6730). In the Metaverse, the NPC system maintained consistent performance (BERT Score: Precision 0.6340, Recall 0.7266, F1 0.6766), demonstrating robust applicability and effectiveness.