Craft an Iron Sword: Dynamically Generating Interactive Game Characters by Prompting Large Language Models Tuned on Code

Here's some preliminary work from Microsoft from 2022 that incorporates OpenAI's Codex model to make NPCs that can interact with the player using natural language instructions. It works by defining an API of functions the bot can use, then having Codex generate function calls in response to the player's instructions.

Paper: https://aclanthology.org/2022.wordplay-1.3/
Repo: https://github.com/microsoft/interactive-minecraft-npcs
Videos: Introductory Demo, Escape Room Demo