Teaching RPG enemies to learn from the player — a 2007 neural-network prototype
In 2007, my final-year engineering project asked a question that still shapes my work today: could an enemy learn from the way a player behaves instead of following a completely fixed script?
I built a small 3D action RPG prototype in C++ and DirectX. The player fought a group of slime-like agents inside an arena using physical attacks, spells, healing and target locking. Behind that playable demo was an experimental neural network intended to choose when an NPC should wander, follow or evade.
What I built
The project combined three areas that are often treated separately: game-system architecture, real-time graphics and machine learning. The engine used a Win32 game loop with initialise, update and render stages, an object-oriented character controller, and roughly 50 C++ source files with a similar number of headers.
The rendering work included 2D sprites inside a 3D world, camera-facing billboards, animated texture frames, alpha blending and mouse picking. The prototype also included a skybox, lighting, particle effects, camera occlusion, HP and MP systems, frame-rate monitoring and debug information for each agent.

Neural networks, from the ground up
A neural network is a function built from many small units, called neurons, joined by weighted links. Each neuron does three simple things: it takes some numbers in, multiplies each one by a weight, and adds them up.
In the prototype every input was first scaled into the range 0 to 1, so that a naturally large value such as distance could not drown out a small one such as a health ratio. The neuron’s net input is then the weighted sum:
X = x₁w₁ + x₂w₂ + … + xₙwₙ
That sum still has to become a decision, which is the job of the activation function. The simplest is a step: output +1 if X reaches a threshold θ, otherwise −1. Because I wanted the outputs to behave like probabilities, the network used a sigmoid instead, which squeezes any value into a smooth curve between 0 and 1:
y = 1 / (1 + e−X)
Arrange neurons in layers and you get a multi-layer perceptron, the type used in the project. Information moves forward: from the input layer, through one or more hidden layers, to the output layer. The weights on the links are the network’s long-term memory, and learning simply means adjusting those weights until the outputs become useful.
The network inside each monster
I deliberately kept the model compact enough to train during play: four inputs, one hidden layer with three neurons, and three outputs.
- Inputs: distance to the player, the monster’s health ratio, the number of nearby monsters and its current action.
- Outputs: wander, follow or evade. Whichever output is most strongly activated becomes the monster’s action at that moment.
How the network learns
Training followed four steps that are still the backbone of most neural-network training today.
- Initialise. Every weight and threshold starts as a small random number. The network also needs a training set: example inputs paired with the outputs we would want. In the demo each example held four input values and three desired outputs between 0 and 1, written by hand or captured from a player.
- Activate. An example is fed forward through the network to produce its actual outputs.
- Learn. The actual output y is compared with the desired output t. The difference, the error e = t − y, drives the delta rule w ← w + α · x · e. The learning rate α (typically 0.01–0.1) trades speed of learning against sensitivity to noise. Backpropagation carries the error from the output layer back through the hidden layer, so every weight receives its share of the correction.
- Iterate. Move on to the next example and repeat until the error falls below an acceptable level.
Turning combat into training data
A game does not come with a labelled dataset, so the report explored several ways to create one from play.
- Predicting the outcome. When an encounter starts, the network estimates the probability that the NPC will win. When it ends with the NPC defeated, the target for that prediction becomes 0. After a few dozen weight updates the estimate becomes a reasonable guide.
- Learning from mistakes. If a monster follows the network’s suggestion and dies, that decision is treated as a bad one, and the network is adjusted to suppress it under the same input conditions, a simple form of reinforcement.
- Seed rules. Before any learning, a few authored rules gave the network something sensible to start from: attack the player and stop chasing once the hit lands, evade when health is low, and follow the player when no other monster is around them.
The larger idea was imitation learning. In an AI test bench, a developer or player fights the agents while the game records the player’s current action, the enemy’s range, obstacle signals and HP/MP ratios. Those recordings become the training set, so an opponent can learn to respond the way a human did in a similar situation. Swapping roles between player and AI then generates more examples.
The report also sketched a next step: inferring whether the player is a fighter, priest or mage from how they behave, using Bayesian analysis, instead of letting the AI “cheat” by reading the class directly. It was an early attempt to treat player behaviour as game data rather than encode every reaction by hand.
What the experiments actually found
The adaptive system was promising, but it was not reliable enough to call finished. That result became the most useful part of the project.
- Adding more hidden layers did not improve the agents. The deeper versions became less predictable and required more computation.
- A smaller network gave more direct mappings and was more practical for an in-game experiment, although reducing it too far also produced poor decisions.
- Adding momentum to the weight-correction process (carrying part of the previous update into the next one) improved stability.
- Online training depends heavily on the quality and range of the captured player examples. A small or noisy training set cannot produce convincing behaviour.
- Debug visibility mattered. Showing an agent’s state, target, health and output made the learning system possible to evaluate inside the running game.
The prototype therefore demonstrated the pipeline and the possibility of learning player-like manoeuvres, while also showing that a neural network is not automatically better than authored behaviour. A game still needs clear inputs, measurable outcomes, good training data and fallbacks that protect the player experience.

Why this 2007 project still matters to me
Many of the conclusions feel current again. Today we can build larger models and collect more data, but interactive AI still has to run within a game loop, explain its state to a developer and produce behaviour that feels fair. Complexity only helps when it improves the player’s experience.
The report also proposed practical tools around the model: an AI test bench, character and map editors, profiling, editable game data and better ways to collect player attributes during play. Those ideas connect directly to how I now think about AI-assisted development: useful intelligence needs an interface, an iteration loop and a clear creative purpose.
Archive note: this retrospective is based on my 2006–07 Bachelor of Engineering final-year report, AI Role Playing Game Development, which is publicly available from City University of Hong Kong: read the full report (PDF). The screenshots are from the prototype I produced for the project; the diagrams have been redrawn and the text rewritten for today’s journal.
Sources
- Chow Ying Kit (Rick Chow), AI Role Playing Game Development, BEng (Hons) Computer Engineering final-year project report, Department of Electronic Engineering, City University of Hong Kong, 2006–07. CityU Library PDF