Foxgrad/graphwar-arena

Graphwar Arena: eight bots that learn to fire equations

In Graphwar, every shot is a mathematical function. You type y = f(x), the curve leaves your soldier, and it has to bend around obstacles to hit an enemy. This repository contains a small Python re-implementation of that idea and eight bots that fight in it. Six of them start from a neural network with random weights and learn to aim on their own, each with a different learning method. One of them shoots in three dimensions.

Credits: what this project reproduces

This project would not exist without two people's work.

• Graphwar is an open-source artillery game created by Lucas Catabriga Rocha, in which players hit each other with mathematical functions. Source code (GPL-3.0): github.com/catabriga/graphwar. Website: graphwar.com.
• The idea of this arena comes from the video “I Gave a Graphwar Bot a 3rd Dimension. It Fought 7 Bots” by the YouTube channel Onion Das. In it, a 3D bot whose shots also move along a z(t) axis takes on seven other bots: Peace, Pole-Zero, Irrational, Evolution, Fly, Locus and Dot. The eight bot names and the 3D idea come from that video.

This is an independent, unofficial project. It is not affiliated with either creator, and it contains no code or assets from Graphwar or from the video. The game engine was written from scratch in Python with simplified rules. I did not have access to the video's bots, so each bot here is my own interpretation of its name. Most of the video's bots are described as fixed formulas. Here, only Dot and Locus follow fixed rules, and the other six learn from scratch.

The game

• The board spans x ∈ [−25, 25] and y ∈ [−15, 15]. Obstacles are cylinders with a random radius and height. A shot that hits an obstacle digs a crater, so later shots can pass through that spot.
• A shot is a curve that starts at the shooter and moves away from it. It stops when it hits an obstacle, a soldier (allies included) or the edge of the board. In the equations shown in game, x is the distance travelled from the shooter.
• Normal shots stay at ground level, so any obstacle stops them. The 3D bot adds a height z(x) = H·sin(πx/D) for x < D. It can jump over an obstacle that is lower than its arc, as long as it comes back down to soldier height before reaching the target.
• A match is 3 soldiers against 3. Teams take turns, one shot per turn, and a team wins when the other team has no soldiers left.

The eight bots

Bot	How it learns	What it writes	Brain
Dot	nothing: fixed rule, shoots straight at the clearest target	y = a·x	none
Locus	nothing: plans a free path column by column, then writes it as a sum of absolute values	`y = a·x + Σ cᵢ·	x − uᵢ
Evolution	genetic algorithm	slope + 2 Gaussian bumps	61 → 8 → 5
Pole-Zero	evolution strategies (OpenAI-ES)	slope + 2 rational terms with complex poles	117 → 16 → 7
Fly	cross-entropy method	slope + 2 sine waves	61 → 8 → 6
Irrational	PPO	slope + √x + `	x − p
Peace	PPO, and hitting an ally costs 3× a win	slope + 2 sigmoid steps	173 → 64 → 64 → 5
3D	PPO in three dimensions	slope + 1 bump, plus a jump z(x)	173 → 64 → 64 → 5

What a bot sees. A bot gets 5 numbers about its target, plus a "corridor" grid of 8 × 7 cells aligned with the straight line to the target. For each cell it gets the share covered by obstacles. Depending on the bot, it also gets the height of those obstacles (3D), the position of its allies (Pole-Zero, Irrational, Peace, 3D) and the position of other enemies (Peace). Inputs are normalized.

What a bot outputs. A few numbers between −1 and 1. The first one is an angle offset from the straight line to the target. The others set the shape of the curve: where to put a bump, how high, how fast a wave oscillates, and so on. The curve is then corrected so that it lands on the target. This is an aiming aid: a bot that outputs all zeros shoots straight like Dot. To beat Dot, a bot has to learn to bend its shot at the right place to get around obstacles. Setting AIDE_VISEE = False in graphwar/config.py removes this aid, which makes training much longer.

Reward. +1 for hitting the target, +0.7 for hitting another enemy, −1 for hitting an ally (−3 for Peace). A miss gets a small bonus that grows as the shot passes closer to the target. Each training situation is a random map with 0–2 allies and 1–3 enemies.

Starting point. The PPO bots start from fully random weights and hit only 6–10 % of their shots. The evolutionary bots start from random weights with a small last layer, which is the usual initialization in RL, so they begin by shooting almost straight and hit 23–27 %, just below Dot's 31 %.

Training

Everything was trained on one desktop CPU (Intel i7-7700K, 4 cores / 8 threads, no GPU), using about 6 worker processes to simulate shots. After every 5 iterations, each bot is measured on 300 fixed maps it never trains on, and the best brain is kept.

• PPO bots improve fast: about 3 minutes each for 300 iterations. 3D reaches the level of the hand-written path planner Locus in about 100 iterations.
• Evolutionary bots barely move in 400 generations (4–8 minutes each). They stay around Dot's level.

Since the evolutionary bots had stalled, I then let them train overnight for about 9 hours, all three in parallel.

Fly (cross-entropy method) climbed above Dot within the first hour, then plateaued around 37 %. Pole-Zero and Evolution stayed flat at 25–30 % the whole night, even after 11,000–22,000 more generations.

Results

Hit rate on 1,000 brand-new maps

These maps were used neither for training nor for choosing the best brain.

Bot	Shots that hit an enemy	Shots that hit an ally
Locus (rule)	71.6 %	0.4 %
3D	71.2 %	0.1 %
Irrational	43.9 %	1.6 %
Peace	39.5 %	2.0 %
Fly	34.1 %	3.0 %
Pole-Zero	29.3 %	3.5 %
Evolution	28.8 %	3.6 %
Dot (rule)	25.6 %	3.5 %

Tournament

Every bot plays every other bot 30 times (3 vs 3, sides swapped every game), for 210 games per bot.

Rank	Bot	Wins	Draws	Losses	Teamkills
1	3D	182	0	28	0
2	Locus	177	1	32	9
3	Irrational	124	5	81	8
4	Peace	114	4	92	15
5	Fly	62	9	139	38
6	Pole-Zero	62	5	143	26
7	Dot	57	7	146	20
8	Evolution	44	5	161	44

3D finishes first overall and never shot an ally. Head to head, Locus still beats it 16–14.

What I learned

• The third dimension changes everything. With the same method (PPO) and a similar brain, 3D hits 71 % of its shots while Irrational hits 44 %. Jumping over low obstacles is much easier than finding a curve that goes around them.
• PPO learned in 3 minutes what evolution did not learn in 9 hours. PPO judges each shot separately, and its critic network predicts how hard each situation is. Evolutionary methods only see one average score per brain, and with this much randomness between maps, luck often matters more than the small differences between two brains.
• Records are optimistic. Fly's best score on the 300 control maps was 41.7 %, but it scores 34.1 % on new maps. The best of thousands of noisy measurements always benefits from some luck, which is why the tables above use fresh maps.
• Punishing teamkills was not enough for Peace. It sees its allies and is punished three times harder for hitting them, yet in matches it hit more allies than Irrational.

Limitations

• These are simplified rules, not the real Graphwar. The real game's functions, scales and physics differ.
• The aiming aid makes the task much easier than in the real game, where a player has to find the direction alone.
• The bots only choose one shot at a time. They do not plan over several turns.
• The code comments and the in-game text are in French.

Files

Path	Content
checkpoints/<bot>.npz	trained weights, in the format loaded by the code
checkpoints/<bot>.safetensors	the same weights, readable without this code (see below)
logs/<bot>.csv	training curves: iteration, seconds, training reward, control reward, hit rates, ally hits
graphwar/	engine, sensors, function families, bots, learning methods
arena.py	the pygame arena (challenge, duel, live demo)
train.py	training from scratch, --resume, --sans-limite (runs until Ctrl+C)
tournoi.py	headless tournament
courbes.py	plots the training curves
*.bat	Windows shortcuts

Usage

git clone https://huggingface.co/Foxgrad/graphwar-arena
cd graphwar-arena
pip install -r requirements.txt

python arena.py                      # the 3D bot challenges the other 7
python arena.py duel evolution locus # endless duel between two bots
python tournoi.py                    # everyone against everyone, then a ranking
python train.py all                  # retrain the 6 learning bots from scratch

PyTorch is only needed to train the PPO bots. Playing and evaluating only need numpy and pygame.

Using the weights without this code

Each .safetensors file contains the policy (and, for PPO bots, the critic) as plain matrices, the input normalization, and metadata: bot name, method, function family, sensors and layer sizes.

import numpy as np
from safetensors.numpy import load_file

w = load_file("checkpoints/3d.safetensors")

def policy(obs):
    """obs: the observation vector built by graphwar/sensors.py (173 values for 3D)."""
    x = np.clip((obs - w["input_norm.mean"]) / w["input_norm.std"], -5, 5)
    i = 0
    while f"policy.{i}.weight" in w:
        x = x @ w[f"policy.{i}.weight"] + w[f"policy.{i}.bias"]
        if f"policy.{i + 1}.weight" in w:
            x = np.tanh(x)
        i += 1
    return np.tanh(x)   # numbers in [-1, 1], turned into an equation by graphwar/vocab.py

License

The code and weights in this repository are released into the public domain under the Unlicense: use them, modify them, sell them, no conditions and no attribution required. Graphwar itself is the work of Lucas Catabriga Rocha and is distributed under GPL-3.0. None of its code is included here. The bot names and the 3D-bot idea come from Onion Das's video, which is credited above.