Current AI image models can generate all sorts of different images without a problem, but they won't be able to handle extremely niche things, like a Worms map.
But things are different with local models (the ones that you can run on your own PC). You can create small extra datasets called LoRAs to teach them new concepts (a person/character, an artstyle, an object, etc). Image models equipped with a LoRA will be able to generate images containing the LoRA concept.
So I thought, why not try to create a BnG LoRA!
Technical details- Base model: FLUX.1-dev
- Training UI: Flux Gym
- Training dataset: 23 BnG maps (converted to PNG and resized to 1920x720) from this map pack.
- Captioning template: {terrain_name} themed BnG map
- Training duration: 2h11m
- Training settings: Default Flux Gym settings, except:
- Trained for 8 epochs.
- Image size of 720px.
- PC Specs:
- GPU: RTX 4070 Ti SUPER 16 GB
- RAM: 64GB DDR5-6000 CL30
- CPU: Ryzen 5 7600X
ResultsLoRA model succeed at some things, failed at others.
Here's four different generations using the prompt "forest themed BnG map":
As you can see, the first problem is that it does not use a black background for maps.
Adding "black background", 'black sky", or"'pitch black sky" will sometimes fix these problems, but not always. Occasionally, it also darkens the terrain:
Now, we have a f*ck load of forest maps, so how about other terrains? Better yet, how about non-existent terrains? It was able to generate BnG maps with unique terrains, but it occasionally generated weird-looking terrain objects:
Potential ways to improve the model- Increasing the number of images in the training dataset.
- Diversifying training dataset with more maps with different terrains, including custom terrains.
- By increasing the diversity, the model could become better at generalizing (i.e. it can become better at generating maps with non-existent terrains, rather than only being good at generating maps similar to those in its dataset, such as forest maps).
- Using more detailed captions for training dataset images.
- This might solve the non-black background problem, and could potentially give more control over terrain objects.
Closing remarksOverall, this was a fun experiment. Results may not be perfect, but there is one thing we should keep in mind: This is the worst AI will ever be. This tech will keep improving. Who knows, maybe we'll be able to automate map generation for every scheme one day.
