USING OODA TO WRITE BATTLER AI

Write battler AI that tries its best to minimize the chance for the players to win

  • DoubleX
  • 09/11/2015 04:08 PM
  • 3149 views
First, let's cite what OODA is:
- OODA loop
- The Tao of Boyd: How to master the OODA Loop
- Unlocking the power of Colonel John Boyd's OODA Loop

After reading all those, you, as a game developer, might wonder: How can we use OODA to write battler AI? Let's start with exploring the following oversimplified version:
(The below assumes that the battler AI are built to minimize the chance for the players to win. The OODA will be used slightly differently if that's not the battler AI's ultimate goal.)

Observe
To observe is to receive as many relevant info as accurately(including fake info detections by verifications) as possible. However, what can be observed by a battler?
(For the sake of writing battler AI, issues with fake info can be safely ignored as all relevant info can always be accurately received by any RMVXA battler AI.)
I think that any normal battler should be able to observe the opponents' current party/troop formations(like consisting of which actors/enemies), all their executed actions, and the allies' current statuses(like having which states/buffs/debuffs and how many hp/mp/tp).
Sometimes it could, however, be unfair for a battler to be able to observe the opponents' current capabilities(like classes, equips, parameters, extra parameters, special parameters, level and skill lists) or carried items.

For example, at the start of a battle, a boss facing a 6 actor party might be able to observe that all those actors are:
- heavily geared towards dealing damage with significant sacrifice from every other battle aspect, and/or
- mages only excelling at using magics, and/or
- fighters only excelling at using physical moves, and/or
- totally unresistant to some states/debuffs, and/or
- drastically faster than that boss(in action battle system, active time battle, charge turn battle, tactical battle system, etc), and/or something else.

Orient
To orient is to use mental models to make sense of those observed info(including judging the underlying implications and meanings) to fathom the current situations to generate working action plans. This process are affected by the following factors:
(For the sake of battler AI, mental models means the complete frameworks consisting of all the known strategies and tactics with all their known counters in the addressed RMVXA games that can be used by the allies, opponents and the battlers themselves.)
- New Information
- Previous Experience
- Cultural Traditions
- Genetic Heritage
- Analysis and Synthesis
For the sake of writing battler AI, cultural traditions and genetic heritage can be regarded as a battler's characters if they've to be taken info account, or simply ignored if they don't.
About the rest:
- New information must always be taken into account. Otherwise the battler AI will fail to realize that the situations have changed, let alone adapting to them(or better yet, controlling them).
- Previous experience, which will be fed by the new information, must always be taken into account too. In terms of battler AI, it means the AI will have to store all the relevant info in the current battle so far. Sometimes the full pictures, like the opponents' strategies, can only be accurately formed by combining the previous experience with the new information.
- Analysis means breaking down the existing mental models into smaller pieces, while synthesis means using those smaller pieces to form new mental models that can accurately address the new situations. This implies that the battler AI should be written to include as many known strategies and tactics with as many of their counters as possible, and they should all be able to be broken down into independent modular pieces that can be integrated into new strategies and tactics with their own counters. It also means that the battler AI should be able to judge which pieces are called for and run the integrated strategies and tactics by combining all those pieces.

For example:
- At the start of a battle, a boss fighting a 6 actor party observed that they're all heavily geared towards dealing damage with significant sacrifice from every other battle aspect.
By recalling all the known strategies and tactics with their counters, the boss AI will realize that it's extremely likely that that 6 actor party will try to kill the boss as quickly as possible(strategies) by having as high damage throughput as possible(tactics).
The boss AI will then use all the known strategies and tactics to generate all the ones that can counter the players' ones.
- In action battle system, active time battle, charge turn battle, tactical battle system, or any other battle system having the time dimension, a boss noticed that some actors just halted momentarily even when they should have acted instead(whether they should act instead needs orientations to tell) in some situations.
The boss AI will know that it's highly probable that the players aren't good at addressing those situations yet.
The boss AI will then use all the known strategies and tactics to generate all the ones that can make those situations happen more frequently.

Orientation's the most important part as it determines how accurate the observations will be interpreted and how useful the generated solutions will be executed.
For the sake of the battler AI, orientation's also the hardest part to be implemented well as it needs to stores all the known strategies and tactics with all their known counters, and the algorithms decoding the opponents' ones and coming up with all the working counters.

Decide
To decide is to judge which of all the generated action plans will be the best calls for the addressed situations which are already accurately fathomed. All these judgments should be treated as the best hypothesis which are to be tested rather than absolutely correct choices.

For example:
- At the start of a battle, a boss fighting a 6 actor party figured out that the party tried to kill the boss as quickly as possible(strategies) by having as high damage throughput as possible(tactics), and the boss AI has already generated all counters that can work.
The boss AI then forms the hypothesis that "it's better to mix several counters together to confuse the players by obfuscating the intentions" and decided to build action plans around that.

Act
To act is to build the action plans around the decisions and test those action plans by executing them(either simultaneously or treating the rest as backups that can always be quickly called). Then observations will be needed to check if those action plans worked and which ones work the best(and they'll be the main ones until the new best comes) in order to "finish"(and then restart) the "infinite" loop.

For example:
- A boss executes several counters together in a mixed manner to stop the players' strategies and tactics.
As the players realized they've been countered, they begin to adapt and use new strategies and tactics instead.
The boss AI noticed that the players are adapting to the new situations so the boss reorient in order to form new hypothesis to build and execute new action plans that will work.

Tempo
When it comes to oppositions, the faster one can effectively and efficiently run the OODA loop(higher velocity) and the faster one can change its tempo(higher acceleration), the more advantageous one will usually be in general. By controlling that tempo well via changing it rapidly and surprisingly, it's even possible to get inside the others' OODA loops to consistently reset them to the observe phase so they'll be confused and have to passively react to the new situations(they'll eventually end up being paralyzed in observations or making rash actions that won't work at all), while the one gaining the upper hand can constantly have an accurate full pictures and active control to the new situations.

For example:
- At the start of a battle(In action battle system, active time battle, charge turn battle, tactical battle system, or any other battle system having the time dimension) having 2 bosses, they act whenever they become able to by making moves that don't need charging(time delay between attempting to make a move and actually making that move).
This possibly led to players think that "act whenever becoming able to" are those bosses' patterns and those players build their action plans around their hypothesis.
Then suddenly those bosses delayed for a long time even when they become able to act, and that potentially lures players into thinking that those bosses are trying to make moves with long charging times so they make moves(which need to charge) that can cancel opponents' charging moves.
But that's exactly what those bosses want the players to do so instead those bosses make some other moves that don't need charging when the players are charging their moves.
The players now become confused and have to observe the new situations to form new mental models that can accurately address the new situations, while those bosses can take advantage from the resets of the players' OODA loops.

Building such battler AI won't be easy nor simple even for AI professionals. The builders need to have a thorough understanding to the battle systems, the battles and all the possible battlers involved as well as a fluent command on programming AI in general. Nevertheless, I still think that such AI would be incredibly challenging and extraordinarily difficult to beat if it could ever be built.

P.S.: Bear in mind that the aforementioned are still just an oversimplified version of the original OODA, which would certainly be overkill in writing battler AI.

Posts

Pages: 1
Worth noting: This is the RPG equivalent of a fighting game boss that reads your input to block.

It's cute on paper but it is remarkably unsatisfying to play against. No one likes being 100%, perfectly countered with nothing they can do about it. Imagine a FPS where enemies are perfectly accurate, or--more relevantly--trying to beat a modern chess AI. It isn't fun. Don't put these in your game.

In a game that uses this, all you do by equipping the Fire Shield is make enemies target someone else. There's no point in making any character beefier than normal, because that makes enemies target the next ally down. You can't have frail healers, because they will be focused down. And above all else, unlike any action game where you can overcome these issues with good positioning and reaction time, in an RPG you are purely at the mercy of enemy attack rolls.

I strongly encourage anyone who read this and thought, "Wow! My game needs this!" to fully rethink why they are making a game in the first place.
author=LouisCyphre
Worth noting: This is the RPG equivalent of a fighting game boss that reads your input to block.

It's cute on paper but it is remarkably unsatisfying to play against. No one likes being 100%, perfectly countered with nothing they can do about it. Imagine a FPS where enemies are perfectly accurate, or--more relevantly--trying to beat a modern chess AI. It isn't fun. Don't put these in your game.

In a game that uses this, all you do by equipping the Fire Shield is make enemies target someone else. There's no point in making any character beefier than normal, because that makes enemies target the next ally down. You can't have frail healers, because they will be focused down. And above all else, unlike any action game where you can overcome these issues with good positioning and reaction time, in an RPG you are purely at the mercy of enemy attack rolls.

I strongly encourage anyone who read this and thought, "Wow! My game needs this!" to fully rethink why they are making a game in the first place.
Then I'd tweak the OODA application a bit: Instead of using it to beat the players, use it to entertain them, by making the battlers to try their best to do what the players expect them to do so the players will feel/think that they're controlling the battles.

For example, the battler AI observed that all party members will be healed from fire elemental damages but are especially vulnerable to ice elemental damages. Then the battler AI will realize that the players are expecting their enemies to deal fire elemental damages to them frequently.

To entertain the players, the battler AI can be designed to shift among fire, ice and some other elements(making the battlers to use that element to deal damage, to be healed from damages with that element, and to be especially vulnerable to the opposite element), in the way that the battlers appear to be randomly changing elements so the players will constantly have to react to those changes. But in fact, the battler AI is written in such a way that it'll always deal ice elemental damage to the players only if the actors have lots of hp, and it'll deal fire elemental damage to them if the actors are about to die. The perceived randomness can be achieved if there are lots of elements to shift.

That way, the players will consistently experience imminent dangers, yet they're actually safe(unless the players do want to be defeated) as now the battler AI wants to entertain the players instead of beating them.

But yeah, the original post explicitly stated that it assumes that the battler AI is to beat the players lol
author=LouisCyphre
It's cute on paper but it is remarkably unsatisfying to play against


I wonder. That's definitely true if you also make enemies stronger and/or more numerous. I could see the appeal of a game where you are the guy with 5-digit HP fighting four puny humans who happen to be clever. Though of course they have to be able to fail in predictable ways.
author=Hasvers
author=LouisCyphre
It's cute on paper but it is remarkably unsatisfying to play against
I wonder. That's definitely true if you also make enemies stronger and/or more numerous. I could see the appeal of a game where you are the guy with 5-digit HP fighting four puny humans who happen to be clever. Though of course they have to be able to fail in predictable ways.

Exactly. It's completely possible that the battles are designed in ways that no matter how well the enemy AI utilizes the OODA, players can still consistently defeat them if they can constantly make the best moves.
Obviously, perfect defense by the enemy is undesirable. However, I'd also just say that changing the probability for the enemy to properly respond is reasonable.

For example, Ralph has a Fire Shield equipped against a fire boss. Normally, boss should attack each player in a four-actor party 25% of the time. Instead of changing the time Ralph gets attacked to 0%, maybe instead 20%. If Ralph also has a Fire Helmet, maybe dock another 5% probability. After all, if the player is beefing one at the expense of others, he should be able to accept the changing odds...assuming he notices the subtlety in the first place. Plus, if the change is obvious and Ralph is always ignored because of the fire equipment, the player now has a blatant counter-strategy with equipment distribution.

Another example. Let's say the boss has stance tendencies. If the player decides to spam super attacks, the boss starts mixing in more defensive options after a while. Realizing that there seems to be an uptick in boss defense, the player switches to stat-effects to get beefy. Eventually the boss catches on and instead of more defense, mixes in more offense while the player is goofing off with silly stat-effects. Thus, we now have a battle of cat-and-mouse.

Anyway, I like this article and I wouldn't mind fighting a smarter enemy...but maybe not this smart.
author=the13thsecret
Obviously, perfect defense by the enemy is undesirable. However, I'd also just say that changing the probability for the enemy to properly respond is reasonable.

For example, Ralph has a Fire Shield equipped against a fire boss. Normally, boss should attack each player in a four-actor party 25% of the time. Instead of changing the time Ralph gets attacked to 0%, maybe instead 20%. If Ralph also has a Fire Helmet, maybe dock another 5% probability. After all, if the player is beefing one at the expense of others, he should be able to accept the changing odds...assuming he notices the subtlety in the first place. Plus, if the change is obvious and Ralph is always ignored because of the fire equipment, the player now has a blatant counter-strategy with equipment distribution.

Another example. Let's say the boss has stance tendencies. If the player decides to spam super attacks, the boss starts mixing in more defensive options after a while. Realizing that there seems to be an uptick in boss defense, the player switches to stat-effects to get beefy. Eventually the boss catches on and instead of more defense, mixes in more offense while the player is goofing off with silly stat-effects. Thus, we now have a battle of cat-and-mouse.

Anyway, I like this article and I wouldn't mind fighting a smarter enemy...but maybe not this smart.

This remind me the "chaotic" parts of OODA, as it also suggested that some probability and uncertainty should be added into the mix when making decisions(the D part), thus increasing variety and reducing the chance being deciphered.
Pages: 1