I'm coding an AI engine for a simple board game. My simple implementation for now is to iterate over all optional board states, weight each one according to the game rules and my simple algorithm, and selecting the best move according to that score.
As the scoring algorithm is totally stateless, I want to save computation time by creating a hash table of some (all?) board configurations and get the score from there instead of calculating it on the fly.
My questions are:
1. Is my approach logical? (and if not, can you give me some tips to better it? :))
2. What is the most suitable thread-safe STL container for my needs? I'm thinking to use the char array (board configuration) as key and the score as value.
3. Can you give some tips for making my AI a killer one? :)
edit: more info:
The board is 10x10 and there are two players, each with 10 pawns. The rules are much like checkers.
Yes it's common to store evaluated boards into a hash table it's called transposition table. A STL container could be std::vector. In general you have to create a hash function (e.g. zobrist hashing). The hash function calculates a hash value of a particulare board. The result of hash_value modulo HASH_TABLE_SIZE would be the index to the std::vector.
A transposition table entry can hold more information than only board-score and best-move, you can also store to which depth the board is evaluated and if the evaluated-score (in case you are doing alpha-beta search) is
exact
a Upper Bound
or Lower Bound
I can recommend the chessprogramming site, where I have learned a lot. Look for the terms alpha-beta, transposition table, zobrist hashing, iterative deepening. There are also good papers for further reading:
TA Marsland - Computer Chess and Search
TA Marsland - A Review of Game Tree Pruning
AL Zobrist - A New Hashing Method with Application for Game Playing
J Schaeffer - The games Computers (and People) Play
Your logical approach is ok, you should read and maybe try to use Minimax algorithm :
http://en.wikipedia.org/wiki/Minimax
I think that except from tic tac toe game the number of states would be much too big, you should work on making the counting fast.
Chess and checkers can be done with this approach, but it's not one I'd recommend.
If you go this route then I would use some form of tree. If you think about it, every move reduces the total possibilities that existed before the move was made. Plus, this allows levels of difficulty. Don't pick the best all the time, sometimes pick second best.
The reason I wouldn't go this route is that it's not generally fun. People pick up on this intuitively and they feel it's unfair. I wrote a connect 4 game that was unbeatable, but was rule based rather than game board state based. It was dull. Every move was met with the same response. I think this is what happens in this approach as well. Also, it depends on why you are doing this. If it's to learn AI, very little AI is done like this. If it's to have a fun game, it usually isn't. If it's for the reasons Deep Blue was made, to stretch the limits of what a computer can do, then sure.
I would either use a piece based individual AI and then select the one with the most compelling argument or I would use a variation of hill climbing and put a kind of strategy height into the board. It depends on how much support pieces give one another. For the individual AI I would use neural nets.
A strategy height system would be good for an FPS where soldiers want to know what path has the most cover. Neural nets give each entity more personality. You can even use cascading neural nets where one is the strategy and the second is the personality.
Related
I'm working on the game in C++. Basically, the game is going to be a simulation of the world with some animals, plants, and of course a player itself (Animals move in random directions). I wanna do it to practice OPP, algorithm library, and work a little with collections from STL. I want to mention that I'm rewriting this project, and there is only one concern that keeps bothers me - Checking collisions. The game is going to be two-dimensional (only x,y coordinates).
Because if I keep my organisms in vector it's going to take a linear time to find if there are other organisms on these coordinates. When there is more organisms, this search takes a lot of time (seriously, for 400 animals their turn could take 20-30 seconds). So my question is, which STL collection would fit my problem best. Thank you in advance for your help.
Animals may die, their strength may be modified, and they can also create offsprings.
I am trying to implement a minimax algorithm for an AI player within a simple card game. However, from doing research I am confused what are the key differences between state evaluation and heuristics.
From what I understand heuristics are calculated by the current information available to the player (e.g. in chess, the pieces and their relevant locations). With this infomation, they come to a conclusion based on a heuristics function which essentially provides a "rule of thumb".
A state evaluation is the exact value of the current state.
However I am unsure why both things co-exist as I cannot see how they are much different from one another. Please can someone ellaborate, and clear up my confusion. Thanks.
Assuming a zero-sum game, you can implement a state-evaluation for end-states (game ended with win, draw, loss from perspective of player X) which results 1,0,-1. A full tree-search will then get you perfect-play.
But in practice the tree is huge and can't be searched completely. Therefore you have to stop the search at some point, which is not an end-state. There is no determined winner or loser. Now it's hard to mark this state with 1,0,-1 as the game might be too complex to easily evaluate the winner from some state far away from the end-state. But you still need to evaluate this positions and can use some assumptions about the game, which equals to heuristic-information. One example is piece-mass in chess (queen is more valuable then a pawn). This is heuristic information incorporated into the non-perfect evaluation-function (approximation of the real one). The better your assumptions / heuristics, the better the approximation of the real evaluation!
But there are other parts where heuristic information can be incorporated. One very imporant area is controlling the tree-search. Which first-move will be evaluated first, which last. Selecting good moves first allows algorithms like alpha-beta to prune huge parts of the tree. But of course you need to state some assumptions/ heuristic-information to order your moves (e.g. queen-move more powerful than pawn-move; this is a made-up example, i'm not sure about the effect of this heuristic in chess-AIs here)
I'm currently trying to find good parameters for my program (about 16 parameters and execution of the program takes about a minute). Evolutionary algorithms seemed like a nice idea and I wanted to see how they perform.
Unfortunately I don't have a good fitness function because the variance of my objective function is very high (I can not run it often enough without waiting until 2016). I can, however, compute which set of parameters is better (test two configurations against each other). Do you know if there are evolutionary algorithms that only use that information? Are there other optimization techniques more suitable? For this project I'm using C++ and MATLAB.
// Update: Thank you very much for the answers. Both look promising but I will need a few days to evaluate them. Sorry for the delay.
If your pairwise test gives a proper total ordering, i.e. if a >= b, and b >= c implies a >= c, and some other conditions . Then maybe you can construct a ranking objective on the fly, and use CMA-ES to optimize it. CMA-ES is an evolutionary algorithm and is invariant to order preserving transformation of function value, and angle-preserving transformation of inputs. Furthermore because it's a second order method, its convergence is very fast comparing to other derivative-free search heuristics, especially in higher dimensional problems where random search like genetic algorithms take forever.
If you can compare solutions in a pairwise fashion then some sort of tournament selection approach might be good. The Wikipedia article describes using it for a genetic algorithm but it is easily applied to an evolutionary algorithm. What you do is repeatedly select a small set of solutions from the population and have a tournament among them. For simplicity the tournament size could be a power of 2. If it was 8 then pair those 8 up at random and compare them, selecting 4 winners. Pair those up and select 2 winners. In a final round -- select an overall tournament winner. This solution can then be mutated 1 or more times to provide member(s) for the next generation.
In our program we use a genetic algorithm since years to sole problems for n variables, each having a fixed set of m possible values. This typically works well for ~1,000 variables and 10 possibilities.
Now i have a new task where only two possibilities (on/off) exist for each variable, but i'll probably need to solve systems with 10,000 or more variables. The existing GA does work but the solution improves only very slowly.
All the EA i find are designed rather for continuous or integer/float problems. Which one is best suited for binary problems?
Well, the Genetic Algorithm in its canonical form is among the best suited metaheuristics for binary decision problems. The default configuration that I would try is such a genetic algorithm that uses 1-elitism and that is configured with roulette-wheel selection, single point crossover (100% crossover rate) and bit flip mutation (e.g. 5% mutation probability). I would suggest you try this combination with a modest population size (100-200). If this does not work well, I would suggest to increase the population size, but also change the selection scheme to a tournament selection scheme (start with binary tournament selction and increase the tournament group size if you need even more selection pressure). The reason is that with a higher population size, the fitness-proportional selection scheme might not excert the necessary amount of selection pressure to drive the search towards the optimal region.
As an alternative, we have developed an advanced version of the GA and termed it Offspring Selection Genetic Algorithm. You can also consider trying to solve this problem with a trajectory-based algorithm like Tabu Search or Simulated Annealing that just uses mutation to move from one solution to another by just making small changes.
We have a GUI-driven software (HeuristicLab) that allows you to experiment with a number of metaheuristics on several problems. Your problem is unfortunately not included, but it's GPL licensed and you can implement your own problem there (through just the GUI even, there's a howto for that).
Like DonAndre said, canonical GA was pretty much designed for binary problems.
However...
No evolutionary algorithm is in itself a magic bullet (unless it has billions of years runtime). What matters most is your representation, and how that interacts with your mutation and crossover operators: together, these define the 'intelligence' of what is essentially a heuristic search in disguise. The aim is for each operator to have a fair chance of producing offspring with similar fitness to the parents, so if you have domain-specific knowledge that allows you to do better than randomly flipping bits or splicing bitstrings, then use this.
Roulette and tournament selection and elitism are good ideas (maybe preserving more than 1, it's a black art, who can say...). You may also benefit from adaptive mutation. The old rule of thumb is that 1/5 of offspring should be better than the parents - keep track of this quantity and vary the mutation rate appropriately. If offspring are coming out worse then mutate less; if offspring are consistently better then mutate more. But the mutation rate needs an inertia component so it doesn't adapt too rapidly, and as with any metaparameter, setting this is something of a black art. Good luck!
Why not try a linear/integer program?
First off, I am a noob. I am also a Janitor that has never made a dime writing code. This is just something that I love doing. It is for fun:) That being said, I wrote this console based tic-tak-toe game that has enough ai to not lose every game. (I guess ai is what it should be called.) It has something like 70 if/else if statements for the computers turn. I used 3 int arrays like so:
int L[2], M[2], R[2];
0 = blank;
1 = X;
2 = O;
The board then 'Looks' likeL[0] | M[0] | R[0]L[1] | M[1] | R[1]L[2] | M[2] | R[2]
So I basically wrote out every possible scenario I could think something like:
if(M[0]==1 & M[1]==1 & M[2]==0){M[2] = 2;}//here the computer prevents a win
else if(L[0] ==2&M[1]==2&R[2]==0){R[2]=2;}//here the computer wins
//and so on....68 more times!
I guess my question(s) is(are): Is there a better way? Is there a way to achieve the same result with less lines of code?Is this considered Artificial Intelligence?
The standard algorithm for this is called Minimax. It basically builds a tree, where the beginning of the game is the root, and then the children represent every possible move X can make on the first turn, then the children of each of those nodes are all the moves O can make in response, etc. Once the entire tree is filled (which is possible for Tic-Tac-Toe, but for games like Chess computers still don't have enough memory), you work your way back up, assuming both players are smart enough to make the best move, and arrive at the optimal move. Here is another explanation of Minimax specifically using Tic Tac Toe as an example.
The Wikipedia page on Tic-Tac-Toe has a very good algorithm outline for winning (or tying) every game:
http://en.wikipedia.org/wiki/Tic-tac-toe which is what I used to make a Tic-Tac-Toe game several years ago.
After you understand the algorithm, one of the cleverest ways to implement a Tic-Tac-Toe computer player is with a magic square. The method is discussed here. As far as size goes, I've seen this implemented in about 50 lines of code, I'll post the code if I find it :)
This isn't technically artificial intelligence, as AI usually refers to artificial neurons, neuron layers, gradient descent, support vector machines, solving complex polynomials, and the like. Solving Tic-Tac-Toe
Yes there are better ways.
The most obvious would be to consider how different mirror views of the board would simplify the number of cases.
Also, consider pre-storing "interesting" patterns in arrays and then comparing the game state against the data. For example, one series of pattern would be all the ways a player can win in the next move.
Also, note that with the declaration int L[2], there are only two entries in array L, namely L[0] and L[1]. The references you have to L[2], M[2], etc. are errors that should have been caught by the compiler. Consider turning up the warning level. How this is done depends on the compiler. For gcc it is -Wall.
This counts as a form of artificial intelligence. The series of if statements are accumulated knowledge: how to recognize a situation and the appropriate best reaction to it.
The closest thing to real AI to solve such a game would be to code an artificial network and train it with all combinations of the tictactoe game.
In that case the code would not do so much if then else to solve the problem but would solve the problem by taking the most reasonable choice that solves the problem from a pattern trained in it.
But coding a neural network is not a trivial thing :)
When in need to code a rule-based system (like the AI you are building), you can use a rule engine, like for example CLIPS (which is a tool developed at NASA for creating Expert Systems written in C).
http://en.wikipedia.org/wiki/CLIPS
Perhaps it's an overkill for playing Tic Tac Toe, but if you are in the mood for learning cool AI stuff, Expert Systems is a very interesting area, but different (and perhaps less trickier) than Neural Networks.
Have fun!