This question may be a duplicate of nested for loops/if statements, but please bear with me. I'm walking through the process of making a tower defense, and I will need to check if the enemy is within range of a tower. I could do some sort of physics circle, but I feel the best way is to check the distance of a tower for each enemy to see if it's within a radius. The thing is, if there's 500 'enemies' and 30 towers, that's 15,000 if statements per frame. Would this be performance heavy? The only way I know to make this any easier is to try these:
Only check every x frame (do the if loop 10 times a second instead of 100)
Do some guessing - nearby towers will use the same tower as theirs
Is there any other way to do any of this or am I on a good track?
Is it likely that all enemies will have moved each time you check? You could just recalculate each time an enemy moves, which towers are within the radius of that particular enemy, then update the list of enemies for just those towers that have either moved into the radius of the enemy, or out of it. You'd need to map each enemy to a list of towers, as well as each tower to a list of enemies.
Related
I am attempting to use Q-learning to learn minesweeping behavior on a discreet version of Mat Buckland's smart sweepers, the original available here http://www.ai-junkie.com/ann/evolved/nnt1.html, for an assignment. The assignment limits us to 50 iterations of 2000 moves on a grid that is effectively 40x40, with the mines resetting and the agent being spawned in a random location each iteration.
I've attempted performing q learning with penalties for moving, rewards for sweeping mines and penalties for not hitting a mine. The sweeper agent seems unable to learn how to sweep mines effectively within the 50 iterations because it learns that going to specific cell is good, but after a the mine is gone it is no longer rewarded, but penalized for going to that cell with the movement cost
I wanted to attempt providing rewards only when all the mines were cleared in an attempt to make the environment static as there would only be a state of not all mines collected, or all mines collected, but am struggling to implement this due to the agent having only 2000 moves per iteration and being able to backtrack, it never manages to sweep all the mines in an iteration within the limit with or without rewards for collecting mines.
Another idea I had was to have an effectively new Q matrix for each mine, so once a mine is collected, the sweeper transitions to that matrix and operates off that where the current mine is excluded from consideration.
Are there any better approaches that I can take with this, or perhaps more practical tweaks to my own approach that I can try?
A more explicit explanation of the rules:
The map edges wrap around, so moving off the right edge of the map will cause the bot to appear on the left edge etc.
The sweeper bot can move up down, left or right from any map tile.
When the bot collides with a mine, the mine is considered swept and then removed.
The aim is for the bot to learn to sweep all mines on the map from any starting position.
Given that the sweeper can always see the nearest mine, this should be pretty easy. From your question I assume your only problem is finding a good reward function and representation for your agent state.
Defining a state
Absolute positions are rarely useful in a random environment, especially if the environment is infinite like in your example (since the bot can drive over the borders and respawn at the other side). This means that the size of the environment isn't needed for the agent to operate (we will actually need it to simulate the infinite space, tho).
A reward function calculates its return value based on the current state of the agent compared to its previous state. But how do we define a state? Lets see what we actually need in order to operate the agent like we want it to.
The position of the agent.
The position of the nearest mine.
That is all we need. Now I said erlier that absolute positions are bad. This is because it makes the Q table (you call it Q matrix) static and very fragile to randomness. So let's try to completely eliminate abosulte positions from the reward function and replace them with relative positions. Luckily, this is very simple in your case: instead of using the absolute positions, we use the relative position between the nearest mine and the agent.
Now we don't deal with coordinates anymore, but vectors. Lets calculate the vector between our points: v = pos_mine - pos_agent. This vector gives us two very important pieces of information:
the direction in which the nearst mine is, and
the distance to the nearest mine.
And these are all we need to make our agent operational. Therefore, an agent state can be defined as
State: Direction x Distance
of which distance is a floating point value and direction either a float that describes the angle or a normalized vector.
Defining a reward function
Given our newly defined state, the only thing we care about in our reward function is the distance. Since all we want is to move the agent towards mines, the distance is all that matters. Here are a few guesses how the reward function could work:
If the agent sweeps a mine (distance == 0), return a huge reward (ex. 100).
If the agent moves towards a mine (distance is shrinking), return a neutral (or small) reward (ex. 0).
If the agent moves away from a mine (distance is increasing), retuan a negative reward (ex. -1).
Theoretically, since we penaltize moving away from a mine, we don't even need rule 1 here.
Conclusion
The only thing left is determining a good learning rate and discount so that your agent performs well after 50 iterations. But, given the simplicity of the environment, this shouldn't even matter that much. Experiment.
I want to do a project, which will consist in detecting possible collision of the pool balls, using opencv, webcam and C++ programming language. For now I just want to prediction collision of 2 balls on minibilard table. I detect them by change rgb to hsv and then use thereshold, in future i will probably use another method for detect a random amount of balls, but it's not so important now.
So, for now I can detect two balls, i know their position, radius, now I'm thinking how to predict whether there will be a collision between them, if we if we assume that they will move in straight lines. I think that I should check their position in every frame update (and i have to know a time between frames in my webcamera) and by that, i I will be able to determine the value of speed, acceleration and direction of the ball. So, if i will know those parameters for for both balls, I will be able to determine where can they collide, and then, using parametric equastion I will be able to check, if they will be on collision point on the same time.
I wonder if this is the right approach to the problem, maybe there is a simpler and more effective method to do this?
Thanks for any kind of help.
Karol
This sounds like you are on track for a good project...
Calculating acceleration seems, from what I briefly read here, reasonably difficult though. So as a preliminary step, you could just assume a constant velocity. So take the difference between a balls position last frame and current frame as a vector and add it on to the current frames position to find where it will be next frame. Doing this for both balls will allow you to check for a collision.
You can check for a collision by comparing the distance between the balls centers using Pythagoras to the sum of their radii. If the sum of their radii will be greater than the distances between their centers, you have a collision.
Obviously, calculating one frame ahead is not very useful, but if you assume a constant velocity or manage to calculate their acceleration, there is no reason to why you can't calculate 30 or 100 frames in the future with this method.
I recently made a billiards ball simulation in javascript which you could take a quick look at here if you want to see how this could work.
I want to make a non-real time simulation of overlapping repulsive balls, initially in 2D and later in 3D.
First of all, consider a simple closed domain for simplicity. In practice, domains will be complex and irregular but always closed.
The balls on the boundaries are fixed and could be overlapping. A fixed ball duplicates itself to produce a free ball of the same size whenever no other ball overlaps it. Both fixed and free balls repel each other but fixed balls cannot move. Note that, duplicant ball should be sufficiently tilted to start repulsion. In elastic colliding balls case, after two balls collide they change direction with some velocity but in this case the balls can stop quickly once they stop overlapping. Free balls move until there is no motion or let's say we solve motion problem until convergence. Then each fixed ball produce a free ball again and this process goes on until no fixed ball can duplicate due to being overlapped by any other ball(s).
I think GPU (CUDA) would be faster to solve this problem but initially I am thinking to write on CPU. However, before proceeding to coding I would like to know "feasibility" of this work. That is, considering a million of balls, approximately how long it would take to simulate this or similar kind of problems in non-real time. For a million of balls, if solution time is in order of minutes, I will dive into the problem.
You might look into using Box2D for a prototype. Setting up your collision constraints to be 'soft' would give you about the kind of behavior you're showing in your diagrams.
As for simulating a million objects in real time, you're going to be working on a GPU.
I am making a 2D game and I have created two circles, one named player and one named enemy. The circles are drawn by calling a drawPlayerCircle and drawEnemyCircle function, both which are called in my display() function. At the moment my project isn't Object Orientated which you can already probably tell. My collision works for both circles as I have calculated penetration vectors and normalised vectors etc... However my question is, expanding this game further by having 10-20 enemies in the game, how can I work out the collision to every single enemy quickly rather than calculating manually the penetration and normalised vectors between my player and every enemy in the game which is obviously very inefficient. I'm guessing splitting my player and enemy entities into their own class but I'm still struggling to visualise how the collision will work.
Any help will be greatly appreciated! :)
Collision between two circles is very quick to test. Just check if the distance between the circle centre points is less than the sum of the two radii.
I would just start by comparing the player against every other enemy. Doing 20 comparisons is not going to tax any computer from the last 10 years. I'm a big believer in just writing the code that you need right now and not complicating things, especially when you are just starting out.
However if you end up having millions of circles and you've profiled your code and found that testing all circles is slow, there are quite a few options. The simplest might be to have a grid over the play area.
Each grid cell covers a fixed amount of space over the play area and stores a list of circles. Whenever a circle moves calculate which grid cell it should be in. Remove it from the grid cell that it is currently in, and add it to the grid cell that it should be in. Then you can compare only the circles that overlap the player's grid cell.
Space partitioning. Specifically, if you're in 2D, you can use a quadtree to speed up collision detection.
I need some some help. I'm working on game, and have problem with bullets trajectory. I have enemies and have player. Enemies and player can be positioned in random positions on screen. So when enemies are shooting in player, i need to have some formula to know the bullet trajectory. Bullet must move from enemy to player and move to out of screen bounds, in the same trajectory, like in others shooters. Can some one help me with this? Thanks.
Whilst I can't give you the code in the language you're looking for (don't have it to hand atm), I can explain the process. This requires a bit of trigonometry. Essentially, the most basic thing to do here is calculate the correct X and Y speed of the bullet to simulate "locking on". That can be achieved by working out the x and y distance between the player and enemy, the calculating (most likely using trig) the angle at which to fire, and then basing the xSpeed and ySpeed of the bullet on that angle.
When I first learned how to do this, I overlayed a triangle on top of the enemy which would indicate whether my trigonometry was correct by pointing directly to the player.
Hopefully understanding this process will give you a means to approach it!
There are also a lot of actionscript 3 tutorials on this which you could translate the logic from.
Good luck!