Circle - Axis Alined Rectangle Intersection - c++

I want to create a function which takes a rectangle and a circle returns a boolean as to whether or not they intersect. What's the most efficient and simple way of doing this? The function looks something like this:
bool intersect(rectX, rectY, rectWidth, rectHeight, circleX, circleY, radius)
{
bool intersect;
//code I need
return intersect;
}
Please help me find the code I need. Thanks!

Your answer is at last, but I also want to inform you/future viewer, the following.
When you are denoting a rectangle by one point AND height-width that is not specific rectangle on axis-system, that is general form of a rectangle can be draw anywhere.
So, In your case: Rectangle can be draw in any direction(to make it away from circle, eg: UP,LEFT,RIGHT,BOTTOM etc. side to rectX,rectY). That's why here two possibilities generates AND your possibility will depends on what you need:
[A] 100% assurance of intersection, no matter how you draw rectangle
[b] At least one way to draw the rectangle so that it will intersect with circle
Case A:
bool assuredIntersect(rectX, rectY, rectWidth, rectHeight, circleX, circleY, radius){
bool intersect;
float distance=((rectX-circleX)^2+(rectY-circleY)^2)^0.5;
intersect=(radious>=distance);
return intersect;
}
Case B:
bool canIntersect(rectX, rectY, rectWidth, rectHeight, circleX, circleY, radius){
bool intersect;
float distance=((rectX-circleX)^2+(rectY-circleY)^2)^0.5;
float diagonal=((rectX+rectHeight)^2+(rectY-rectWidth)^2)^0.5;
intersect=((radious+diagonal)<distance);
return intersect;
}
But, if you are denoting rectangle by two-points-with-one-side (rectX1,rectY1 AND rectX2,rectY2 AND height or width). Then you can denote a rectangle specifically.
Note: If direction of rectangle is fixed(like shorter-side-from-point(rectX,rectY) is perpendicular-or-at-an-angle to x-axis) then also rectangle become specific as we can calculate (rectX2,rectY2). Eg: if angle is 90 then second point will be (rectX+rectHeight, rectY+rectWidth).
If we have function parameter like this:
#include <math.h>
#define PI 3.14159265
bool intersect(rX1, rY1, rX2, rY2, rAngle, cX, cY, cR){
//can be `intersect(rX1,rY1,rH,rW,rAngle, cX,cY, cR)`, and calculate rX2,rY2
//can be `intersect(rX1,rY1,rX2,rY2,rH, cX,cY, cR)`, and calculate rAngle
bool intersect;
//assume (rX1,rY1) as origin AND rectangle`s-side attached to this point is on both axis,
//THEN we need to recalculate coordinates according to this assumption
rAngle=rAngle*PI/180; //angle in radian
//NOTE: if in place of rAngle, rHeight or rWidth is given then you can calculate rAngle by trigonometry.
//moving rX1,xY1 to (0,0)
cX=cX-rX1; cY=cY-rY1;
rX2-=rX1; rY2-=rY1; rX1=rY1=0;
//rotating axis, rectangle, circle...
float cosA=cos(rAngle), sinA=sin(rAngle);
float tempX= cosA*rX2 + sinA*rY2;
float tempY= sinA*rX2 + cosA*rY2;
rX2=tempX; rY2=tempY;
tempX=cosA*cX + sinA*cY;
tempY=sinA*cX + cosA*cY;
cX=tempX; cY=tempY;
rX1-=cR;rY1-=cR; //enlarging(creating) virtual rectangle around original; After this...
rX2+=cR;rY2+=cR; //...if circle centre is inside this rectangle it will intersect with original rectangle
intersect=(cX<=rX2 && cX>=rX1 && cY<=rY2 && cY>=rY1);
return intersect;
}
point(x,y) rotation
# So, if you don't know angle then you can consider first two case.
[ANSWER] If rectangle is axis aligned then function will be:
bool intersect(rectX, rectY, rectWidth, rectHeight, circleX, circleY, radius){
bool intersect;
//calculating rX2,xY2
rX2=rectX + rectWidth; rY2=rectY + rectHeight;
rectX-=radius;rectY-=radius; //enlarging(creating) virtual rectangle around original; After this...
rX2+=radius;rY2+=radius; //...if circle centre is inside this rectangle it will intersect with original rectangle ...
intersect=(circleX<=rX2 && circleX>=rectX && circleY<=rY2 && circleY>=rectY);
return intersect;
}

Related

The collision in SFML is not that good, how to improve it?

I've been lately working on a simple game using C++ and SFML latest version, but I had a problem which is that the collision detection is not that good, for example the player dies even if the enemy didn't touch him yet, but just near him. Here is the code of the player class with the move function and collision detection code AND the moves of the enemy class:
`class PlayerA : public CircleShape
{
public:
//Constructor:
PlayerA(float xposition, float yposition, float radius, float s)
{
setRadius(radius);
setFillColor(Color::Yellow);
setOutlineColor(Color(00,80,00));
setOutlineThickness(-2);
setPointCount(3);
setSpeed(s);
setPosition(xposition,yposition);
}
//Movements of the player:
void up()
{
move(0,-10*speed);
}
void down()
{
move(0,10*speed);
}
void right()
{
move(10*speed,0);
}
void left()
{
move(-10*speed,0);
}
void checkA(ObsA *obs1=NULL,ObsA *obs2=NULL, ObsA *obs3=NULL, ObsA *obs4=NULL, ObsA *obs5=NULL)
{
if(obs2==NULL)
{
if(getGlobalBounds().intersects(obs1->getGlobalBounds()))
{
relevel();
}
}
private:
float speed=0.00;
void obs()
{
if(speed > 0)
{
rotate(0.5*speed);
}
else
{
rotate(0.5*speed);
}
}
private:
float speed = 0.00;
void obs()
{
if(speed > 0)
{
rotate(0.5*speed);
}
else
{
rotate(0.5*speed);
}
}
private:
float speed = 0.00;
Is there something wrong with the code, how to fix the problem, thank you!
The intersects function just check if two rectangles intersect. If you want pixel perfect collision detection in SFML you have to write that yourself.
Basically, start with intersects, if it is true, then get the intersecting rectangle and check if any pixels therein from both original rectangles contains overlaping relevant pixels.
You can use this function to perform better collision detection.Its a basic one but works well
bool circleTest(const sf::Sprite &first, const sf::Sprite &second)
{
sf::Vector2f firstRect(first.getTextureRect().width, first.getTextureRect().height);
firstRect.x *= first.getScale().x;
firstRect.y *= first.getScale().y;
sf::Vector2f secondRect(second.getTextureRect().width, second.getTextureRect().height);
secondRect.x *= second.getScale().x;
secondRect.y *= second.getScale().y;
float r1 = (firstRect.x + firstRect.y) / 4;
float r2 = (secondRect.x + secondRect.y) / 4;
float xd = first.getPosition().x - second.getPosition().x;
float yd = first.getPosition().y - second.getPosition().y;
return std::sqrt(xd * xd + yd * yd) <= r1 + r2;
}
Are you using a circle? If I remember correctly, the circle will have a rectangle hitbox. If that is the case, then you may have collision between the invisible rectangle corners.
If you're using a circle, Perhaps change class to a square rectangle and see if collision works correctly. Or try testing collision directly on an x or y axis with your circles; i.e. having them moving in a straight line towards each other only changing 1 axis. (the edge of the circle will be the same as the edge of the rectangle at the left, right, top, and bottom sections).
If you're needing a better collision for circles, there may be one already built in SFML. But I don't think it would be too much to write your own logic using the radius of your two circles, the center of your two objects, and the angle hypotenuse between the centers.
edit based on Merlyn Morgan-Graham's comment.

Project an object with rotation around another object in C++/OpenGL

I'm Using OpenGL/C++ to create a game.
One aspect of this game that I'm working on is having a character that shoots a projectile the way said character is facing. To do this I have a 'player' and a 'projectile'.
I pass to the projectile the characters x and y co-ordinates, the angle the player is facing. From this I want to shoot the projectile in that direction.
In my draw I am currently using glTranslate with the characters x and y and rotating the projectile on the way the character is facing. This moves my projectile to the way the player is facing.
glTranslatef(this->m_X, this->m_Y, 0);
glRotatef(angle, 0, 0, 1);
This is where i'm stuck, I can move the projectile position by incrementing/decrementing the X and Y values in the translate. But what I'm trying to ask is how can I move the projectile along the line the player is facing.
Thanks for the help!
You can use polar vectors for these calculations.
http://mathworld.wolfram.com/PolarVector.html
A polar vector will allow you to make several calculations that would normally be complicated and convoluted in a simple way. Using their applied mathematics your request won't be an issue.
Here's an implementation of mine of polar vectors.
The header file:
#include <cmath>
//Using SFML Vector2 class, making a similar class is easy.
//Check this URL: http://www.sfml-dev.org/documentation/2.3.2/classsf_1_1Vector2.php
class PolarVector
{
public:
float r;
float t; ///Angle stored in degrees.
PolarVector();
PolarVector(float radius, float angle);
PolarVector(const sf::Vector2f V2); ///Conversion constructor.
sf::Vector2f TurnToRectangular() const;
};
PolarVector TurnToPolar(const sf::Vector2f point);
float getConvertedRadius(const sf::Vector2f point);
float getConvertedAngle(sf::Vector2f point);
bool operator ==(const PolarVector& left, const PolarVector& right);
bool operator !=(const PolarVector& left, const PolarVector& right);
And the source file:
#include "PolarVector.hpp"
PolarVector::PolarVector()
:r(0.f)
,t(0.f)
{}
PolarVector::PolarVector(float radius, float angle)
:r(radius)
,t(angle)
{}
PolarVector::PolarVector(const sf::Vector2f V2)
:r(getConvertedRadius(V2))
,t(getConvertedAngle(V2))
{}
sf::Vector2f PolarVector::TurnToRectangular() const
{ return sf::Vector2f(static_cast<float>(r* std::cos(t)), static_cast<float>(r* std::sin(t))); }
PolarVector TurnToPolar(const sf::Vector2f point)
{
PolarVector PV;
PV.r = getConvertedAngle(point);
PV.t = getConvertedRadius(point);
return PV;
}
float getConvertedRadius(const sf::Vector2f point)
{ return std::sqrt((point.x * point.x) + (point.y * point.y) ); }
float getConvertedAngle(const sf::Vector2f point)
{ return std::atan2(point.y, point.x); }
bool operator ==(const PolarVector& left, const PolarVector& right)
{
float diffR = left.r - right.r;
float diffA = left.t - right.t;
return ((diffR <= EPSILON) && (diffA <= EPSILON));
}
bool operator !=(const PolarVector& left, const PolarVector& right)
{
float diffR = left.r - right.r;
float diffA = left.t - right.t;
return !((diffR <= EPSILON) && (diffA <= EPSILON));
}
The reason why I suggest this is because you can do the following.
Let's say you have a 2 dimensional vector:
sf::Vector2f character(0.f, 0.f); //Origin point. First parameter is X, second is Y
float angleCharFacesAt = 0.698132; //40 degrees in radians. C++ Trigonometry uses Radians. std::cos, std::sin and std::atan2 are used internally.
For the first object or character. You want the other object to have the same angle, but a different position.
Let's say the other object has a position above it:
sf::Vector2f object(0.f, 10.f); //Above the origin point.
float angleObjectFacesAt = 0.f; //0 degrees.
So all you need to do is rotate it using a polar vector:
PolarVector PV = TurnToPolar(object); //Use this for calculations.
PV.t += angleCharFacesAt; //t is the angle parameter of the polar vector.
object = PV.TurnToRectangular(object);
By doing this you will get the rotated position of the object.
The distance between one object and the other will always be the r (Radius) value of the polar vector. So you could make the distance longer or shorter by doing this:
PolarVector PV = TurnToPolar(object); //Use this for calculations.
PV.r += 10; //Increase the radius to increase the distance between the objects.
object = PV.TurnToRectangular(object);
You should try to understand the rotation matrix and polar math to be able to achieve more things with this, but with this code it is possible. You should also put all this code in a class, but first play with it until you understand it well.
Sorry for the lengthy answer, but this is a topic that isn't very easy to explain without delving into linear algebra. The classes are for actual code manageability (I use these in my own game), but you can reproduce the same effects with the calculations only.
I personally prefer Polar Vectors over using the rotation matrix due to their usefulness in more than just rotating an object. But here's a link to understanding the rotation matrix better: https://en.wikipedia.org/wiki/Rotation_matrix
After you've done the transformation with the polar vector, you can just glTranslate to the final position given by the polar vector. You have to make sure that you rotate around the origin you are using. Otherwise rotation might not occur as you desire to use it.

How do I calculate collision with rotation in 3D space?

In my program I need to calculate collision between a rotated box and a sphere as well as collision between 2 rotated boxes. I can't seem to find any information on it and trying to figure the math out in my own is boggling my mind.
I have collision working for 2 boxes and a sphere and a box, but now I need to factor in angles. This is my code so far:
class Box
{
public:
Box();
private:
float m_CenterX, m_CenterY, m_CenterZ, m_Width, m_Height, m_Depth;
float m_XRotation, m_YRotation, m_ZRotation;
};
class Sphere
{
public:
Sphere();
private:
float m_CenterX, m_CenterY, m_CenterZ, radius;
unsigned char m_Colour[3];
};
bool BoxBoxCollision(BoxA, BoxB)
{
//The sides of the Cubes
float leftA, leftB;
float rightA, rightB;
float topA, topB;
float bottomA, bottomB;
float nearA, nearB;
float farA, farB;
//center pivot is at the center of the object
leftA = A.GetCenterX() - A.GetWidth();
rightA = A.GetCenterX() + A.GetWidth();
topA = A.GetCenterY() - A.GetHeight();
bottomA = A.GetCenterY() + A.GetHeight();
farA = A.GetCenterZ() - A.GetDepth();
nearA = A.GetCenterZ() + A.GetDepth();
leftB = B.GetCenterX() - B.GetWidth();
rightB = B.GetCenterX() + B.GetWidth();
topB = B.GetCenterY() - B.GetHeight();
bottomB = B.GetCenterY() + B.GetHeight();
farB = B.GetCenterZ() - B.GetDepth();
nearB = B.GetCenterZ() + B.GetDepth();
//If any of the sides from A are outside of B
if( bottomA <= topB ) { return false; }
if( topA >= bottomB ) { return false; }
if( rightA <= leftB ) { return false; }
if( leftA >= rightB ) { return false; }
if( nearA <= farB ) { return false; }
if( farA >= nearB ) { return false; }
//If none of the sides from A are outside B
return true;
}
bool SphereBoxCollision( Sphere& sphere, Box& box)
{
float sphereXDistance = abs(sphere.getCenterX() - box.GetCenterX());
float sphereYDistance = abs(sphere.getCenterY() - box.GetCenterY());
float sphereZDistance = abs(sphere.getCenterZ() - box.GetCenterZ());
if (sphereXDistance >= (box.GetWidth() + sphere.getRadius())) { return false; }
if (sphereYDistance >= (box.GetHeight() + sphere.getRadius())) { return false; }
if (sphereZDistance >= (box.GetDepth() + sphere.getRadius())) { return false; }
if (sphereXDistance < (box.GetWidth())) { return true; }
if (sphereYDistance < (box.GetHeight())) { return true; }
if (sphereZDistance < (box.GetDepth())) { return true; }
float cornerDistance_sq = ((sphereXDistance - box.GetWidth()) * (sphereXDistance - box.GetWidth())) +
((sphereYDistance - box.GetHeight()) * (sphereYDistance - box.GetHeight()) +
((sphereYDistance - box.GetDepth()) * (sphereYDistance - box.GetDepth())));
return (cornerDistance_sq < (sphere.getRadius()*sphere.getRadius()));
}
How do I factor in rotation? Any suggestions would be great.
First of all, your objects are boxes, not rectangles. The term rectangle is strictly reserved for the 2D figure.
When you are dealing with rotations, you should generally view them as a special form of an affine transform. An affine transform can be a rotation, a translation, a scaling operation, a shearing operation, or any combination of these, and it can be represented by a simple 4x4 matrix that is multiplied to the vectors that give the vertices of your boxes. That is, you can describe any rotated, scaled, sheared box as the unit box (the box between the vectors <0,0,0> to <1,1,1>) to which an affine transform has been applied.
The matrix of most affine transforms (except those that scale by a factor of zero) can be inverted, so that you can both transform any point into the coordinate system of the box and then compare it against <0,0,0> and <1,1,1> to check whether its inside the box, and transform any point in the coordinates of the box back into your world coordinate system (for instance you can find the center of your box by transforming the vector <0.5, 0.5, 0.5>). Since any straight line remains a straight line when an affine transform is applied to it, all you ever need to transform is the vertices of your boxes.
Now, you can just take the vertices of one box (<0,0,0>, <0,0,1>, ...), transform them into your world coordinate system, then transform them back into the coordinate system of another box. After that, the question whether the two boxes overlap becomes the question whether the box described by the transformed eight vertices overlaps the unit box. Now you can easily decide whether there is a vertex above the base plane of the unit box (y > 0), below the top plane (y < 1), and so on. Unfortunately there is a lot of cases to cover for a box/box intersection, it is much easier to intersect spheres, rays, planes, etc. than such complex objects like boxes. However, having one box nailed to the unit box should help a lot.
Sidenote:
For rotations in 3D, it pays to know how to use quaternions for that. Euler angles and similar systems all have the issue of gimbal lock, quaternions do not have this restriction.
Basically, every unit quaternion describes a rotation around a single, free axis. When you multiply two unit quaternions, you get a third one that gives you the rotation that results from applying the two quaternions one after the other. And, since it is trivial to compute the multiplicative inverse of a quaternion, you can also divide one quaternion by another to answer the question what one-axis rotation you would need to apply to get from one rotation state to another. That last part is simply impossible to do in terms of Euler angles. Quaternions are really one of the sweetest parts of mathematics.
I simply cannot cover all the details in this answer, the topic is quite a broad and interesting one. That is why I linked the four wikipedia articles. Read them if you need further details.
For Box-Box collision transform the coordinates in such a way that the first box is centered at the origin and is aligned with the axis. Then checking if the second box collides with it is easier even tho is not quite trivial. For most cases (physics engine at small dt*v where you can assume movement is continuous) it suffices to check if any of the vertices fall inside the first box.
For Box-Sphere is simpler. Like before, transform the coordinates in such a way that the box is centered at the origin and is aligned with the axis. Now you only need to check that the distance between the center of the box and each of the canonical planes (generated by the axes) is less than the radius of the sphere plus half of the span of the box in the normal direction.

Rotating an equilateral triangle about the centroid in DirectX

I have a project in which I want to make a game that uses Asteroids-style ships in order to demonstrate AI pathfinding and steering behaviors.
The object stores values for the position the centroid of the triangle (Otherwise the player's position) and the ships' orientation in degrees, then calculates the position of the three vertices like so:
A is the nose of the triangle. When the the orientation is 0 degrees, the ship's nose sits on the X-axis offset by SIZE, which detemines the size of the ship which is equal to the radius of its bounding circle.
Vertices B and C are rotated from the position of A 120 and 240 degrees respecively, then the program draws line segments between each vertex.
When I start the program, this works beautifully, and my bounding circle, orientation vector, and ship look how I expect them to.
However, I implemented a control such that angle is incremented and decremented with the left and right arrow keys. This control has been tested and proven to work as desired. When this happens, the orientation vector adjusts accordingly, and the ship rotates... but only sort of. The points immediately move in to the center of the mass until they collapse on the center and disappear forever, regardless of which direction I rotate in.
//Convert degrees into radians
float Ship::convert(float degrees){
return degrees*(PI/180);
}
//Rotation X-Coord
int Ship::rotateX(int x, int y, float degrees){
return (x*cos(convert(degrees)))-(y*sin(convert(degrees)));
}
//Rotate Y-Coord
int Ship::rotateY(int x, int y, float degrees){
return x*sin(convert(degrees))+y*cos(convert(degrees));
}
//Rotate the Ship
void Ship::rotateShip(float degrees){
vertA [0]=rotateX(vertA [0], vertA [1], degrees);
vertA [1]=rotateY(vertA [0], vertA [1], degrees);
vertB [0]=rotateX(vertB [0], vertB [1], degrees);
vertB [1]=rotateY(vertB [0], vertB [1], degrees);
vertC [0]=rotateX(vertC [0], vertC [1], degrees);
vertC [1]=rotateY(vertC [0], vertC [1], degrees);
}
The rotate functions work fine, but I can't be sure that rotating the Ship is working. It seems that it should- the vert variables are arrays that store the X and Y coord of that vertex. If I rotate each vertex by the orientation angle of the ship, it should in theory spin the triangle on its center, right?
The verts are all calibrated about the origin, and the drawShip() function in the Game object takes those coords and adds the playerX and playerY to translate the fully drawn ship to the position of the player. So essentially, translating it to the origin, rotating it, and translating it back to the position of the player.
If it helps, I will include the relevant functions from the Game Object. In this code, psp is the Ship object. Draw ship takes all the parameters of the ship object and should work, but I'm getting unwanted results.
//Draw Ship
void Game::drawShip1(int shipX, int shipY, int vel, int alpha, int size, int* vertA, int* vertB, int* vertC, int r, int g, int b){
//Draw a bounding circle
drawCircle(shipX, shipY, size, 0, 255, 255);
//Draw the ship by connecting vertices A, B, and C
//Segment AB
drawLine(shipX+vertA[0], shipY+vertA[1], shipX+vertB [0], shipY+vertB [1], r, g, b);
//Segment AC
drawLine(shipX+vertA[0], shipY+vertA[1], shipX+vertC [0], shipY+vertC [1], r, g, b);
//Segment BC
drawLine(shipX+vertB[0], shipY+vertB[1], shipX+vertC [0], shipY+vertC [1], r, g, b);
}
void Game::ComposeFrame()
{
float pAlpha=psp.getAlpha();
if(kbd.LeftIsPressed()){
pAlpha-=1;
psp.setAlpha(pAlpha);
psp.rotateShip(psp.getAlpha());
}
if(kbd.RightIsPressed()){
pAlpha+=1;
psp.setAlpha(pAlpha);
psp.rotateShip(psp.getAlpha());
}
//Draw Playership
drawShip1(psp.getX(), psp.getY(), psp.getV(), psp.getAlpha(), psp.getSize(), psp.getVertA(), psp.getVertB(), psp.getVertC(), psp.getR(), psp.getG(), psp.getB());
drawLine(psp.getX(), psp.getY(), psp.getX()+psp.rotateX(0+psp.getSize(), 0, psp.getAlpha()), psp.getY()+psp.rotateY(0+psp.getSize(), 0, psp.getAlpha()), psp.getR(), psp.getG(), psp.getB());
}
I see several problems in your code which might add up to the result you are seeing.
The first thing is that you handle coordinates as integers. Due to this, decimal places get cut off and the value gets smaller and smaller. Use float or even double.
The second thing is that the rotation is applied in two steps. The first step (calculating the x coordinate) is correct. However, the second step (calculating the y coordinate) is errornous. You use the new x-coordinate, but should provide the old one. You should implement a method that rotates the complete vector and not part of them, anyway. Why don't you use the DirectX maths functions?

Ray tracing vectors

So I decided to write a ray tracer the other day, but I got stuck because I forgot all my vector math.
I've got a point behind the screen (the eye/camera, 400,300,-1000) and then a point on the screen (a plane, from 0,0,0 to 800,600,0), which I'm getting just by using the x and y values of the current pixel I'm looking for (using SFML for rendering, so it's something like 267,409,0)
Problem is, I have no idea how to cast the ray correctly. I'm using this for testing sphere intersection(C++):
bool SphereCheck(Ray& ray, Sphere& sphere, float& t)
{ //operator * between 2 vec3s is a dot product
Vec3 dist = ray.start - sphere.pos; //both vec3s
float B = -1 * (ray.dir * dist);
float D = B*B - dist * dist + sphere.radius * sphere.radius; //radius is float
if(D < 0.0f)
return false;
float t0 = B - sqrtf(D);
float t1 = B + sqrtf(D);
bool ret = false;
if((t0 > 0.1f) && (t0 < t))
{
t = t0;
ret = true;
}
if((t1 > 0.1f) && (t1 < t))
{
t = t1;
ret = true;
}
return ret;
}
So I get that the start of the ray would be the eye position, but what is the direction?
Or, failing that, is there a better way of doing this? I've heard of some people using the ray start as (x, y, -1000) and the direction as (0,0,1) but I don't know how that would work.
On a side note, how would you do transformations? I'm assuming that to change the camera angle you just adjust the x and y of the camera (or the screen if you need a drastic change)
The parameter "ray" in the function,
bool SphereCheck(Ray& ray, Sphere& sphere, float& t)
{
...
}
should already contain the direction information and with this direction you need to check if the ray intersects the sphere or not. (The incoming "ray" parameter is the vector between the camera point and the pixel the ray is sent.)
Therefore the local "dist" variable seems obsolete.
One thing I can see is that when you create your rays you are not using the center of each pixel in the screen as the point for building the direction vector. You do not want to use just the (x, y) coordinates on the grid for building those vectors.
I've taken a look at your sample code and the calculation is indeed incorrect. This is what you want.
http://www.csee.umbc.edu/~olano/435f02/ray-sphere.html (I took this course in college, this guy knows his stuff)
Essentially it means you have this ray, which has an origin and direction. You have a sphere with a point and a radius. You use the ray equation and plug it into the sphere equation and solve for t. That t is the distance between the ray origin and the intersection point on the spheres surface. I do not think your code does this.
So I get that the start of the ray would be the eye position, but what is the direction?
You have camera defined by vectors front, up, and right (perpendicular to each other and normalized) and "position" (eye position).
You also have width and height of viewport (pixels), vertical field of view (vfov) and horizontal field of view (hfov) in degrees or radians.
There are also 2D x and y coordinates of pixel. X axis (2D) points to the right, Y axis (2D) points down.
For a flat screen ray can be calculated like this:
startVector = eyePos;
endVector = startVector
+ front
+ right * tan(hfov/2) * (((x + 0.5)/width)*2.0 - 1.0)
+ up * tan(vfov/2) * (1.0 - ((y + 0.5f)/height)*2.0);
rayStart = startVector;
rayDir = normalize(endVector - startVector);
That assumes that screen plane is flat. For extreme field of view angles (fov >= 180 degreess) you might want to make screen plane spherical, and use different formulas.
how would you do transformations
Matrices.