I'm trying to apply a local min/max stretch (or other) for a moving window over an image. It works, basically, but takes forever, since I loop over each pixel, calculate the min/max around it, stretch them to the available value range, write them and then move on. I heard about "sliced arrays" and "striding", but don't know how to adopt them. This is what I do at the moment:
import os
import numpy as np
from osgeo import gdal
def localStretch(image, radius, output):
ds = gdal.Open(image, gdal.GA_ReadOnly)
drv = ds.GetDriver()
cols = ds.RasterXSize
rows = ds.RasterYSize
bands = ds.RasterCount
if os.path.exists(output):
os.remove(output)
out_ds = drv.Create(output, cols, rows, bands, ds.GetRasterBand(1).DataType)
window = radius * 2 + 1
for b in range(bands):
data = ds.GetRasterBand(b + 1).ReadAsArray()
out_data = np.zeros(data.shape)
for x in range(radius, cols - radius):
for y in range(radius, rows - radius):
minimum = np.min(data[y - radius: y + radius, x - radius: x + radius])
maximum = np.max(data[y - radius: y + radius, x - radius: x + radius])
out_data[y, x] = (data[y, x] * 1. - minimum * 1.) / (maximum * 1. - minimum * 1.) * np.iinfo(data.dtype).max
out_ds.GetRasterBand(b + 1).WriteArray(out_data)
out_ds = None
ds = None
Ideally, I would also like to preserve the image size by inserting np.pad(data, radius, mode='reflect'), but then I'm not sure how to index everything correctly.
So how can I improve the performance of this?
In the meantime, I found a function from skimage here, which can do local stretching, but I need to do similar stuff with moving windows, so the principal question still remains.
Related
This question already has answers here:
Finding whether a point lies inside a rectangle or not
(10 answers)
Closed 2 years ago.
I have a rotated rectangle with these coordinates as vertices:
1 670273 4879507
2 677241 4859302
3 670388 4856938
4 663420 4877144
And I have points with these coordinates:
670831 4867989
675097 4869543
Using only the Python 2.7 standard library, I want to determine if the points fall within the rotated rectangle.
I am not able to add additional Python libraries to my Jython implementation
What would it take to do this?
A line equation of the form ax+by+c==0 can be constructed from 2 points. For a given point to be inside a convex shape, we need testing whether it lies on the same side of every line defined by the shape's edges.
In pure Python code, taking care of writing the equations avoiding divisions, this could be as follows:
def is_on_right_side(x, y, xy0, xy1):
x0, y0 = xy0
x1, y1 = xy1
a = float(y1 - y0)
b = float(x0 - x1)
c = - a*x0 - b*y0
return a*x + b*y + c >= 0
def test_point(x, y, vertices):
num_vert = len(vertices)
is_right = [is_on_right_side(x, y, vertices[i], vertices[(i + 1) % num_vert]) for i in range(num_vert)]
all_left = not any(is_right)
all_right = all(is_right)
return all_left or all_right
vertices = [(670273, 4879507), (677241, 4859302), (670388, 4856938), (663420, 4877144)]
The following plot tests the code visually for several shapes. Note that for shapes with horizontal and vertical lines usual line equations could provoke division by zero.
import matplotlib.pyplot as plt
import numpy as np
vertices1 = [(670273, 4879507), (677241, 4859302), (670388, 4856938), (663420, 4877144)]
vertices2 = [(680000, 4872000), (680000, 4879000), (690000, 4879000), (690000, 4872000)]
vertices3 = [(655000, 4857000), (655000, 4875000), (665000, 4857000)]
k = np.arange(6)
r = 8000
vertices4 = np.vstack([690000 + r * np.cos(k * 2 * np.pi / 6), 4863000 + r * np.sin(k * 2 * np.pi / 6)]).T
all_shapes = [vertices1, vertices2, vertices3, vertices4]
for vertices in all_shapes:
plt.plot([x for x, y in vertices] + [vertices[0][0]], [y for x, y in vertices] + [vertices[0][1]], 'g-', lw=3)
for x, y in zip(np.random.randint(650000, 700000, 1000), np.random.randint(4855000, 4880000, 1000)):
color = 'turquoise'
for vertices in all_shapes:
if test_point(x, y, vertices):
color = 'tomato'
plt.plot(x, y, '.', color=color)
plt.gca().set_aspect('equal')
plt.show()
PS: In case you are running a 32-bit version of numpy, with this size of integers it might be necessary to convert the values to float to avoid overflow.
If this calculation needs to happen very often, the a,b,c values can be precalculated and stored. If the direction of the edges is known, only one of all_left or all_right is needed.
When the shape is fixed, a text version of the function can be generated:
def generate_test_function(vertices, is_clockwise=True, function_name='test_function'):
ext_vert = list(vertices) + [vertices[0]]
unequality_sign = '>=' if is_clockwise else '<='
print(f'def {function_name}(x, y):')
parts = []
for (x0, y0), (x1, y1) in zip(ext_vert[:-1], ext_vert[1:]):
a = float(y1 - y0)
b = float(x0 - x1)
c = a * x0 + b * y0
parts.append(f'({a}*x + {b}*y {unequality_sign} {c})')
print(' return', ' and '.join(parts))
vertices = [(670273, 4879507), (677241, 4859302), (670388, 4856938), (663420, 4877144)]
generate_test_function(vertices)
This would generate a function as:
def test_function(x, y):
return (-20205.0*x + -6968.0*y >= -47543270741.0) and (-2364.0*x + 6853.0*y >= 31699798882.0) and (20206.0*x + 6968.0*y >= 47389003912.0) and (2363.0*x + -6853.0*y >= -31855406372.0)
This function then can be copy-pasted and optimized by the Jython compiler. Note that the shape doesn't need to be rectangular. Any convex shape will do, allowing to use a tighter box.
Take three consequent vertices A, B, C (your 1,2,3)
Find lengths of sides AB and BC
lAB = sqrt((B.x - A.x)^2+(B.y - A.y)^2)
Get unit (normalized) direction vectors
uAB = ((B.x - A.x) / lAB, (B.y - A.y) / lAB)
For tested point P get vector BP
BP = ((P.x - B.x), (P.y - B.y))
And calculate signed distances from sides to point using cross product
SignedDistABP = Cross(BP, uAB) = BP.x * uAB.y - BP.y * uAB.x
SignedDistBCP = - Cross(BP, uBC) = - BP.x * uBC.y + BP.y * uBC.x
For points inside rectangle both distances should have the same sign - either negative or positive depending on vertices order (CW or CCW), and their absolute values should not be larger than lBC and lAB correspondingly
Abs(SignedDistABP) <= lBC
Abs(SignedDistBCP) <= lAB
As the shape is an exact rectangle, the easiest is to rotate all points by the angle
-arctan((4859302-4856938)/(677241-670388))
Doing so, the rectangle becomes axis-aligned and you just have to perform four coordinate comparisons. Rotations are easy to compute with complex numbers.
In fact you can simply represent all points as complex numbers, compute the vector defined by some side, and multiply everything by the conjugate.
A slightly different approach is to consider the change of coordinate frame that brings some corner to the origin and two incident sides to (1,0) and (0,1). This is an affine transformation. Then your test boils down to checking insideness to the unit square.
i am working on a project of swarm algorithms and i am trying to make complex shapes using the swarm consensus. However, the mathematics to achieve that seems quite difficult for me.
I have been able to make shapes like stars, circle and triangle but to make other complex shapes seems more harder. It would be very helpful if i get the idea of using numpy arrays to build these complex shapes using swarms....................................................
# general function to reset radian angle to [-pi, pi)
def reset_radian(radian):
while radian >= math.pi:
radian = radian - 2*math.pi
while radian < -math.pi:
radian = radian + 2*math.pi
return radian
# general function to calculate next position node along a heading direction
def cal_next_node(node_poses, index_curr, heading_angle, rep_times):
for _ in range(rep_times):
index_next = index_curr + 1
x = node_poses[index_curr][0] + 1.0*math.cos(heading_angle)
y = node_poses[index_curr][1] + 1.0*math.sin(heading_angle)
node_poses[index_next] = np.array([x,y])
index_curr = index_next
return index_next
##### script to generate star #####
filename = 'star'
swarm_size = 30
node_poses = np.zeros((swarm_size, 2))
outer_angle = 2*math.pi / 5.0
devia_right = outer_angle
devia_left = 2*outer_angle
# first node is at bottom left corner
heading_angle = outer_angle / 2.0 # current heading
heading_dir = 0 # current heading direction: 0 for left, 1 for right
seg_count = 0 # current segment count
for i in range(1,swarm_size):
node_poses[i] = (node_poses[i-1] +
np.array([math.cos(heading_angle), math.sin(heading_angle)]))
seg_count = seg_count + 1
if seg_count == 3:
seg_count = 0
if heading_dir == 0:
heading_angle = reset_radian(heading_angle - devia_right)
heading_dir = 1
else:
heading_angle = reset_radian(heading_angle + devia_left)
heading_dir = 0
print(node_poses)
with open(filename, 'w') as f:
pickle.dump(node_poses, f)
pygame.init()
# find the right world and screen sizes
x_max, y_max = np.max(node_poses, axis=0)
x_min, y_min = np.min(node_poses, axis=0)
pixel_per_length = 30
world_size = (x_max - x_min + 2.0, y_max - y_min + 2.0)
screen_size = (int(world_size[0])*pixel_per_length, int(world_size[1])*pixel_per_length)
# convert node poses in the world to disp poses on screen
def cal_disp_poses():
poses_temp = np.zeros((swarm_size, 2))
# shift the loop to the middle of the world
middle = np.array([(x_max+x_min)/2.0, (y_max+y_min)/2.0])
for i in range(swarm_size):
poses_temp[i] = (node_poses[i] - middle +
np.array([world_size[0]/2.0, world_size[1]/2.0]))
# convert to display coordinates
poses_temp[:,0] = poses_temp[:,0] / world_size[0]
poses_temp[:,0] = poses_temp[:,0] * screen_size[0]
poses_temp[:,1] = poses_temp[:,1] / world_size[1]
poses_temp[:,1] = 1.0 - poses_temp[:,1]
poses_temp[:,1] = poses_temp[:,1] * screen_size[1]
return poses_temp.astype(int)
disp_poses = cal_disp_poses()
# draw the loop shape on pygame window
color_white = (255,255,255)
color_black = (0,0,0)
screen = pygame.display.set_mode(screen_size)
screen.fill(color_white)
for i in range(swarm_size):
pygame.draw.circle(screen, color_black, disp_poses[i], 5, 0)
for i in range(swarm_size-1):
pygame.draw.line(screen, color_black, disp_poses[i], disp_poses[i+1],2)
pygame.draw.line(screen, color_black, disp_poses[0], disp_poses[swarm_size-1], 2)
pygame.display.update()
Your method for drawing takes huge advantage of the symmetries in the shapes you are drawing. More complex shapes will have fewer symmetries and so your method will require a lot of tedious work to get them drawn with stars. Without symmetry you may be better served writing each individual line 'command' in a list and following that list. For example, drawing the number 4 starting from the bottom (assuming 0 degrees is --> that way):
angles = [90,225,0]
distances = [20,15,12]
Then with a similar program to what you have, you can start drawing dots in a line at 90 degrees for 20 dots, then 225 degrees for 15 dots etc... Then by adding to these two lists you can build up a very complicated shape without relying on symmetry.
I would like to fit a 2D array by an elliptic function: (x / a)² + (y / b)² = 1 ----> (and so get the a and b)
And then, be able to replot it on my graph.
I found many examples on internet, but no one with this simple Cartesian equation. I probably have searched badly ! I think a basic solution for this problem could help many people.
Here is an example of the data:
Sadly, I can not put the values... So let's assume that I have an X,Y arrays defining the coordinates of each of those points.
This can be solved directly using least squares. You can frame this as minimizing the sum of squares of quantity (alpha * x_i^2 + beta * y_i^2 - 1) where alpha is 1/a^2 and beta is 1/b^2. You have all the x_i's in X and the y_i's in Y so you can find the minimizer of ||Ax - b||^2 where A is an Nx2 matrix (i.e. [X^2, Y^2]), x is the column vector [alpha; beta] and b is column vector of all ones.
The following code solves the more general problem for an ellipse of the form Ax^2 + Bxy + Cy^2 + Dx +Ey = 1 though the idea is exactly the same. The print statement gives 0.0776x^2 + 0.0315xy+0.125y^2+0.00457x+0.00314y = 1 and the image of the ellipse generated is also below
import numpy as np
import matplotlib.pyplot as plt
alpha = 5
beta = 3
N = 500
DIM = 2
np.random.seed(2)
# Generate random points on the unit circle by sampling uniform angles
theta = np.random.uniform(0, 2*np.pi, (N,1))
eps_noise = 0.2 * np.random.normal(size=[N,1])
circle = np.hstack([np.cos(theta), np.sin(theta)])
# Stretch and rotate circle to an ellipse with random linear tranformation
B = np.random.randint(-3, 3, (DIM, DIM))
noisy_ellipse = circle.dot(B) + eps_noise
# Extract x coords and y coords of the ellipse as column vectors
X = noisy_ellipse[:,0:1]
Y = noisy_ellipse[:,1:]
# Formulate and solve the least squares problem ||Ax - b ||^2
A = np.hstack([X**2, X * Y, Y**2, X, Y])
b = np.ones_like(X)
x = np.linalg.lstsq(A, b)[0].squeeze()
# Print the equation of the ellipse in standard form
print('The ellipse is given by {0:.3}x^2 + {1:.3}xy+{2:.3}y^2+{3:.3}x+{4:.3}y = 1'.format(x[0], x[1],x[2],x[3],x[4]))
# Plot the noisy data
plt.scatter(X, Y, label='Data Points')
# Plot the original ellipse from which the data was generated
phi = np.linspace(0, 2*np.pi, 1000).reshape((1000,1))
c = np.hstack([np.cos(phi), np.sin(phi)])
ground_truth_ellipse = c.dot(B)
plt.plot(ground_truth_ellipse[:,0], ground_truth_ellipse[:,1], 'k--', label='Generating Ellipse')
# Plot the least squares ellipse
x_coord = np.linspace(-5,5,300)
y_coord = np.linspace(-5,5,300)
X_coord, Y_coord = np.meshgrid(x_coord, y_coord)
Z_coord = x[0] * X_coord ** 2 + x[1] * X_coord * Y_coord + x[2] * Y_coord**2 + x[3] * X_coord + x[4] * Y_coord
plt.contour(X_coord, Y_coord, Z_coord, levels=[1], colors=('r'), linewidths=2)
plt.legend()
plt.xlabel('X')
plt.ylabel('Y')
plt.show()
Following the suggestion by ErroriSalvo, here is the complete process of fitting an ellipse using the SVD. The arrays x, y are coordinates of the given points, let's say there are N points. Then U, S, V are obtained from the SVD of the centered coordinate array of shape (2, N). So, U is a 2 by 2 orthogonal matrix (rotation), S is a vector of length 2 (singular values), and V, which we do not need, is an N by N orthogonal matrix.
The linear map transforming the unit circle to the ellipse of best fit is
sqrt(2/N) * U * diag(S)
where diag(S) is the diagonal matrix with singular values on the diagonal. To see why the factor of sqrt(2/N) is needed, imagine that the points x, y are taken uniformly from the unit circle. Then sum(x**2) + sum(y**2) is N, and so the coordinate matrix consists of two orthogonal rows of length sqrt(N/2), hence its norm (the largest singular value) is sqrt(N/2). We need to bring this down to 1 to have the unit circle.
N = 300
t = np.linspace(0, 2*np.pi, N)
x = 5*np.cos(t) + 0.2*np.random.normal(size=N) + 1
y = 4*np.sin(t+0.5) + 0.2*np.random.normal(size=N)
plt.plot(x, y, '.') # given points
xmean, ymean = x.mean(), y.mean()
x -= xmean
y -= ymean
U, S, V = np.linalg.svd(np.stack((x, y)))
tt = np.linspace(0, 2*np.pi, 1000)
circle = np.stack((np.cos(tt), np.sin(tt))) # unit circle
transform = np.sqrt(2/N) * U.dot(np.diag(S)) # transformation matrix
fit = transform.dot(circle) + np.array([[xmean], [ymean]])
plt.plot(fit[0, :], fit[1, :], 'r')
plt.show()
But if you assume that there is no rotation, then np.sqrt(2/N) * S is all you need; these are a and b in the equation of the ellipse.
You could try a Singular Value Decomposition of the data matrix.
https://docs.scipy.org/doc/numpy-1.13.0/reference/generated/numpy.linalg.svd.html
First center the data by subtracting mean values of X,Y from each column respectively.
X=X-np.mean(X)
Y=Y-np.mean(Y)
D=np.vstack(X,Y)
Then, apply SVD and extract
-eigenvalues (members of s) -> axis length
-eigenvectors(U) -> axis orientation
U, s, V = np.linalg.svd(D, full_matrices=True)
This should be a least-squares fit.
Of course, things can get more complicated than this, please see
https://www.emis.de/journals/BBMS/Bulletin/sup962/gander.pdf
I am drawing a graph with 2000+ points to a pdf file. The resolution of the pdf is 612 x 792. I can only draw 612 points to the pdf because the width is 612. I am mapping 1 point to 1 pixel. How can I plot all 2000+ samples to the pdf. I am using this lib http://www.vulcanware.com/cpp_pdf/index.html.
Option 1: Scale the points, using x = (x * 612) / 2000. This will mean that if 2 points are close to each other (including "similar y") they will overwrite each other.
Option 2: Treat each point as a square; and calculate floating point values for the "left edge x" and "right edge x" that have been scaled (left_x = ( (x-width/2.0) * 612.0) / 2000.0; right_x = ( (x+width/2.0) * 612.0) / 2000.0;), and draw the square using anti-aliasing, by calculating "area of destination pixel that the square overlaps" for each destination pixel that overlaps with the square. In this case you will need to do "dest_pixel = max(dest_pixel + area, 1);" to clamp pixel values when squares overlap.
Option 3: Rotate the whole thing 90 degrees so that "x axis" goes vertically down the page (and can be split across multiple pages if necessary); and if this causes a problem for y then use one of the options above for y.
Note that "option 2" can be done in both (vertical and horizontal) directions at the same time. To do this, start by determining the edges of the square, like:
left_x = point_x / MAX_SRC_X * MAX_DEST_X;
right_x = (point_x + 1) / MAX_SRC_X * MAX_DEST_X;
top_y = point_y / MAX_SRC_Y * MAX_DEST_Y;
bottom_y = (point_y + 1) / MAX_SRC_Y * MAX_DEST_Y;
Then have a "for each row that is effected" loop that calculates how much each row is effected, like:
for(int y = top_y; y < bottom_y; y++) {
row_top = fmax(y, top_y);
row_bottom = fmin(y+1, bottom_y);
row_weight = row_bottom - row_top;
Then have a similar "for each column that is effected" loop, like:
for(int x = left_x; x < right_x; x++) {
column_left = fmax(x, left_x);
column_right = fmin(x+1, right_x);
column_weight = column_right - column_left;
Then calculate the area for the pixel, set the pixel, and complete the loops:
dest_pixel_area = row_weight * column_weight;
pixel[y][x].red = min(pixel[y][x].red + dest_pixel_area * red, MAX_RED);
pixel[y][x].green = min(pixel[y][x].green + dest_pixel_area * green, MAX_GREEN);
pixel[y][x].blue = min(pixel[y][x].blue + dest_pixel_area * blue, MAX_BLUE);
}
}
Note: All code above is untested and simplified. It can be faster to break the loops up into "first line/column; loop for middle area only; then last line/column" to remove most of the fmin/fmax.
If you only need to do this in one direction, delete the parts for the direction you don't need and use 1.0 for the corresponding row_weight or column_weight.
I'm trying to do a polar transform on the first image below and end up with the second. However my result is the third image. I have a feeling it has to do with what location I choose as my "origin" but am unsure.
radius = sqrt(width**2 + height**2)
nheight = int(ceil(radius)/2)
nwidth = int(ceil(radius/2))
for y in range(0, height):
for x in range(0, width):
t = int(atan(y/x))
r = int(sqrt(x**2+y**2)/2)
color = getColor(getPixel(pic, x, y))
setColor( getPixel(radial,r,t), color)
There are a few differences / errors:
They use the centre of the image as the origin
They scale the axis appropriately. In your example, you're plotting your angle (between 0 and in your case, pi), instead of utilising the full height of the image.
You're using the wrong atan function (atan2 works a lot better in this situation :))
Not amazingly important, but you're rounding unnecessarily quite a lot, which throws off accuracy a little and can slow things down.
This is the code combining my suggested improvements. It's not massively efficient, but it should hopefully work :)
maxradius = sqrt(width**2 + height**2)/2
rscale = width / maxradius
tscale = height / (2*math.pi)
for y in range(0, height):
dy = y - height/2
for x in range(0, width):
dx = x - width/2
t = atan2(dy,dx)%(2*math.pi)
r = sqrt(dx**2+dy**2)
color = getColor(getPixel(pic, x, y))
setColor( getPixel(radial,int(r*rscale),int(t*tscale)), color)
In particular, it fixes the above problems in the following ways:
We use dx = x - width / 2 as a measure of distance from the centre, and similarly with dy. We then use these in replace of x, y throughout the computation.
We will have our r satisfying 0 <= r <= sqrt( (width/2)^2 +(height/2)^2 ), and our t eventually satisfying 0 < t <= 2 pi so, I create the appropriate scale factors to put r and t along the x and y axes respectively.
Normal atan can only distinguish based on gradients, and is computationally unstable near vertical lines... Instead, atan2 (see http://en.wikipedia.org/wiki/Atan2) solves both problems, and accepts (y,x) pairs to give an angle. atan2 returns an angle -pi < t <= pi, so we can find the remainder modulo 2 * math.pi to it to get it in the range 0 < t <= 2pi ready for scaling.
I've only rounded at the end, when the new pixels get set.
Any questions, just ask!