I have raster data for built up areas around the globe with 40m resolution as vrt file, download data from here , and I am trying to crop the data by a mask and then extract color index value for each cell.
Note: another 2 files exist with the data: vrt.clr and vrt.ovr
Here is a sample of data:
view of vrt data in arcmap.
My question: why I am getting empty cells values when I crop by mask ?
I have tried the following:
extract by mask using arcmap toolbox
using gdal in python 2.7
import gdal
ds = gdal.Open('input.vrt')
ds = gdal.Translate('output.vrt', ds, projWin =
[80.439,5.341,81.048,4.686])
ds = None
I have also try to save the data as tif
Also, is there any way to read the color index value at given coordinates (x,y) after masking the data?
The data appears to be in the Pseudo Mercator projection (EPSG 3857). So therefore you should either specify the extent for projWin in that coordinate system, or add projWinSRS if you want to provide them in a different coordinate system.
Also, if you want gdal.Translate to output to a VRT file, you should add format='VRT. Because in your code snippet outputs to the default file format, which is GeoTIFF.
When i assume your coordinates are WGS84 (EPSG 4326), it defines a small region over the ocean south of Sri Lanka. That doesn't make much sense given the nature of the data.
If you want to read the array given by your coordinates you could use:
invrt = 'GHS_BUILT_LDSMT_GLOBE_R2015B_3857_38_v1_0.vrt'
outfile = '/vsimem/tmpfile'
ds = gdal.Translate(outfile, invrt, projWin=[80.439, 5.341, 81.048, 4.686], projWinSRS='EPSG:4326')
data = ds.ReadAsArray()
ds = None
gdal.Unlink(outfile)
The plotted array looks like:
Related
I am using OpenCV 3.4.1.
I am working on a video classification project and am trying to use KNearest to classify between 2 categories. I have 8 areas of interest in each video frame. To make decision on each frame, each KNearest is done on the pixel values on each area. The majority win (and favor to one category if it is a tie). So, I have 8 sets of training data (one for each area of interest).
Problem: The response generated from the knn model changed when I labelled the categories differently.
The training data sets are organized as rows of:
[category label], data0, data1, data2....etc. (different dimensions for each
training set)
where dataX = pixel data of a frame (1 row = 1 frame)
Then, I build the model by:
Ptr<TrainData> tdata = TrainData::loadFromCSV(filename, 0, 0, -1, String("cat"));
Mat raw = tdata->getTrainSamples();
Mat res = tdata->getResponses();
PCA pca(raw, noArray(), PCA::DATA_AS_ROW, 0.99);
Mat knnIn = pca.project(raw);
Ptr<ml::KNearest> knn = ml::KNearest::create();
knn->train(knnIn, ml::ROW_SAMPLE, res);
After that, testing data is passed to the pca and knn to get the response.
To testing it, I put 1 set of testing data to the 8 knn models.
If I use 0 & 1 as the [category label] in the training data sets, the 8 responses from KNN are 1,1,1,1,1,0,0,1.
If I change the label to 2 & 1 instead (replace all '0' by '2' in the first column of the training data), the 8 responses becomes 1,1,1,1,1,1,1,1, while I am expecting 1,1,1,1,1,2,2,1.
Some observations:
no editing error on the training data(while changing the category labels).
The KNN model isClassifier()=true. DefaultK=10. AlgorithmType=1 (BRUTE_FORCE).
The result is consistent with the same training data set and testing data set.
I don't see any pattern on the difference responses for the 2 label sets (after using different
training data sets and testing data sets).
Please shed some light. Thank you very much!
I have created a map of precipitation levels in a region based on precipitation data from NetCDF files. I would like to add a custom scale such that if precipitation is less than 800mm it would be one colour, 800-1000mm another, etc. Similar to the map found here: http://www.metmalawi.com/climate/climate.php
At the moment I am using a gradient scale but it isn't showing the detail I need. This is the code for the plot at the moment (where 'Average' is my data that I have already formatted).
#load color palette
colourA = mpl_cm.get_cmap('BuPu')
#plot map with physical features
ax = plt.axes(projection=cartopy.crs.PlateCarree())
ax.add_feature(cartopy.feature.COASTLINE)
ax.add_feature(cartopy.feature.BORDERS)
ax.add_feature(cartopy.feature.LAKES, alpha=0.5)
ax.add_feature(cartopy.feature.RIVERS)
#set map boundary
ax.set_extent([32.5, 36., -9, -17])
#set axis tick marks
ax.set_xticks([33, 34, 35])
ax.set_yticks([-10, -12, -14, -16])
lon_formatter = LongitudeFormatter(zero_direction_label=True)
lat_formatter = LatitudeFormatter()
ax.xaxis.set_major_formatter(lon_formatter)
ax.yaxis.set_major_formatter(lat_formatter)
#plot data and set colour range
plot = iplt.contourf(Average, cmap=colourA, levels=np.arange(0,15500,500), extend='both')
#add colour bar index and a label
plt.colorbar(plot, label='mm per year')
#give map a title
plt.title('Pr 1990-2008 - Average_ERAINT ', fontsize=10)
#save the image of the graph and include full legend
plt.savefig('ERAINT_Average_Pr_MAP_Annual', bbox_inches='tight')
plt.show()
Anyone know how I can do this?
Thank you!
This is a matplotlib question disguised as an Iris question as the issue has appeared via Iris plotting routines, but to answer this we need only a couple of matplotlib commands. As such, I'm basing this answer on this matplotlib gallery example. These are levels (containing values for the upper bound of each contour) and colors (specifying the colours to shade each contour). It's best if there are the same number of levels and colours.
To demonstrate this, I put the following example together. Given that there's no sample data provided, I made my own trigonometric data. The levels are based on the trigonometric data values, so do not reflect the levels required in the question, but could be changed to the original levels. The colours used are the hex values of the levels specified by image in the link in the question.
The code:
import matplotlib.pyplot as plt
import numpy as np
x = np.arange(-25, 25)
y = np.arange(-20, 20)
x2d, y2d = np.meshgrid(x, y)
vals = (3 * np.cos(x2d)) + (2 * np.sin(y2d))
colours = ['#bf8046', '#df9f24', '#e0de30', '#c1de2d', '#1ebf82',
'#23de27', '#1dbe20', '#11807f', '#24607f', '#22427e']
levels = range(-5, 6)
plt.contourf(vals, levels=levels, colors=colours)
plt.colorbar()
plt.show()
The produced image:
Colours could also be selected from a colormap (one way of doing this is shown in this StackOverflow answer). There are also other ways, including in the matplotlib gallery example linked above. Given, though, that the sample map linked in the question had specific colours I chose to use those colours directly.
I have a polygon in my model citys but in my map for example
bogota has the coordinate -8243997.66798 , 517864.86656 -> open street maps; but i need make query with coordinates like (4.697857, -74.144554) -> google maps.
pnt = 'POINT(%d %d)'%(lon, lat)
zonas = ciudad.zona_set.filter(polygon__contains=pnt)
zonas is empty :/ ,
how i can convert lat and lon to standar coordinates in open street map , or how know the srid code for one input (lat,lon)
pnt = GEOSGeometry('POINT(-96.876369 29.905320)', srid=srid_from_lat_lon)
thanks
When making spatial queries, its good practice to pass a geometry that has a specified spatial reference system (srid). Like this, GeoDjango will automatically convert the input query geometry to the coordinate system of your table (the coordinate systems of your city model in your case).
In the first code example you gave, you do not specify an srid on the geometry, so pnt = 'POINT(%d %d)'%(lon, lat) does not have an srid. In this case, GeoDjango will assume the srid is the same for the input and the model data table. Which is not the case in your example, and that is why you dont get any matches.
So you will need to create you point with the correct SRID. If you get the coordinates from OSM, most likely the coordinates are in the Web Mercator projection, which has the srid 3857. This projection is often used in web mapping.
For this, you can use the EWKT format (which is essentially SRID + WKT) like so:
pnt = 'SRID=4326;POINT(-96.876369 29.90532)'
Or if you have the coordinates in Web Mercator Projection, the following should work:
pnt = 'SRID=3857;POINT(-8243997.66798 517864.86656)'
zonas = ciudad.zona_set.filter(polygon__contains=pnt)
Just for reference, here are a few examples on how to go back an forth between EWKT and GEOSGeometries:
So this (normal WKT, with srid specified on creation of geometry)
GEOSGeometry('POINT(-8243997.66798 517864.86656)', srid=3857)
is equivalent to this (srid contained in EWKT string):
GEOSGeometry('SRID=3857;POINT(-8243997.66798 517864.86656)')
I am trying to train a Custom Object Detector by using the HOG+SVM method on OpenCV.
I have managed to extract HOG features from my positive and negative samples using the below line of code:
import cv2
hog = cv2.HOGDescriptor()
def poshoggify():
for i in range(1,20):
image = cv2.imread("/Users/munirmalik/cvprojek/cod/pos/" + str(i)+ ".jpg")
(winW, winH) = (500, 500)
for resized in pyramid(image, scale=1.5):
# loop over the sliding window for each layer of the pyramid
for (x, y, window) in sliding_window(resized, stepSize=32, windowSize=(winW, winH)):
# if the window does not meet our desired window size, ignore it
if window.shape[0] != winH or window.shape[1] != winW:
continue
img_pos = hog.compute(image)
np.savetxt('posdata.txt',img_pos)
return img_pos
And the equivalent function for the negative samples.
How do I format the data in such a way that the SVM knows which is positive and which is negative?
Furthermore, how do I translate this training to the "test" of detecting the desired objects through my webcam?
How do I format the data in such a way that the SVM knows which is positive and which is negative?
You would now create another list called labels which would store the class value associated with a corresponding image. For example, if you have a training set of features that looks like this:
features = [pos_features1, pos_features2, neg_features1, neg_features2, neg_features3, neg_features4]
you would have a corresponding labels class like
labels = [1, 1, 0, 0, 0, 0]
You would then feed this to a classifier like so:
clf=LinearSVC(C=1.0, class_weight='balanced')
clf.fit(features,labels)
Furthermore, how do I translate this training to the "test" of detecting the desired objects through my webcam?
Before training, you should have split your labelled dataset (groundtruth) into training and testing datasets. You can do this using skilearns KFold module.
I handle iris cubes containing meteorological data (lon, lat, precipitation, temperature,...) and I am interested in calculating statistics in defined areas (for example a country).
This post explains how to crop the cube with a box (min lon, min lat, max lon, max lat) but I would like to go a step further and select a precise area using a shapefile.
This post explains that it is possible to crop an image using a shapefile associated to a mask, but I don't know how I can make it work for my iris cubes.
If somebody could give me an example or explain me how to do that it would be very useful.
PS: I am quite noobie with python
Having read the shapefile using e.g. Fiona something like this should work:
from shapely.geometry import MultiPoint
# Create a mask for the data
mask = np.ones(cube.shape, dtype=bool)
# Create a set of x,y points from the cube
x, y = np.meshgrid(cube.coord(axis='X').points, cube.coord(axis='Y').points)
lat_lon_points = np.vstack([x.flat, y.flat])
points = MultiPoint(lat_lon_points.T)
# Find all points within the region of interest (a Shapely geometry)
indices = [i for i, p in enumerate(points) if region.contains(p)]
mask[np.unravel_index(indices)] = False
# Then apply the mask
if isinstance(cube.data, np.ma.MaskedArray):
cube.data.mask &= mask
else:
cube.data = np.ma.masked_array(cube.data, mask)
This only works for 2D cubes, but just needs tweaking for higher dimensions so that the mask is only over the lat/lon dimensions.
I actually implemented this behaviour in CIS recently so that you can do cube.subset(shape=region) which might be easier for you.