so I posted a question on OpenData and after advancing me a bit further in my quest to create maps from real cartographic information, I was advised to post my follow up questions on this website.
So after this intro, here is the original question:
I'm creating a 2D realistic RTS (Real time strategy game) and I wanted to be able to use real locations as the scenarios for the games.
The game will be developed via unreal engine which uses c++. The idea is for the engine to read an file and convert into a grid (coordinate point) where each square has type of terrain associated, like in this image of a scenario editor.
The file resulting from the other website is a GML (Geographic Markup Language) of a given location in the globe. GML is a XML extension.
The problem I'm facing is converting that GML for a given location into data that can be used by my game, like an location array or something like that.
Any sugestions?
There are many possibilities into doing this but the general guidelines or concepts should be as follows:
Know the data structure used by your engine
Open the incoming file and being able to parse and extract the needed data
Populate your structures or classes
Use your custom structures or classes to create game content (map data).
The first suggestion that I can give would be to read up on any and all documentation on the GML file format to know how the tags of the marked up language works then from there look to see if you can find any libraries that make it easy to read in a GML or XML file type and to generate C++ classes or structures, otherwise you will have to write both the file loader and gml(xml)parser with schema manually. Then from those structures being able to extract the data that you need and to pass them or store them into your engine's custom data structures. Then finally you should be able to use the stored data in your engine to generate the content that you want.
References
www.opengeospatial.org
www.w3.org
www.ogcnetwork.net
www.gdal.org
www.svgopen.org
Books
Geo-Informatics... 200+ page preview of almost 1,000 pages
Whitepapers
ResearchGate
Tools - Some Free Some Not
Google Search - GML File Converter
GoLoader by Snowflake Software
Various Sources by OGCNetwork
Open Source software from opengeospatial
GML Reader by tsusiatsoftware
Libraries
Question found on Stack Exchange
Various Libraries & Frameworks for Different Formats and Languages
gmXML Parser by YOYO games - *This Might be for the GameMaker Language
Java Code Examples for org.geotools.xml.Parser from programcreek
GML API for C/C++ Documentation
GML - The Geometric Modelling Library - API & GUI
Open Source - CityGML
The information needed is out there; it just takes time and effort; good research and the ability to apply what you have read into an actual code base.
Related
My ultimate goal is to have map data (offline, because I will customize it myself) and display it in an app (Android). I could make osmdroid work to load maps online and I was trying to figure out how to download and display offline maps. I downloaded MOBAC (Mobile Atlas Creator) and export the data to SQLite format and when I had a look at it I realized that tiles are saved in image format (PNG).
What I would like to do is to import data to the phone to later use it in algorithms such as a search engine or a routing algorithm, so I need the "nodes" and "ways" (as I get them from the original OSM XML), import them to the phone and visualize it to later have this data available for the algorithms I want to develop. Basically, what MAPS.ME does. I think it wouldn't be difficult to convert the XML into the SQLite since a simple script could make it, but then, how can I generate the tiles from this custom SQLite database? Or, is there a way I can download the data in a more appropriate way to do what I'm planning to do?
Thanks.
Rendering the tiles in an app from raw Openstreetmap data would be computation heavy and inefficient. I would suggest to use image tiles you exported for visual representation.
In addition to tiles you should export a data set you will need in the application for desired functionality. You will not need all data from Openstreetmap so you should identify what you need and build your custom export (there are tools and libraries for processing and filtering of Openstreetmap data. I have used pyosmium for some filtering and processing but there are others.) For example, you can build your custom database with POIs you want to search for.
Routing is another chapter. You can implement it yourself but it is a very complex task. There is java library called Graphopper which can do the data extraction (from Openstreetmap) and offline routing for you. They have an online API too but it is possible to make it working completely offline (I did it for one application). Try to look at the source code because than you can see how complex topic routing is. Final note: data exported from Graphopper contains information about some POIs along routes. It may be possible to search for some things via its java API but I haven't investigated this yet.
I am working on a project which needs to deal with large seismic data of SEGY format (from several GB to TB). This data represents the 3D underground structure.
Data structure is like:
1st tract, 2,3,5,3,5,....,6
2nd tract, 5,6,5,3,2,....,3
3rd tract, 7,4,5,3,1,....,8
...
What I want to ask is, in order to read and deal with the data fast, do I have to convert the data into another form? Or it's better to read from the original SEGY file? And is there any existing C package to do that?
If you need to access it multiple times and
if you need to access it randomly and
if you need to access it fast
then load it to a database once.
Do not reinvent the wheel.
When dealing of data of that size, you may not want to convert it into another form unless you have to - though some software does do just that. I found a list of free geophysics software on Wikipedia that look promising; many are open source and read/write SEGY files.
Since you are a newbie to programming, you may want to consider if the Python library segpy suits your needs rather than a C/C++ option.
Several GB is rathe medium, if we are toking about poststack.
You may use segy and convert on the fly, you may invent your own format. It depends whot you needed to do. Without changing segy format it's enough to createing indexes to traces. If segy is saved as inlines - it's faster access throug inlines, although crossline access is not very bad.
If it is 3d seismic, the best way to have the same quick access to all inlines/crosslines is to have own format - based od beans, e.g 8x8 traces - loading all beans and selecting tarces access time may be very quick - 2-3 secends. Or you may use SSD disk, or 2,5x RAM as your SEGY.
To quickly access timeslices you have 2 ways - 3D beans or second file stored as timeslices (the quickes way). I did same kind of that 10 years ago - access time to 12 GB SEGY was acceptable - 2-3 seconds in all 3 directions.
SEGY in database? Wow ... ;)
The answer depends upon the type of data you need to extract from the SEG-Y file.
If you need to extract only the headers (Text header, Binary header, Extended Textual File headers and Trace headers) then they can be easily extracted from the SEG-Y file by opening the file as binary and extracting relevant information from the respective locations as mentioned in the data exchange formats (rev2). The extraction might depend upon the type of data (Post-stack or Pre-stack). Also some headers might require conversions from one format to another (e.g Text Headers are mostly encoded in EBCDIC format). The complete details about the byte locations and encoding formats can be read from the above documentation
The extraction of trace data is a bit tricky and depends upon various factors like the encoding, whether the no. of trace samples is mentioned in the trace headers, etc. A careful reading of the documentation and getting to know about the type of SEG data you are working on will surely make this task a lot easier.
Since you are working with the extracted data, I would recommend to use already existing libraries (segpy: one of the best python library I came across). There are also numerous free available SEG-Y readers, a very nice list has already been mentioned by Daniel Waechter; you can choose any one of them that suits your requirements and the type file format supported.
I recently tried to do something same using C++ (Although it has only been tested on post-stack data). The project can be found here.
I´m trying to use the route-me framework together with C++ library libosmscout.
What I have done so far:
import OSM data into libosmscout format
compiling the iOS example-project from libosmscout-repo
The example project only draws one image of the map depending on given lat, long and zoom-factor.
What I want to do is, using the map view functionality of route-me but with map source data from libosmscout.
Is there anybody who has been managed this problem or could give me some hints?
I'm trying to do exactly the same thing.
But I haven' been able to compile the libosmscout for iOS up to now.
Anyway, I'm no sure how comfortable you are with route-me but I'm gonna try and answer your question.
If you are going to use libosmscout to leverage map data form osm xml or pbf files (probably for offline use) you're gonna need to implement a new MapSource and write the code yourself.
you'd better inherit from RMAbstractWebMapSource and implement imageForTile:tile inCache:tileCache method to meet your requirements. Route-me is a tile based map viewer, therefore you need to convert those vector data to tiles, for that you're gonna need to convert tile requests to boundingbox (a few examples are available in OSM wiki) and then try and extract the data from your OSM file and pass it to route-me as an image instance (a png for example).
Don't forget that Route-me caches loaded images, so you have to consider passing the right image for future uses. don't be fooled with the fact that your data is on device, reading vector files can be as demanding as reading raster files from web.
What I have in mind is that user will select the part of world he/she wants to generate roads and retrieve openStreetMap data and use it to render roads in openGL.
On searching the web and experimenting, I thought of this approach:
get xml file of selected map
parse the xml and generate roads by openGL.
But I think this is very naive approach.
Also to experiment a bit I used OSM2WorldViewer to convert the xml file to obj file and imported that as a model in openGL, but this method is cumbersome and takes time
and I am unfamiliar with OpenStreetMap api and how it can be used in such a project.
Any suggestions, or helpful links how to start this project ?
EDIT: How it ended: Link to the project wiki
Why do you think your approach is naive? Either you have the user to provide a self-downloaded XML file or you have to use an API to retrieve one yourself. The latter approach allows you to implement an automatic update mechanism whenever the user pans the map.
Instead of the main API you can use the Overpass API for downloading data. It's faster and more flexible to use, allowing to specify which element types to download (e.g. only roads and buildings) and much more.
You already mentioned OSM2World, take a look at its freely available code to see an example implementation of a 3D OpenGL renderer. Or take a look at one of the other 3D renderers for OSM.
I am looking for a good java library to manipulate box, arrows and labels in order to dynamically create and fill diagrams like the following and render them to a png :
Another example
I can create a static template to be filled later, but I will need to dynamically create labels for every box and edges in the diagrams.
For now I have tried using batik to manipulate an svg template but creating multiline labels for the edges is proving quite complicated. The SVG way with batik seems to force me to create 1 text object for each label line with absolute positioning for each which is a real pain.
I would like to be able to define the label specifying only the text and the link they relate to, eventually some hints as to how it should be placed and let the library place them.
Any suggestions ?
If this is in a commercial scenario, the Java graph drawing library yFiles can be a good match for your requirements:
You can use a convenient API to create and style your diagram and automatically layout the diagram with lots and lots options to constrain the resulting layout to suit your needs.
Multi-line and (simple) HTML labels are supported out-of-the-box.
Also you can export to PNG (as well as other pixel based image formats) and vector graphics formats like SVG, PDF, EPS.
This can be done both in a regular Java (Swing) application as well as in a headless environment (e.g. to create images on a server and to send them to a web-based thin client dynamically).
To get a feel what can be done using that library, I suggest you take a look at the freely available graph editor application "yEd" that is based on yFiles and lets you try out the above steps in an interactive manner.
(Disclosure: I work for yWorks, but on Stackoverflow I do not represent my employer.)