Given a Pyomo model and corresponding data command file input.dat, I would like to verify the model I built with a 3rd party tool. To be more specific, it is a network flow model with the topology defined in the data command file. The strategy would then be to load the topology from that input.dat and use it as an input to the 3rd party tool.
The actual question: Is there a way to load the topology defined in the Pyomo input.dat directly into Python data structures (e.g. sets, dicts, etc.) instead of round tripping through Pyomo data structures (pyomo.environ.Set et al.) and then building the Python data structures from the Pyomo data structures - for the sake of convenience? Is there maybe an undocumented or unofficial internal API function that does this?
You might be able to do this using the DataPortal infrastructure in Pyomo. See the documentation here: https://pyomo.readthedocs.io/en/latest/working_abstractmodels/data/dataportals.html
Related
Given a Pyomo model that is fully specified, is it possible to write it's Set and Param components back to Data Command Files?
One use case would be to feed that data to another Pyomo model, or even load the Data Command Files into an existing AMPL toolchain to compare and debug models when porting from AMPL to Pyomo.
I could not find anything useful in their docs or examples :(
They have docs of how to deserialize schema, and then how to traverse it - that works great.
They have also two examples:
how to use bond::bonded<void> with runtime schema to convert binary to json
how to use bond::bonded<void> with runtime schema and MapTo to parse data into some custom ViewStruct
What I need is: to traverse the bond:bonded<void> to extract fields from it.
In protobuf you have field descriptors (created using FinFieldByName) and then you use them in runtime methods msg->getString, msg->getInt32 to get the typed value.
There's nothing built in that allows you to traverse a bonded<void> and inspect its fields. If you need to do that generically, you'll need to implement a custom transform and then use bond::Apply() to apply the transform to a bonded<void> constructed from a reader and a runtime schema. See in particular the transform and access_control examples for how to start writing a transform.
Bond needs this level of indirection compared to Protocol Buffers because Bond supports pluggable encoding protocols. If you know that you will only have to process a specific protocol, you could use that protocol's reader type with the runtime schema to walk the fields in the payload. See Microsoft Bond deserialization without knowing underlying schema for a C# example of the core of this algorithm.
My ultimate goal is to have map data (offline, because I will customize it myself) and display it in an app (Android). I could make osmdroid work to load maps online and I was trying to figure out how to download and display offline maps. I downloaded MOBAC (Mobile Atlas Creator) and export the data to SQLite format and when I had a look at it I realized that tiles are saved in image format (PNG).
What I would like to do is to import data to the phone to later use it in algorithms such as a search engine or a routing algorithm, so I need the "nodes" and "ways" (as I get them from the original OSM XML), import them to the phone and visualize it to later have this data available for the algorithms I want to develop. Basically, what MAPS.ME does. I think it wouldn't be difficult to convert the XML into the SQLite since a simple script could make it, but then, how can I generate the tiles from this custom SQLite database? Or, is there a way I can download the data in a more appropriate way to do what I'm planning to do?
Thanks.
Rendering the tiles in an app from raw Openstreetmap data would be computation heavy and inefficient. I would suggest to use image tiles you exported for visual representation.
In addition to tiles you should export a data set you will need in the application for desired functionality. You will not need all data from Openstreetmap so you should identify what you need and build your custom export (there are tools and libraries for processing and filtering of Openstreetmap data. I have used pyosmium for some filtering and processing but there are others.) For example, you can build your custom database with POIs you want to search for.
Routing is another chapter. You can implement it yourself but it is a very complex task. There is java library called Graphopper which can do the data extraction (from Openstreetmap) and offline routing for you. They have an online API too but it is possible to make it working completely offline (I did it for one application). Try to look at the source code because than you can see how complex topic routing is. Final note: data exported from Graphopper contains information about some POIs along routes. It may be possible to search for some things via its java API but I haven't investigated this yet.
I am currently developing an application for facial recognition.
The algorithms are implemented and trained using the MatConvnet library (http://www.vlfeat.org/matconvnet/). At the end, I have a Network (.mat file) which looks like that:
I would like to know if it were possible to extract the weights of the Network using its .mat file, write them in a XML file and read them with Caffe C++. I would like to reuse them in Caffe C++ in order to do some testing and hardware implementation. Is there an efficient and practical way to proceed so ?
Thank you for very much for your help.
The layer whose parameters you'd like to store, must be set as 'precious'. In net.var you can access the parameters and write them.
There is a conversion script that converts matconvnet models to caffe models here which you may find useful.
You can't use weights of the trained Network by matconvnet for caffe. You can merely import your model from matconvnet to caffe.(https://github.com/vlfeat/matconvnet/blob/4ce2871ec55f0d7deed1683eb5bd77a8a19a50cd/utils/import-caffe.py). But this script does not support all layers and you may have difficulties in employing it.
The best way is to define your caffe prototxt in python as the matconvnet model.
first of all: I do not want to implement AI import functionality based on AI file format specification for my own. Second: Adobe Illustrator is not installed on my target system, so I can not use it's programming interface.
But I want to import AI files into some CAD-like application to access the vector data out of this file afterwards (can be any kind of data structure, converting it to my own format is not a problem). How can this be done? Is there a library or something like this available which provides related functionality?
Thanks!