I'm using gfortran, I need to write a function that reads records from a .dbf file associated with an ESRI Shapefile. The file I should be able to read is available from internet http://diss.rm.ingv.it/diss/DISS_3.0.4.shp.zip
The opinion of the file command about the format of the file is:
$ file GGSources_polyline.dbf
GGSources_polyline.dbf: \012- DBase 3 data file\012- (119 records)
Thanks for your suggestions
I found a rough description of the file format here. It looks like there is quite a mix of variable types and sizes throughout, which is going to complicate things somewhat. I don't know if using Fortran to try and read this data is the best option, but if you must here are some hints:
Open the file for direct access unformatted I/O. Unformatted means that you can just read the bytes straight out of the file, and direct access won't add any padding to records.
Set the record length as the lowest common length between fields
Use the transfer() function to interpret a location in memory as a particular type. This will allow you to read the binary data from the file into a variable of type integer but then assign to a real without doing a type cast.
I'm in a similar situation now trying to read a file with a structure very similar to the dBase file (i.e. varying sizes of headers pointing to regions of the file with different types) and ended up using Python and Numpy to read the file. Reading consists of seeking to a location in the file, reading a bunch of bytes, then using the numpy.fromstring option to convert that into real*4, real*8, integer*8, etc. You can make this work, but you may want to keep your options open.
Your best bet is to conver the dbf file into something else, using e.g. the OGR tools, available in most linux distributions. You can just convert the contents of the dbf file into a CSV file using ogr2ogr:
ogr2ogr -f "CSV" output.csv FaultScarps_polyline.shp FaultScarps_polyline
(note that you need to include the layername, which for Shapefiles, is identical to the shapefile's name). The first 3 lines of the CSV look like this:
IDSOURCE,IDSCARP,SOURCENAME,FAULTSCARP,LENGHT,HEIGHT,AVGVOFFSET,MAXVOFFSET,VOFFSETTYP,AVGHOFFSET,MAXHOFFSET,HOFFSETTYP,AGE,NOEVENTS,LENGHTQ,HEIGHTQ,VOFFSETQ,HOFFSETQ,AGEQ,NOEVENTSQ,LENGHTN,HEIGHTN,VOFFSETN,HOFFSETN,AGEN,NOEVENTSN,REFERENCE
ITGG001, 1,Ovindoli-Pezza,Ovindoli-Pezza Fault Piano Pezza, 4.40, 18.00, 9.750, 16.000, 1, 0.000, 0.000,3, 10.000000000000000,3,1,0,1,1,1,1,Based on topographic observations.,Max height in late Pleistocene-Holocene fluvioglacial deposits.,Based on geological survey and refers to late Pleistocene-Holocene deposits.,Based on geological survey.,Based on geological observations.,Refers to Holocene and based on paleoseismology.,Pantosti et al. [1996].
ITGG001, 2,Ovindoli-Pezza,Ovindoli-Pezza Fault Campo Porcaro, 8.60, 0.00, 8.700, 12.000, 1, 3.045, 4.025,1, 18.000000000000000,3,1,0,1,1,1,1,Based on topographic observations.,,Max offset observed in the late Pleistocene-Holocene fluvioglacial and moraine deposits.,"Calculated as 35 % of the vertical component, on the basis of literature data.",Based on geological observations.,Refers to Holocene and based on paleoseismology.,Pantosti et al. [1996]
An alternative would be to access the Shapefile using OGR (or Shapelib) and doing the processing in C, returning it to the main Fortran program.
You may struggle reading binary unformatted files in Fortran which were not written from a Fortran write statement unless your compiler has some extensions.
Fortran binary unformatted files have beginning of record and end of record marks. These marks are usually the length of the record in bytes.
So the runtime system will try to interpret characters in the file as record marks and get confused.
Converting to csv ascii and reading that from Fortran will work. If you were going to try reading other file types then writing some C functions to interface to the C I/O library should allow you to read the files directly.
The FortranGIS package has Fortran bindings to the shapelib library, allowing to encode/decode shapefiles and the associated dbf files directly from a Fortran program:
http://fortrangis.berlios.de/ later moved to https://github.com/ARPA-SIMC/fortrangis
It works with gfortran 4.1.2 or later (F2003 ISO_C_BINDING module).
Related
I have got a trace file that is binary in nature. I want to convert it to a text file and convert the data inside it to decimal form. I mean I am not sure, how to do this. This .trc file contains data in the form of telegrams and I want to extract particular kind of telegram and save them in text file which is readable in nature. I have to do all of this using C++.
Do you suggest any other language for it or does anyone has any idea about doing this in C++?
Binary trace files are usually encoded in proprietary formats. And there are applications or profilers specifically built to parse them.
Unless you know the file format, the only way to decode it is through reverse engineering. And in most cases it's not worth the effort.
Try to find documentation about it. Or maybe an application or utility that loads the file and exports data that is easier to read.
In case you are speaking about .trc binary files from Teledyne Lecroy Oscilloscopes, I would suggest to any of the following libraries out there for that:
https://pypi.org/project/lecroyparser/
https://github.com/jneer/lecroy-reader
https://github.com/yetifrisstlama/readTrc
https://igit.ific.uv.es/ferhue/lecroyparser
How can I, using a function, library, whatever I have to, concatenate two .wav files? The input should be the absolute paths, and the output an audio file created and placed (not just played) somewhere, it doesn't really matter where.
I am writing a Mac command line application in XCode 6.
The .wav file format is a very simple format, consisting of the fixed header that defines the audio file's properties; namely the endian-ness, the number of channels, and the sampling rate. Its documentation is widely defined on the intertubes.
Off the top of my head I don't recall if any common library offers a convenient way to do this (it's worth looking through libsndfile's API documentation, for something that would fit the bill).
In any case, it shouldn't be too tough to read the headers of both WAV files, to check their format, and then create the output file. If both WAV files have the same endian-ness, number of channels, and sampling rate, the procedure is trivial, otherwise you will have to resample/remix at least one of the files.
There is a very simple, lightweight and mature open source C API library for reading-writing several common audio file formats. I haven't worked with it for a while, if I remember well, it has routines for opening a sound file for writing, seeking the end, appending data from another file and updating the header. I hope this can help.
I've seen a lot of examples of i/o with text files I'm just wondering if you can do the same with other file types like mp3's, jpg's, zip files, etc..?
Will iostream and fstream work for all of these or do I need another library? Do I need a new sdk?
It's all binary data so I'd think it would be that simple. But I've been unpleasently surprised before.
Could I convert all files to text or binary?
It depend on what you mean by "work"
You can think of those files as a book written in Greek.
If you want to just mess with binary representation (display text in Greek on screen) then yes, you can do that.
If you want to actually extract some info: edit video stream, remove voice from audio (actually understand what is written), then you would need to either parse file format yourself (learn Greek) or use some specialized library (hire a translator).
Either way, filestreams are suited to actually access those files data (and many libraries do use them under the hood)
You can work on binary streams by opening them with openmode binary :
ifstream ifs("mydata.mp3", ios_base::binary);
Then you read and write any binary content. However, if you need to generate or modify such content, play a video or display a piture, the you you need to know the inner details of the format you are using. This can be exremely complex, so a library would be recomended. And even with a library, advanced programming skills are required.
Examples of open source libraries: ffmpeg for usual audio/video format, portaudio for audio, CImg for image processing (in C++), libpng for png graphic format, lipjpeg for jpeg. Note that most libraries offer a C api.
Some OS also supports some native file types (example, windows bitmaps).
You can open these files using fstream, but the important thing to note is you must be intricately aware of what is contained within the file in order to process it.
If you just want to open it and spit out junk, then you can definitely just start at the first line of the file and exhaustively push all data into your console.
If you know what the file looks like on the inside, then you can process it just as you would any other file.
There may be specific libraries for processing specific files, but the fstream library will allow you to access any file you'd like.
All files are just bytes. There's nothing stopping you from reading/writing those bytes however you see fit.
The trick is doing something useful with those bytes. You could read the bytes from a .jpg file, for example, but you have to know what those bytes mean, and that's complicated. Usually it's best to use libraries written by people who know about the format in question, and let them deal with that complexity.
I want to split a big file into smaller ones without copying part of file, and without using filestream or functions which use it (if it is possible).
Imagine, we have big file which is consisted of 3 files:
[[File1bytes][File2bytes][File3bytes]]
In my opinion we can do this with these steps:
Use SetEndOfFile function to truncate the bytes of the last file ([File3bytes] in our example)
Somehow force our file system to recognize those truncated bytes ([File3bytes]) as a real file (maybe by adding some info to MFT table, or doing something with NTFS if it is possible, or using some function or method which can do all mentioned for us).
Any suggestions?
How about create a file system nesting over the existing file system where the very large file actually resides and define some IOCTL commands for splitting? Check this link:
How can I write my own 'filesystem' within Windows?
I've seen a lot of games use something similar to a .DAT file or a specific file type that the game has for itself. I'm just beginning with C++ and DirectX and I was interested in keeping my information in something similar to a .DAT.
My initial conception was that it would hold information on the files you wanted to store within the .DAT file. Something similar to a .RAR file. Unfortunately, my googleing skills did not help me in finding the answers.
Right now I'm simply loading textures and sound files from a folder called Data.
EDIT: While I understand that .DAT is short for data, and I've found that a .DAT file generally contains any assortment of information, I'm still unsure about how to go about doing something as packing images and sound files into any type of file and being able to read them.
I'm not sure about using fstreams to achieve my task, however I will look into streams related to storing data and how to properly read from that data. Meanwhile if anyone has another answer to offer based on this new information, it would be appreciated.
EDIT: Thanks to the answers, I stumbled across a similar question on stackoverflow and felt I'd share it here. Combining resources into a single binary file
I don't think there is really such thing as .dat file format. It's short for "data," and different applications just put in some proprietary stuff in it and call it ".dat." You can read up on fstream classes to do file IO in C++. See Input/Output with files.
What you then do is make up your own file format. For example, first 4 byte is int that indicates the number of blocks in the .dat and for each block, you have 4 byte indicating the length of each block, 4 byte indicating the type of the block, the variable length data itself .. something like that.
DAT obviously stands for data, and there is no real or de facto standard on what that extension actually refers to. Your decisions on the best file formats should be based on technical considerations, not pointless attempts at security through obscurity.
Professional games use a technique where they put all the needed resources (models, textures, sounds, ai, config, etc) zipped/packed into a single file thus making it faster to manage, harder to change (some even make use of a virtual filing system from what's inside the data file). Now, for what's inside the file is different depending on the needs of the game and the data structures that you use.
If you're just starting into gamedev, i recommend you stick with keeping all you assets separate and don't bother too much about packing them into a single file.
Now if you really want to start using a packed format here's a good pointer:
Creating a PAK File Format
Here's a link which claims that .dat is a movie format, 'DAT' being short for Digital Audio Tape.
I'm not sure I believe the link, but I do remember something about a Microsoft supported format called DAT, from long ago, when I used an earlier version of Windows.
It makes more sense as a logical extension for a DATA file of some kind.
.dat, as others have said, is literally just a data file. In reality, the file extension means nothing other than association with a program. For example, I could make a word processor that saves all the documents with the .mp3 file extension. These files wouldn't be playable in any media software, but the software might try. File extensions are used to help programs know what types of files they can and cannot open--however those rules don't have to be followed.
Anyway, you can dump any sort of information to a file. Programmers/software writers will often choose .dat as the extension of that file because it has become the standard to signify 'this file just holds a ton of data' and that the data doesn't necessarily hold any standardized headers, footers, or formatting.
A dat file could really contain anything. It might be as simple as a zip archive with the extension changed, or it could be a completely custom file type. If you're just starting out, you probably don't want to write your own file format, although doing so can be fun and educational. If you want to encapsulate your data files into some kind of container, you should probably go with a zip, paq, or maybe tar.gz.