User Tools

Site Tools


Data Format vs File Format

The import and export tools mainly interact with files external to the model. Using these files requires some understanding of the type of files and the layout of the data in the files. The import and export tools conceptualize that by file format and data format respectively.

File format is the the makeup of the file itself, for example text is ASCII and readable by various text editors and is pretty universal, and there are also various binary types that depend more specifically on the program reading and writing the files.

Data format is the logical order and organization of the data within the file.

Here are is an example of each tool using the parameters related to format:

local myVar[] = import (; dataFormat=coordinate, fileFormat=text, dataFile=myVarTextFile.txt)

export (myVar[]; dataFormat=tab, fileFormat=text, dataFile=myVarOutput.txt)

File Formats

These are the supported file formats for each tool

text ASCII and readable by any text editor yes yes
dbf standard database format yes yes
shape1) ARCInfo GIS format yes yes
mif2) MapInfo GIS format (geographic object definition) yes yes
mid MapInfo GIS format (attribute table) yes yes
tool A file written by whatIf?'s tool languages in binary format yes yes

Data Formats

These are the supported data formats for each tool

coordinate The data is in delimited columns where the first columns specify the element names in the respective dimension sets and the data is in the last column.

Pros: hunt & peck (order resilience). This is the preferred format.
yes yes See Example
tool A file written by whatIf?'s tool languages in binary format yes yes
odometer Data is delimited as each data item is read it fills the object in the order of it's dimensions. No validation that the data lands in the right cells is done.

Pros: row and column headings not required, good for reading legacy TOOL output
yes yes See Example
mapping yes yes
record The data is in delimited columns where the source data is read into a two dimensional variable consisting of a numerical record id and a user defined set of fields.

Pros: Can convert text data into integer codes, extract data sets that are not continuous, can easily tabulate or cross-tabulate variables.
Cons: The source data must have unique column descriptions for each field that are usually related in some way.
yes yes See Example
whatIfGEO (rename) yes yes

Provide more conceptual description of the different data formats, their pros and cons, etc. FIXME

howtos/workwithdata/data_format_vs_file_format.txt · Last modified: 2011/06/24 19:14 by chris.strashok