User Tools

Site Tools


howtos:workwithdata:import_quick_reference

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision
Previous revision
howtos:workwithdata:import_quick_reference [2010/09/16 15:00]
marcus.williams created
howtos:workwithdata:import_quick_reference [2017/05/18 14:51] (current)
marcus.williams [advanced import examples]
Line 1: Line 1:
-123+Examples of common import/​create calls are provided below. 
 + 
 +It is convenient to set up one or more import directory paths up front. For example: 
 + 
 +<​code>​ 
 +string $importDir= $modelHome + "/​V1/​primaryData"​ 
 +</​code>​ 
 + 
 +===== Specifying data directly through a parameter ===== 
 + 
 +<​code>​ 
 +hhVehOwnRelProp_hhs[hhs] = import (; data="​1.03 2.01 2.32 2.38 2.41 2.41 2.31"​) 
 +</​code>​ 
 + 
 +===== csv file ===== 
 + 
 +==== basic import ==== 
 + 
 +This is an example of importing into a very simple mask variable, //​zoneMask[z]//,​ indexed by a single pre-defined informant, //zone//. Here is the top of the source data file //​zonesMask.csv//:​ 
 + 
 +<​file>​ 
 +"​zone","​maskValue"​ 
 +201,1 
 +202,1 
 +211,0 
 +212,1 
 +213,0 
 +214,1 
 +
 +
 +
 +</​file>​ 
 + 
 +And here is the code: 
 + 
 +<​code>​ 
 +zoneMask[z] = import (; dataFormat=coordinate,​ fileFormat=text,​ \ 
 + allCoord=on,​ delimiter=",",​ firstLine=2,​ \ 
 + file=$importDir/​zonesMask.csv) 
 +</​code>​ 
 + 
 +Note:  
 +  * the use of the //​firstLine=2//​ parameter to skip over the header row 
 +  * no element name modification is required 
 + 
 +Alternatively one may use the heading parameter to skip over the first line: 
 + 
 +<​code>​ 
 +zoneMask[z] = import (; dataFormat=coordinate,​ fileFormat=text,​ \ 
 +    allCoord=on,​ delimiter=",",​ heading=on, \ 
 +    file=$importDir/​zonesMask.csv) 
 +</​code>​ 
 + 
 +==== advanced import examples ==== 
 + 
 +<​code>​ 
 +local occPrivDwellCustSC_1[pzans,​dtd] = create (; \ 
 + dim=PLUMzoneAllNS,​ dim=dwellType_data,​ \ 
 + entityFrom=occPrivDwellCustSC[pzans,​dt,​byrs][$index],​ \ 
 + dataFormat=coordinate,​ fileFormat=text,​ \ 
 + allCoord=off,​ delimiter=",",​ firstLine=2,​ \ 
 + searchElemName1=",",​ replaceElemName1="",​ \ 
 + searchElemName2="/",​ replaceElemName2="​_",​ \ 
 + searchElemName3="​-",​ replaceElemName3="​_",​ \ 
 + replaceWhiteSpaceInElemName="​_",​ \ 
 + stripLeadingWhiteSpace=off,​ \ 
 + searchData=",",​ replaceData=""​ , \ 
 + file=$importDir/​dwellUnitsByType.csv) 
 +</​code>​ 
 +==== read localinformants ==== 
 + 
 +Read informant definition from column: 
 + 
 +<​code>​ 
 +localinformant age_0510002[] = create (; object=set, delimiter=",",​ \ 
 + allowDuplicate=on,​ \ 
 + elemNamesFromColNum=3,​ firstLine=2,​ \ 
 + searchElemName1=",",​ replaceElemName1="",​ \ 
 + searchElemName2="/",​ replaceElemName2="​_",​ \ 
 + searchElemName3="​-",​ replaceElemName3="​_",​ \ 
 + replaceWhiteSpaceInElemName="​_",​ \ 
 + stripLeadingWhiteSpace=off,​ \ 
 + file=$importDir/​051-0002.csv) 
 +</​code>​ 
 + 
 +Read informant definition from line: 
 + 
 +<​code>​ 
 +localinformant date_0510002[] = create (; object=set, delimiter=",",​ \ 
 + allowDuplicate=off,​ \ 
 + elemNamesFromLineNum=1,​ firstCol=4, \ 
 + searchElemName1=",",​ replaceElemName1="",​ \ 
 + searchElemName2="/",​ replaceElemName2="​_",​ \ 
 + searchElemName3="​-",​ replaceElemName3="​_",​ \ 
 + replaceWhiteSpaceInElemName="​_",​ \ 
 + stripLeadingWhiteSpace=off,​ \ 
 + file=$importDir/​051-0002.csv) 
 +</​code>​ 
 + 
 + 
 +===== tab delimited text odometer (block) format ===== 
 +This format is useful for migrating data between models in a human-readable format, as opposed to binary //​tool//​-format files. 
 + 
 +<​code>​ 
 +immigration[s,​ts1,​a][$index] = import (; dataFormat=odometer,​ fileFormat=text,​ \ 
 + firstCol=2,​ file=$importDir_V3/​immigration.txt) 
 +</​code>​ 
 + 
 +reads from the following (header not shown): 
 + 
 +<​file>​ 
 +! sex=male 
 +! 0 1 2  
 +"​1977"​ 1.049407e+02 1.314661e+02 1.317941e+02 
 +"​1978"​ 9.129648e+01 1.133035e+02 1.212180e+02 
 +"​1979"​ 8.643492e+01 9.573391e+01 9.366822e+01 
 +"​1980"​ 9.673356e+01 9.806976e+01 9.806976e+01 
 +</​file>​ 
 + 
 +Note the use of the ''​firstCol=2''​ for objects of rank 2 or greater. See the [[howtos:​workwithdata:​export_quick_reference#​tab_delimited_text_odometer_block_format|corresponding export call]]. 
 + 
 +===== TOOL file ===== 
 + 
 +<​code>​ 
 +USUseComByInd_19982008[com_du,​indio_du,​t_19982008] = import (; \ 
 + format=tool,​ dataFile=$importDir_ind/​USUseComByInd_19982008) 
 +</​code>​ 
howtos/workwithdata/import_quick_reference.1284649204.txt.gz · Last modified: 2010/09/16 15:00 by marcus.williams