howtos:workwithdata:importing_and_tabulating_record-based_datasets
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
howtos:workwithdata:importing_and_tabulating_record-based_datasets [2011/06/09 18:04] chris.strashok |
howtos:workwithdata:importing_and_tabulating_record-based_datasets [2011/06/24 18:55] (current) chris.strashok |
||
|---|---|---|---|
| Line 1: | Line 1: | ||
| ====== Importing and Tabulating Record Based Datasets ====== | ====== Importing and Tabulating Record Based Datasets ====== | ||
| - | The record dataFormat option for the create tool allows you to read in a source data file into a two dimensional variable consisting of a numerical record id and a user defiend set of fields. This technique requires the format of the source data to have unique column descriptions with fields are usually related in some way. This technique is useful for: | + | The record dataFormat option for the create tool allows you to read in a source data file into a two dimensional variable consisting of a numerical record id and a user defined set of fields. This technique requires the format of the source data to have unique column descriptions with fields that are usually related in some way. This technique is useful for: |
| [[importing_and_tabulating_record-based_datasets#creating sets from source data|creating sets from source data]]; \\ | [[importing_and_tabulating_record-based_datasets#creating sets from source data|creating sets from source data]]; \\ | ||
| Line 14: | Line 14: | ||
| ===== creating sets from source data ===== | ===== creating sets from source data ===== | ||
| - | The create-fieldname parameter in the sample code below allows you to populate a pre-defined informant from your source data file. In this example the data is based on age and region, so we are intereseted in creating informants for these two parameters. We can either use these informants directly in the model structure or they can be mapped to an existing informant. | + | The create-fieldname parameter in the sample code below allows you to populate a pre-defined informant from your source data file. In this example the data is based on age and region, so we are intereseted in creating informants for these two parameters. We can either use these informants directly in the model structure or they can be [[howtos:toolcoding:using_maps|mapped]] to an existing informant. |
| <code> | <code> | ||
| Line 20: | Line 20: | ||
| file=$fullFileName, firstLine=1, delimiter=",", \ | file=$fullFileName, firstLine=1, delimiter=",", \ | ||
| replaceWhiteSpaceInData="_", \ | replaceWhiteSpaceInData="_", \ | ||
| - | indexTo-Column_A=Column_A, create-Column_A=on, \ | + | indexTo-Column_A=Column_A, create-Column_A=on, \ (the create-Column parameter is only required if the corresponding informant has not already been created) |
| indexTo-Column_C=Column_C, create-Column_C=on, \ | indexTo-Column_C=Column_C, create-Column_C=on, \ | ||
| takeField=Column_F, \ | takeField=Column_F, \ | ||
| Line 32: | Line 32: | ||
| </code> | </code> | ||
| - | {{:howtos:workwithdata:record_format_ex1_rev1.jpg|}} | + | {{:howtos:workwithdata:record-format-ex1.gif|{{:howtos:workwithdata:record_format_ex1_rev1.jpg|}} |
| \\ | \\ | ||
| [[importing_and_tabulating_record-based_datasets#Importing and Tabulating Record Based Datasets|back to top]] | [[importing_and_tabulating_record-based_datasets#Importing and Tabulating Record Based Datasets|back to top]] | ||
| Line 51: | Line 51: | ||
| </code> | </code> | ||
| - | When using the table tool we see the values in the recordData variable in the Table window as seen below (by default Table hides all zero values) | + | By using the table tool you can see the values in the recordData variable in the Table window as seen below (by default Table hides all zero values) |
| - | {{:howtos:workwithdata:record_format_ex2_1.jpg|}} | + | {{:howtos:workwithdata:record-format-ex2_1.gif|{{:howtos:workwithdata:record_format_ex2_1.jpg|}} |
| However the indexTo-fieldname parameter in the sample code below allows you to convert the text into integer codes allowing you to create an index that can be used as a variable map. | However the indexTo-fieldname parameter in the sample code below allows you to convert the text into integer codes allowing you to create an index that can be used as a variable map. | ||
| Line 62: | Line 62: | ||
| replaceWhiteSpaceInData="_", \ | replaceWhiteSpaceInData="_", \ | ||
| takefield=Column_A, \ | takefield=Column_A, \ | ||
| - | indexTo-Column_C=Column_C, create-Column_C=on, \ | + | indexTo-Column_C=Column_C, create-Column_C=on, \ (the create-Column parameter is only required if the corresponding informant has not already been created) |
| takeField=Column_I) | takeField=Column_I) | ||
| Line 68: | Line 68: | ||
| </code> | </code> | ||
| - | {{:howtos:workwithdata:record_format_ex2_2.jpg|}} | + | {{:howtos:workwithdata:record-format-ex2_2.gif|{{:howtos:workwithdata:record_format_ex2_2.jpg|}} |
| - | When used in conjuction with [[importing_and_tabulating_record-based_datasets#creating sets from source data|creating sets from source data|create-fieldname parameter]] you can use this text to create informant sets that can either be used directly in the model structure or mapped to an existing informant. | + | When used in conjuction with [[importing_and_tabulating_record-based_datasets#creating sets from source data|creating sets from source data]] you can use this text to create informant sets that can either be used directly in the model structure or mapped to an existing informant. |
| \\ | \\ | ||
| \\ | \\ | ||
| Line 90: | Line 90: | ||
| </code> | </code> | ||
| - | ^ ^ Column_D ^ Column_G ^ Column_I ^ | + | By using the table tool you can see the values in the recordData variable in the Table window as seen below (by default Table hides all zero values) |
| - | | 1 | 164.30 | 35.00 | 62.70 | | + | |
| - | | 2 | 278.80 | 65.00 | 75.90 | | + | {{:howtos:workwithdata:record-format-ex3.gif|}} |
| - | | 3 | 503.00 | 66.00 | 38.70 | | + | |
| - | | 4 | 271.80 | 66.00 | 96.00 | | + | |
| - | | 5 | 142.50 | 54.00 | 91.50 | | + | |
| - | | 6 | 253.50 | 71.00 | 82.70 | | + | |
| - | | 7 | 177.90 | 63.00 | 102.00 | | + | |
| - | | 8 | 163.40 | 63.00 | 88.00 | | + | |
| - | | 9 | 325.50 | 71.00 | 65.10 | | + | |
| - | | 10 | 351.10 | 75.00 | 83.70 | | + | |
| - | | 11 | 272.30 | 65.00 | 67.10 | | + | |
| Once you have imported your data you will need to extract the field(s) to populate individual model variables. For example,\\ | Once you have imported your data you will need to extract the field(s) to populate individual model variables. For example,\\ | ||
| Line 110: | Line 101: | ||
| recordFields:"Column_G", recordFields->dataG) | recordFields:"Column_G", recordFields->dataG) | ||
| </code> | </code> | ||
| + | |||
| + | {{:howtos:workwithdata:record-format-ex3_1.gif|}} | ||
| or you can also extract mulitple columns in order to manipulate the data in some way. | or you can also extract mulitple columns in order to manipulate the data in some way. | ||
| Line 118: | Line 111: | ||
| local columnsG_I_sum[recordID] = sum (columnsG_I[recordID,dataG_I]; dim=dataG_I) | local columnsG_I_sum[recordID] = sum (columnsG_I[recordID,dataG_I]; dim=dataG_I) | ||
| </code> | </code> | ||
| + | |||
| + | {{:howtos:workwithdata:record-format-ex3_2.gif|}} | ||
| + | |||
| \\ | \\ | ||
| [[importing_and_tabulating_record-based_datasets#Importing and Tabulating Record Based Datasets|back to top]] | [[importing_and_tabulating_record-based_datasets#Importing and Tabulating Record Based Datasets|back to top]] | ||
| Line 130: | Line 126: | ||
| file=$fullFileName, firstLine=1, delimiter=",", \ | file=$fullFileName, firstLine=1, delimiter=",", \ | ||
| replaceWhiteSpaceInData="_", \ | replaceWhiteSpaceInData="_", \ | ||
| - | indexTo-Column_A=Column_A, \ | + | indexTo-Column_A=Column_A, create-Column_A=on, \ (the create-Column parameter is only required if the corresponding informant has not already been created) |
| - | indexTo-Column_C=Column_C, \ | + | indexTo-Column_C=Column_C, create-Column_C=on, \ |
| takeField=Column_I) | takeField=Column_I) | ||
| Line 141: | Line 137: | ||
| </code> | </code> | ||
| - | ^ ^ Ontario ^ Quebec ^ Alberta ^ British_Columbia ^ Manitoba ^ Saskatchewan ^ New_Brunswick ^ PEI ^ Nova_Scotia ^ Terrirtories ^ Newfoundland ^ | + | By using the table tool you can see the values in the recordData variable in the Table window as seen below (by default Table hides all zero values) |
| - | | 2006 | 1.00 | | | | | | | | | | | | + | |
| - | | 2007 | | 1.00 | | 1.00 | 1.00 | | 1.00 | 1.00 | | 1.00 | | | + | {{:howtos:workwithdata:record-format-ex4.gif|}} |
| - | | 2009 | | | 1.00 | | | | | | | | 1.00 | | + | |
| - | | 2008 | | | | | | 1.00 | | | 1.00 | | | | + | |
| - | \\ | + | |
| [[importing_and_tabulating_record-based_datasets#Importing and Tabulating Record Based Datasets|back to top]] | [[importing_and_tabulating_record-based_datasets#Importing and Tabulating Record Based Datasets|back to top]] | ||
| /* | /* | ||
howtos/workwithdata/importing_and_tabulating_record-based_datasets.1307642666.txt.gz · Last modified: 2011/06/09 18:04 by chris.strashok
Page Tools
