tutorials:python_quick_reference
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision Last revision Both sides next revision | ||
|
tutorials:python_quick_reference [2021/02/05 15:06] marcus.williams [Python data and control structures] |
tutorials:python_quick_reference [2021/08/06 14:20] marcus.williams [Data import and export] |
||
|---|---|---|---|
| Line 76: | Line 76: | ||
| <code> | <code> | ||
| df_structure = gpd.read_file(myLayers.gdb, layer='structure') | df_structure = gpd.read_file(myLayers.gdb, layer='structure') | ||
| + | </code> | ||
| + | |||
| + | Round trip export/import for json files, as Python dictionaries or lists. Requires ''import json''. | ||
| + | |||
| + | <code> | ||
| + | filename = 'folderX/list_CEUDSector_ind.json' | ||
| + | with open(filename, 'w', encoding='utf-8') as f: | ||
| + | json.dump(l_CEUDSector_ind, f, ensure_ascii=False, indent=4) | ||
| + | </code> | ||
| + | |||
| + | <code> | ||
| + | filename = 'folderX/list_CEUDSector_ind.json' | ||
| + | with open(filename) as data_file: | ||
| + | l_CEUDSector_ind= json.load(data_file) | ||
| </code> | </code> | ||
| Line 104: | Line 118: | ||
| <code> | <code> | ||
| - | pt_structure = pd.pivot_table(df_structure, \ | + | pt_structure = pd.pivot_table(df_structure, |
| - | 'OBJECTID', | + | values='OBJECTID', |
| index=['propType'], | index=['propType'], | ||
| columns=['strucType'], | columns=['strucType'], | ||
| margins=True, | margins=True, | ||
| - | aggfunc=len) | + | aggfunc=np.sum) |
| </code> | </code> | ||
| [[http://pandas.pydata.org/pandas-docs/stable/generated/pandas.pivot_table.html#pandas.pivot_table|ref]] | | [[http://pandas.pydata.org/pandas-docs/stable/generated/pandas.pivot_table.html#pandas.pivot_table|ref]] | | ||
| Line 153: | Line 167: | ||
| df_structure['finalID'] = np.where(df_structure.condoID.isnull(), | df_structure['finalID'] = np.where(df_structure.condoID.isnull(), | ||
| df_structure.normalID, df_structure.condoID) | df_structure.normalID, df_structure.condoID) | ||
| + | </code> | ||
| + | |||
| + | **where()**, **mask()** | ||
| + | Replace if condition is false and true respectively. | ||
| + | <code> | ||
| + | df_8.mask(df_8 > 5.0, 5.0) | ||
| + | </code> | ||
| + | |||
| + | ===== Multiindexing ===== | ||
| + | |||
| + | Turns columns A and B the the multiindex for the output dataframe. | ||
| + | <code> | ||
| + | df.set_index(['A','B']) | ||
| + | </code> | ||
| + | |||
| + | Remove multiindexing. | ||
| + | <code> | ||
| + | df.reset_index() | ||
| + | </code> | ||
| + | |||
| + | Remove an unwanted level | ||
| + | <code> | ||
| + | df.columns = df.columns.get_level_values('CEUDEndUse_res') | ||
| </code> | </code> | ||
| Line 397: | Line 434: | ||
| git pull origin master | git pull origin master | ||
| </code> | </code> | ||
| + | |||
| + | ===== Handy Matrix & Vector Operations ===== | ||
| + | |||
| + | **shares / norm** | ||
| + | |||
| + | each row sums to 1 | ||
| + | <code> | ||
| + | df_shr = df_quantity.div(df_quantity.sum(axis='columns'), axis='index')) | ||
| + | </code> | ||
| + | |||
| + | each column sums to 1 | ||
| + | <code> | ||
| + | df_shr = df_quantity.div(df_quantity.sum(axis='index'), axis='columns')) | ||
| + | </code> | ||
| + | |||
| + | ===== Jupyter Notebook Markdown Tricks ===== | ||
| + | Links | ||
| + | <code> | ||
| + | [Github](http://github.com) | ||
| + | </code> | ||
| + | |||
| + | |||
| + | Embed an image | ||
| + | <code> | ||
| + |  | ||
| + | </code> | ||
| + | |||
| + | ===== Python Debugger (PDB) ===== | ||
| + | [[https://docs.python.org/3/library/pdb.html|ref]] | ||
| + | |||
| + | <code> | ||
| + | import pdb; pdb.set_trace() | ||
| + | </code> | ||
| + | |||
| + | ''n'' for next line; ''c'' to continue; ''s'' for step; ''exit'' for exit | ||
| ===== Other functions ===== | ===== Other functions ===== | ||
| ==== A ==== | ==== A ==== | ||
| Line 440: | Line 512: | ||
| <code> | <code> | ||
| df_emissions_2 = pd.melt(df_emissions_1, | df_emissions_2 = pd.melt(df_emissions_1, | ||
| - | id_vars=['finalDemandID', 'Corporate_Group', | + | id_vars=['department','scope','fuelType'], |
| - | 'Corporate_Department', 'emissionsSubSect', | + | value_vars=['2016','2017','2018'], |
| - | 'GPCScope','fuelType_all','Scenario'], | + | |
| var_name='Time', | var_name='Time', | ||
| value_name='Values') | value_name='Values') | ||
| - | </code> | + | </code> |
| + | Note that //value_vars// is optional. If omitted uses all columns that are not set as //id_vars//. | ||
| **merge()** | **merge()** | ||
| Line 522: | Line 594: | ||
| <code> | <code> | ||
| set([1,2,3]) - set([2,3,4]) | set([1,2,3]) - set([2,3,4]) | ||
| + | </code> | ||
| + | |||
| + | See other set [[https://www.w3schools.com/python/python_ref_set.asp|operations]] including: | ||
| + | <code> | ||
| + | set1.union(set2) | ||
| + | set1.intersection(set2) | ||
| </code> | </code> | ||
| ==== T ==== | ==== T ==== | ||
tutorials/python_quick_reference.txt · Last modified: 2021/08/06 22:20 by marcus.williams
Page Tools
