This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision | ||
tutorials:python_quick_reference [2021/02/05 17:14] marcus.williams [M] |
tutorials:python_quick_reference [2021/08/06 22:20] (current) marcus.williams [Data interrogation] |
||
---|---|---|---|
Line 76: | Line 76: | ||
<code> | <code> | ||
df_structure = gpd.read_file(myLayers.gdb, layer='structure') | df_structure = gpd.read_file(myLayers.gdb, layer='structure') | ||
+ | </code> | ||
+ | |||
+ | Round trip export/import for json files, as Python dictionaries or lists. Requires ''import json''. | ||
+ | |||
+ | <code> | ||
+ | filename = 'folderX/list_CEUDSector_ind.json' | ||
+ | with open(filename, 'w', encoding='utf-8') as f: | ||
+ | json.dump(l_CEUDSector_ind, f, ensure_ascii=False, indent=4) | ||
+ | </code> | ||
+ | |||
+ | <code> | ||
+ | filename = 'folderX/list_CEUDSector_ind.json' | ||
+ | with open(filename) as data_file: | ||
+ | l_CEUDSector_ind= json.load(data_file) | ||
</code> | </code> | ||
Line 104: | Line 118: | ||
<code> | <code> | ||
- | pt_structure = pd.pivot_table(df_structure, \ | + | pt_structure = pd.pivot_table(df_structure, |
- | 'OBJECTID', | + | values='OBJECTID', |
index=['propType'], | index=['propType'], | ||
columns=['strucType'], | columns=['strucType'], | ||
- | margins=True, | + | margins=False, |
- | aggfunc=len) | + | aggfunc=np.sum) |
</code> | </code> | ||
[[http://pandas.pydata.org/pandas-docs/stable/generated/pandas.pivot_table.html#pandas.pivot_table|ref]] | | [[http://pandas.pydata.org/pandas-docs/stable/generated/pandas.pivot_table.html#pandas.pivot_table|ref]] | | ||
Line 153: | Line 167: | ||
df_structure['finalID'] = np.where(df_structure.condoID.isnull(), | df_structure['finalID'] = np.where(df_structure.condoID.isnull(), | ||
df_structure.normalID, df_structure.condoID) | df_structure.normalID, df_structure.condoID) | ||
+ | </code> | ||
+ | |||
+ | **where()**, **mask()** | ||
+ | Replace if condition is false and true respectively. | ||
+ | <code> | ||
+ | df_8.mask(df_8 > 5.0, 5.0) | ||
+ | </code> | ||
+ | |||
+ | ===== Multiindexing ===== | ||
+ | |||
+ | Turns columns A and B the the multiindex for the output dataframe. | ||
+ | <code> | ||
+ | df.set_index(['A','B']) | ||
+ | </code> | ||
+ | |||
+ | Remove multiindexing. | ||
+ | <code> | ||
+ | df.reset_index() | ||
+ | </code> | ||
+ | |||
+ | Remove an unwanted level | ||
+ | <code> | ||
+ | df.columns = df.columns.get_level_values('CEUDEndUse_res') | ||
</code> | </code> | ||
Line 397: | Line 434: | ||
git pull origin master | git pull origin master | ||
</code> | </code> | ||
+ | |||
+ | ===== Handy Matrix & Vector Operations ===== | ||
+ | |||
+ | **shares / norm** | ||
+ | |||
+ | each row sums to 1 | ||
+ | <code> | ||
+ | df_shr = df_quantity.div(df_quantity.sum(axis='columns'), axis='index')) | ||
+ | </code> | ||
+ | |||
+ | each column sums to 1 | ||
+ | <code> | ||
+ | df_shr = df_quantity.div(df_quantity.sum(axis='index'), axis='columns')) | ||
+ | </code> | ||
+ | |||
+ | ===== Jupyter Notebook Markdown Tricks ===== | ||
+ | Links | ||
+ | <code> | ||
+ | [Github](http://github.com) | ||
+ | </code> | ||
+ | |||
+ | |||
+ | Embed an image | ||
+ | <code> | ||
+ |  | ||
+ | </code> | ||
+ | |||
+ | ===== Python Debugger (PDB) ===== | ||
+ | [[https://docs.python.org/3/library/pdb.html|ref]] | ||
+ | |||
+ | <code> | ||
+ | import pdb; pdb.set_trace() | ||
+ | </code> | ||
+ | |||
+ | ''n'' for next line; ''c'' to continue; ''s'' for step; ''exit'' for exit | ||
===== Other functions ===== | ===== Other functions ===== | ||
==== A ==== | ==== A ==== | ||
Line 522: | Line 594: | ||
<code> | <code> | ||
set([1,2,3]) - set([2,3,4]) | set([1,2,3]) - set([2,3,4]) | ||
+ | </code> | ||
+ | |||
+ | See other set [[https://www.w3schools.com/python/python_ref_set.asp|operations]] including: | ||
+ | <code> | ||
+ | set1.union(set2) | ||
+ | set1.intersection(set2) | ||
</code> | </code> | ||
==== T ==== | ==== T ==== |