--- title: Geo-Data Intake and Operations keywords: fastai sidebar: home_sidebar summary: "This notebook was made to demonstrate how to work with geographic data." description: "This notebook was made to demonstrate how to work with geographic data." ---
{% raw %}
/usr/local/lib/python3.6/dist-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.
  """)
/content/drive/My Drive/dataplay/dataplay/acsDownload.py:27: FutureWarning: Passing a negative integer is deprecated in version 1.0 and will not be supported in future version. Instead, use None to not limit the column width.
  pd.set_option('display.max_colwidth', -1)
{% endraw %}

This Coding Notebook is the third in a series.

An Interactive version can be found here Open In Colab.

This colab and more can be found on our webpage.

  • Content covered in previous tutorials will be used in later tutorials.

  • New code and or information should have explanations and or descriptions attached.

  • Concepts or code covered in previous tutorials will be used without being explaining in entirety.

  • The Dataplay Handbook development techniques covered in the Datalabs Guidebook

  • If content can not be found in the current tutorial and is not covered in previous tutorials, please let me know.

  • This notebook has been optimized for Google Colabs ran on a Chrome Browser.

  • Statements found in the index page on view expressed, responsibility, errors and ommissions, use at risk, and licensing extend throughout the tutorial.

About this Tutorial:

Whats Inside?

The Tutorial

In this notebook, the basics of working with geographic data are introduced.

  • Reading in data (points/ geoms) -- Convert lat/lng columns to point coordinates -- Geocoding address to coordinates -- Changing coordinate reference systems -- Connecting to PostGisDB's
  • Basic Operations
  • Saving shape data
  • Get Polygon Centroids
  • Working with Points and Polygons -- Map Points and Polygons -- Get Points in Polygons -- Create Choropleths -- Create Heatmaps (KDE?)

Objectives

By the end of this tutorial users should have an understanding of:

  • How to read in and process geo-data asa geo-dataframe.
  • The Coordinate Reference System and Coordinate Encoding
  • Basic geo-visualization strategies

Background

Datatypes and Geo-data

Geographic data must be encoded properly order to attain the full potential of the spatial nature of your geographic data.

If you have read in a dataset using pandas it's data type will be a Dataframe.

It may be converted into a Geo-Dataframe using Geopandas as demonstrated in the sections below.

You can check a variables at any time using the dtype command:

yourGeoDataframe.dtype

Coordinate Reference Systems (CRS)

Make sure the appropriate spatial Coordinate Reference System (CRS) is used when reading in your data!

ala wiki:

A spatial reference system (SRS) or coordinate reference system (CRS) is a coordinate-based local, regional or global system used to locate geographical entities

CRS 4326 is the CRS most people are familar with when refering to latiude and longitudes.

Baltimore's 4326 CRS should be at (39.2, -76.6)

BNIA uses CRS 2248 internally Additional Information: https://docs.qgis.org/testing/en/docs/gentle_gis_introduction/coordinate_reference_systems.html

Ensure your geodataframes' coordinates are using the same CRS using the geopandas command:

yourGeoDataframe.CRS

Coordinate Encoding

When first recieving a spatial dataset, the spatial column may need to be encoded to convert its 'text' data type values into understood 'coordinate' data types before it can be understood/processed accordingly.

Namely, there are two ways to encode text into coordinates:

  • df[geom] = df[geom].apply(lambda x: loads( str(x) ))
  • df[geom] = [Point(xy) for xy in zip(df.x, df.y)]

The first approach can be used for text taking the form "Point(-76, 39)" and will encode the text too coordinates. The second approach is useful when creating a point from two columns containing lat/lng information and will create Point coordinates from the two columns.

More on this later

Raster Vs Vector Data

There exists two types of Geospatial Data, Raster and Vector. Both have different file formats.

This lab will only cover vector data.

Vector Data

Vector Data: Individual points stored as (x,y) coordinates pairs. These points can be joined to create lines or polygons.

Format of Vector data

Esri Shapefile — .shp, .dbf, .shx Description - Industry standard, most widely used. The three files listed above are needed to make a shapefile. Additional file formats may be included.

Geographic JavaScript Object Notation — .geojson, .json Description — Second most popular, Geojson is typically used in web-based mapping used by storing the coordinates as JSON.

Geography Markup Language — .gml Description — Similar to Geojson, GML has more data for the same amount of information.

Google Keyhole Markup Language — .kml, .kmz Description — XML-based and predominantly used for google earth. KMZ is a the newer, zipped version of KML.

Raster Data

Raster Data: Cell-based data where each cell represent geographic information. An Aerial photograph is one such example where each pixel has a color value

Raster Data Files: GeoTIFF — .tif, .tiff, .ovr ERDAS Imagine — .img IDRISI Raster — .rst, .rdc

Information Sourced From: https://towardsdatascience.com/getting-started-with-geospatial-works-1f7b47955438

Vector Data: Census Geographic Data:

Guided Walkthrough

SETUP:

Import Modules

{% raw %}
# @title Run: Install Modules
{% endraw %} {% raw %}
%%capture
! pip install -U -q PyDrive
! pip install geopy
! pip install geopandas
! pip install geoplot
! pip install dexplot
! pip install dataplay
{% endraw %} {% raw %}
{% endraw %}

Configure Enviornment

{% raw %}
# This will just beautify the output

pd.set_option('display.expand_frame_repr', False)
pd.set_option('display.precision', 2)
from IPython.core.interactiveshell import InteractiveShell
InteractiveShell.ast_node_interactivity = "all"

# pd.set_option('display.expand_frame_repr', False)
# pd.set_option('display.precision', 2)
# pd.reset_option('max_colwidth')
pd.set_option('max_colwidth', 20)
# pd.reset_option('max_colwidth')
{% endraw %}

(Optional) Local File Access

{% raw %}
# (Optional) Run this cell to gain access to Google Drive (Colabs only) 
from google.colab import drive

# Colabs operates in a virtualized enviornment
# Colabs default directory is at ~/content.
# We mount Drive into a temporary folder at '~/content/drive' 

drive.mount('/content/drive')
{% endraw %} {% raw %}
cd drive/'My Drive'/colabs/DATA
{% endraw %} {% raw %}
ls
{% endraw %}

File Access Conveince Functions

{% raw %}
# Find Relative Path to Files
def findFile(root, file):
    for d, subD, f in os.walk(root):
        if file in f:
            return "{1}/{0}".format(file, d)
            break 

# To 'import' a script you wrote, map its filepath into the sys
def addPath(root, file): sys.path.append(os.path.abspath( findFile( './', file) ))
{% endraw %}

Retrieve GIS Data

Approach 1: Reading in Data Directly

If you are using Geopandas, Direct imports only work with geojson and shape files

{% raw %}
# A Dataset taken from the public database provided by BNIAJFI hosted by Esro / ArcGIS
# BNIA ArcGIS Homepage: https://data-bniajfi.opendata.arcgis.com/
csa_gdf = gpd.read_file("https://opendata.arcgis.com/datasets/b738a8587b6d479a8824d937892701d8_0.geojson");
{% endraw %} {% raw %}
csa_gdf.plot()
<matplotlib.axes._subplots.AxesSubplot at 0x7fdb39640860>
{% endraw %}

Approach 2: Converting Pandas into Geopandas

Approach 2: Example 1

This approach loads a map using a geometry column

{% raw %}
# The attributes are what we will use.
in_crs = 2248 # The CRS we recieve our data 
out_crs = 4326 # The CRS we would like to have our data represented as
geom = 'geometry' # The column where our spatial information lives.
{% endraw %} {% raw %}
# A url to a public Dataset
url = 'https://docs.google.com/spreadsheets/d/e/2PACX-1vQ8xXdUaT17jkdK0MWTJpg3GOy6jMWeaXTlguXNjCSb8Vr_FanSZQRaTU-m811fQz4kyMFK5wcahMNY/pub?gid=886223646&single=true&output=csv'

# Read in the dataframe
tract_df = pd.read_csv(url)

# Convert the geometry column datatype from a string of text into a coordinate datatype
tract_df[geom] = tract_df[geom].apply(lambda x: loads( str(x) ))

# Process the dataframe as a geodataframe with a known CRS and geom column
tract_gdf = GeoDataFrame(tract_df, crs=in_crs, geometry=geom)
{% endraw %} {% raw %}
tract_gdf.plot()
<matplotlib.axes._subplots.AxesSubplot at 0x7fdb36b10470>
{% endraw %} {% raw %}
tract_gdf.head()
TRACTCE10 GEOID10 NAME10 CSA Tract geometry
0 151000 24510151000 1510.0 Dorchester/Ashbu... 1510 POLYGON ((-76.67...
1 80700 24510080700 807.0 Greenmount East 807 POLYGON ((-76.58...
2 80500 24510080500 805.0 Clifton-Berea 805 POLYGON ((-76.58...
3 150500 24510150500 1505.0 Greater Mondawmin 1505 POLYGON ((-76.65...
4 120100 24510120100 1201.0 North Baltimore/... 1201 POLYGON ((-76.60...
{% endraw %}

Approach 2: Example 2: BANKS

This example is using data constructed at the end of Tutorial 1.

Be sure to access the menu in the left drawer, hit the 'Files' tab and upload it.

{% raw %}
# Primary Table
# Description: I created a public dataset from a google xlsx sheet 'Bank Addresses and Census Tract' from a workbook of the same name.
# Table: FDIC Baltimore Banks
# Columns: Bank Name, Address(es), Census Tract
left_ds = 'https://docs.google.com/spreadsheets/d/e/2PACX-1vTViIZu-hbvhM3L7dIRAG95ISa7TNhUwdzlYxYzc1ygJoaYc3_scaXHe8Rtj5iwNA/pub?gid=1078028768&single=true&output=csv'
left_col = 'Census Tract'

# Alternate Primary Table
# Description: Same workbook, different Sheet: 'Branches per tract' 
# Columns: Census Tract, Number branches per tract
# left_ds = 'https://docs.google.com/spreadsheets/d/e/2PACX-1vSHFrRSHva1f82ZQ7Uxwf3A1phqljj1oa2duGlZDM1vLtrm1GI5yHmpVX2ilTfMHQ/pub?gid=1698745725&single=true&output=csv'
# lef_col = 'Number branches per tract'

# Crosswalk Table
# Table: Crosswalk Census Communities
# 'TRACT2010', 'GEOID2010', 'CSA2010'
crosswalk_ds = 'https://docs.google.com/spreadsheets/d/e/2PACX-1vREwwa_s8Ix39OYGnnS_wA8flOoEkU7reIV4o3ZhlwYhLXhpNEvnOia_uHUDBvnFptkLLHHlaQNvsQE/pub?output=csv'
use_crosswalk = True
crosswalk_left_col = 'TRACT2010'
crosswalk_right_col = 'GEOID2010'

# Secondary Table
# Table: Baltimore Boundaries
# 'TRACTCE10', 'GEOID10', 'CSA', 'NAME10', 'Tract', 'geometry'
right_ds = 'https://docs.google.com/spreadsheets/d/e/2PACX-1vQ8xXdUaT17jkdK0MWTJpg3GOy6jMWeaXTlguXNjCSb8Vr_FanSZQRaTU-m811fQz4kyMFK5wcahMNY/pub?gid=886223646&single=true&output=csv'
right_col ='GEOID10'

merge_how = 'geometry'
interactive = True
merge_how = 'outer'

banksPd = mergeDatasets( left_ds=left_ds, left_col=left_col, 
              use_crosswalk=use_crosswalk, crosswalk_ds=crosswalk_ds,
              crosswalk_left_col = crosswalk_left_col, crosswalk_right_col = crosswalk_right_col,
              right_ds=right_ds, right_col=right_col, 
              merge_how=merge_how, interactive = interactive )
 Handling Left Dataset
retrieveDatasetFromUrl https://docs.google.com/spreadsheets/d/e/2PACX-1vTViIZu-hbvhM3L7dIRAG95ISa7TNhUwdzlYxYzc1ygJoaYc3_scaXHe8Rtj5iwNA/pub?gid=1078028768&single=true&output=csv
checkDataSetExists False
retrieveDatasetFromUrl https://docs.google.com/spreadsheets/d/e/2PACX-1vTViIZu-hbvhM3L7dIRAG95ISa7TNhUwdzlYxYzc1ygJoaYc3_scaXHe8Rtj5iwNA/pub?gid=1078028768&single=true&output=csv
checkDataSetExists True
checkDataSetExists True
checkDataSetExists True
Left Dataset and Columns are Valid

 Handling Right Dataset
retrieveDatasetFromUrl https://docs.google.com/spreadsheets/d/e/2PACX-1vQ8xXdUaT17jkdK0MWTJpg3GOy6jMWeaXTlguXNjCSb8Vr_FanSZQRaTU-m811fQz4kyMFK5wcahMNY/pub?gid=886223646&single=true&output=csv
checkDataSetExists False
retrieveDatasetFromUrl https://docs.google.com/spreadsheets/d/e/2PACX-1vQ8xXdUaT17jkdK0MWTJpg3GOy6jMWeaXTlguXNjCSb8Vr_FanSZQRaTU-m811fQz4kyMFK5wcahMNY/pub?gid=886223646&single=true&output=csv
checkDataSetExists True
checkDataSetExists True
checkDataSetExists True
Right Dataset and Columns are Valid

 Checking the merge_how Parameter
merge_how operator is Valid outer
checkDataSetExists False

 Checking the Crosswalk Parameter

 Handling Crosswalk Left Dataset Loading
retrieveDatasetFromUrl https://docs.google.com/spreadsheets/d/e/2PACX-1vREwwa_s8Ix39OYGnnS_wA8flOoEkU7reIV4o3ZhlwYhLXhpNEvnOia_uHUDBvnFptkLLHHlaQNvsQE/pub?output=csv
checkDataSetExists False
retrieveDatasetFromUrl https://docs.google.com/spreadsheets/d/e/2PACX-1vREwwa_s8Ix39OYGnnS_wA8flOoEkU7reIV4o3ZhlwYhLXhpNEvnOia_uHUDBvnFptkLLHHlaQNvsQE/pub?output=csv
checkDataSetExists True
checkDataSetExists True
checkDataSetExists True

 Handling Crosswalk Right Dataset Loading
retrieveDatasetFromUrl https://docs.google.com/spreadsheets/d/e/2PACX-1vREwwa_s8Ix39OYGnnS_wA8flOoEkU7reIV4o3ZhlwYhLXhpNEvnOia_uHUDBvnFptkLLHHlaQNvsQE/pub?output=csv
checkDataSetExists False
retrieveDatasetFromUrl https://docs.google.com/spreadsheets/d/e/2PACX-1vREwwa_s8Ix39OYGnnS_wA8flOoEkU7reIV4o3ZhlwYhLXhpNEvnOia_uHUDBvnFptkLLHHlaQNvsQE/pub?output=csv
checkDataSetExists True
checkDataSetExists True
checkDataSetExists True

 Assessment Completed

 Ensuring Left->Crosswalk compatability
Converting Local Key from float64 to Int

 Ensuring Crosswalk->Right compatability
PERFORMING MERGE LEFT->CROSSWALK
left_on TRACT2010 right_on GEOID2010 how outer

 Local Column Values Not Matched 
[-1321321321321325            400100            401101            401102
            401507            403401            403500            403803
            411306            411406            411408            420100
            420301            420701            420800            430800
            430900            440100            440200            440702
            441101            450300            452000            490601
            490602            491100            491201            491600
            492300            750101]
43

 Crosswalk Unique Column Values
[ 10100  10200  10300  10400  10500  20100  20200  20300  30100  30200
  40100  40200  60100  60200  60300  60400  70100  70200  70300  70400
  80101  80102  80200  80301  80302  80400  80500  80600  80700  80800
  90100  90200  90300  90400  90500  90600  90700  90800  90900 100100
 100200 100300 110100 110200 120100 120201 120202 120300 120400 120500
 120600 120700 130100 130200 130300 130400 130600 130700 130803 130804
 130805 130806 140100 140200 140300 150100 150200 150300 150400 150500
 150600 150701 150702 150800 150900 151000 151100 151200 151300 160100
 160200 160300 160400 160500 160600 160700 160801 160802 170100 170200
 170300 180100 180200 180300 190100 190200 190300 200100 200200 200300
 200400 200500 200600 200701 200702 200800 210100 210200 220100 230100
 230200 230300 240100 240200 240300 240400 250101 250102 250103 250203
 250204 250205 250206 250207 250301 250303 250401 250402 250500 250600
 260101 260102 260201 260202 260203 260301 260302 260303 260401 260402
 260403 260404 260501 260604 260605 260700 260800 260900 261000 261100
 270101 270102 270200 270301 270302 270401 270402 270501 270502 270600
 270701 270702 270703 270801 270802 270803 270804 270805 270901 270902
 270903 271001 271002 271101 271102 271200 271300 271400 271501 271503
 271600 271700 271801 271802 271900 272003 272004 272005 272006 272007
 280101 280102 280200 280301 280302 280401 280402 280403 280404 280500
  10000]
PERFORMING MERGE LEFT->RIGHT
left_col GEOID2010 right_col GEOID10 how outer
/usr/local/lib/python3.6/dist-packages/pandas/core/ops/array_ops.py:253: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison
  res_values = method(rvalues)
{% endraw %} {% raw %}
# The attributes are what we will use.
in_crs = 2248 # The CRS we recieve our data 
out_crs = 4326 # The CRS we would like to have our data represented as
geom = 'geometry' # The column where our spatial information lives.

# Description: This was created in the previous tutorial. 
# Description: It can be accessed via accessing a google drive, or by using the upload file feature in the left hand menu. 
# Columns: Bank Information with TRACT, CSA, and GEOMETRY columns.

# To create this dataset I had to commit a full outer join in the previous tutorial. 
# In this way geometries will be included even if there merge does not have a direct match. 
# What this will do is that it means at least one (near) empty record for each community will exist that includes (at minimum) the geographic information and name of a Community.
# That way if no point level information existed in the community, that during the merge the geoboundaries are still carried over.

# If a user wanted to create a heatmap of this data, they would first have to perform an aggregation of their columns onto unique geometry columns.
# It would be the aggregate of of a column that gets colorized on the heatmap. 
# Aggregation operations can easily be performed using a pivot table in XL. 
# I hope to embed support for this functionality in the future. 
# Heatmaps are covered in the next tutorial. 
# Pre-Aggregated information is required to continue on to the next tutorial.
#url = 'https://docs.google.com/spreadsheets/d/e/2PACX-1vTViIZu-hbvhM3L7dIRAG95ISa7TNhUwdzlYxYzc1ygJoaYc3_scaXHe8Rtj5iwNA/pub?gid=1078028768&single=true&output=csv'

# Read in the dataframe
#df = pd.read_csv(url)

# Convert the geometry column datatype from a string of text into a coordinate datatype
banksPd[geom] = banksPd[geom].apply(lambda x: loads( str(x) ))

# Process the dataframe as a geodataframe with a known CRS and geom column
banksGdf = GeoDataFrame(banksPd, crs=in_crs, geometry=geom)
{% endraw %} {% raw %}
banksGdf.plot()
<matplotlib.axes._subplots.AxesSubplot at 0x7fdb3663c908>
{% endraw %} {% raw %}
banksGdf.head()
Bank Name Address(es) Census Tract GEOID2010 TRACTCE10 GEOID10 NAME10 CSA Tract geometry
0 Arundel Federal ... 333 E. Patapsco ... 250401.0 2.45e+10 250401 24510250401 2504.01 Brooklyn/Curtis ... 2504 POLYGON ((-76.59...
1 NaN 3601 S Hanover St 250401.0 2.45e+10 250401 24510250401 2504.01 Brooklyn/Curtis ... 2504 POLYGON ((-76.59...
2 Bank of America,... 20 N Howard St 40100.0 2.45e+10 40100 24510040100 401.00 Downtown/Seton Hill 401 POLYGON ((-76.61...
3 NaN 100 S Charles St... 40100.0 2.45e+10 40100 24510040100 401.00 Downtown/Seton Hill 401 POLYGON ((-76.61...
4 Branch Banking a... 2 N CHARLES ST 40100.0 2.45e+10 40100 24510040100 401.00 Downtown/Seton Hill 401 POLYGON ((-76.61...
{% endraw %}

Lets aggregate by banks by CSA just for fun, huh?

{% raw %}
banksGdf['CSA'].value_counts(normalize=False, sort=True, ascending=False, bins=None, dropna=True).head()
Downtown/Seton Hill                     16
Southwest Baltimore                      8
Oldtown/Middle East                      8
Medfield/Hampden/Woodberry/Remington     8
Fells Point                              7
Name: CSA, dtype: int64
{% endraw %}

Thats interesting. Lets see if we can map that!

{% raw %}
# Aggregate dataframe by CSA
banksGdf['banksCount'] = 1
banksCount = banksGdf.groupby('CSA').sum(numeric_only=True) 
banksCount = banksCount.reset_index()
banksCount.head()
CSA Census Tract GEOID2010 TRACTCE10 GEOID10 NAME10 Tract banksCount
0 Allendale/Irving... 0.0 0.00e+00 1333309 147061333309 13333.09 13333 6
1 Beechfield/Ten H... 560802.0 4.90e+10 1091306 98041091306 10913.06 10913 4
2 Belair-Edison 0.0 0.00e+00 680806 98040680806 6808.06 6808 4
3 Brooklyn/Curtis ... 500802.0 4.90e+10 1252304 122551252304 12523.04 12523 5
4 Canton 20500.0 4.90e+10 30800 73530030800 308.00 308 3
{% endraw %}

So now we have the count in a dataframe..

{% raw %}
# A url to a public Dataset
csa_gdf = gpd.read_file("https://opendata.arcgis.com/datasets/b738a8587b6d479a8824d937892701d8_0.geojson");
csa_gdf.head()
csa_gdf.columns
OBJECTID CSA2010 tpop10 male10 female10 paa17 pwhite17 pasi17 p2more17 ppac17 phisp17 racdiv17 age5_17 age18_17 age24_17 age64_17 age65_17 hhs10 femhhs17 fam17 hhsize10 mhhi17 hh25inc17 hh40inc17 hh60inc17 hh75inc17 hhm7517 hhpov17 hhchpov17 Shape__Area Shape__Length geometry
0 1 Allendale/Irving... 16726 7657 9069 90.28 6.53 0.11 1.00 0.00 2.06 20.12 6.61 17.37 9.00 53.29 13.72 6098 71.13 35.20 2.64 39495.63 32.99 17.72 19.91 11.95 17.43 20.70 32.77 6.38e+07 38770.17 POLYGON ((-76.65...
1 2 Beechfield/Ten H... 13391 5985 7406 75.32 18.86 0.42 3.31 0.31 1.78 41.02 7.93 14.58 9.71 55.37 12.42 5076 55.19 26.14 2.40 57572.50 20.42 13.90 18.18 10.87 36.64 10.47 23.92 4.79e+07 37524.95 POLYGON ((-76.69...
2 3 Belair-Edison 17380 7297 10083 85.65 10.03 0.57 1.70 0.81 1.24 27.26 5.42 22.81 8.41 54.45 8.91 6174 77.53 38.27 2.90 39624.48 34.10 16.28 20.07 8.11 21.45 20.27 34.56 4.50e+07 31307.31 POLYGON ((-76.56...
3 4 Brooklyn/Curtis ... 12900 5746 7154 37.96 39.68 2.53 3.61 1.31 14.91 73.93 10.91 16.09 8.25 57.45 7.30 5204 43.39 32.32 2.61 40275.28 31.40 18.32 18.52 9.08 22.68 24.21 46.41 1.76e+08 150987.70 MULTIPOLYGON (((...
4 5 Canton 8326 4094 4232 3.94 85.58 4.38 2.45 0.56 3.09 26.31 5.25 3.12 5.85 75.25 10.53 4310 10.55 11.03 1.86 111891.25 7.41 7.82 9.18 6.43 69.16 3.66 4.02 1.54e+07 23338.61 POLYGON ((-76.57...
Index(['OBJECTID', 'CSA2010', 'tpop10', 'male10', 'female10', 'paa17',
       'pwhite17', 'pasi17', 'p2more17', 'ppac17', 'phisp17', 'racdiv17',
       'age5_17', 'age18_17', 'age24_17', 'age64_17', 'age65_17', 'hhs10',
       'femhhs17', 'fam17', 'hhsize10', 'mhhi17', 'hh25inc17', 'hh40inc17',
       'hh60inc17', 'hh75inc17', 'hhm7517', 'hhpov17', 'hhchpov17',
       'Shape__Area', 'Shape__Length', 'geometry'],
      dtype='object')
{% endraw %} {% raw %}
# merge it to our banks dataset
merged_banks = pd.merge(banksCount, csa_gdf, left_on='CSA', right_on='CSA2010', how='left')
merged_banks.head()
CSA Census Tract GEOID2010 TRACTCE10 GEOID10 NAME10 Tract banksCount OBJECTID CSA2010 tpop10 male10 female10 paa17 pwhite17 pasi17 p2more17 ppac17 phisp17 racdiv17 age5_17 age18_17 age24_17 age64_17 age65_17 hhs10 femhhs17 fam17 hhsize10 mhhi17 hh25inc17 hh40inc17 hh60inc17 hh75inc17 hhm7517 hhpov17 hhchpov17 Shape__Area Shape__Length geometry
0 Allendale/Irving... 0.0 0.00e+00 1333309 147061333309 13333.09 13333 6 1.0 Allendale/Irving... 16726.0 7657.0 9069.0 90.28 6.53 0.11 1.00 0.00 2.06 20.12 6.61 17.37 9.00 53.29 13.72 6098.0 71.13 35.20 2.64 39495.63 32.99 17.72 19.91 11.95 17.43 20.70 32.77 6.38e+07 38770.17 POLYGON ((-76.65...
1 Beechfield/Ten H... 560802.0 4.90e+10 1091306 98041091306 10913.06 10913 4 2.0 Beechfield/Ten H... 13391.0 5985.0 7406.0 75.32 18.86 0.42 3.31 0.31 1.78 41.02 7.93 14.58 9.71 55.37 12.42 5076.0 55.19 26.14 2.40 57572.50 20.42 13.90 18.18 10.87 36.64 10.47 23.92 4.79e+07 37524.95 POLYGON ((-76.69...
2 Belair-Edison 0.0 0.00e+00 680806 98040680806 6808.06 6808 4 3.0 Belair-Edison 17380.0 7297.0 10083.0 85.65 10.03 0.57 1.70 0.81 1.24 27.26 5.42 22.81 8.41 54.45 8.91 6174.0 77.53 38.27 2.90 39624.48 34.10 16.28 20.07 8.11 21.45 20.27 34.56 4.50e+07 31307.31 POLYGON ((-76.56...
3 Brooklyn/Curtis ... 500802.0 4.90e+10 1252304 122551252304 12523.04 12523 5 4.0 Brooklyn/Curtis ... 12900.0 5746.0 7154.0 37.96 39.68 2.53 3.61 1.31 14.91 73.93 10.91 16.09 8.25 57.45 7.30 5204.0 43.39 32.32 2.61 40275.28 31.40 18.32 18.52 9.08 22.68 24.21 46.41 1.76e+08 150987.70 MULTIPOLYGON (((...
4 Canton 20500.0 4.90e+10 30800 73530030800 308.00 308 3 5.0 Canton 8326.0 4094.0 4232.0 3.94 85.58 4.38 2.45 0.56 3.09 26.31 5.25 3.12 5.85 75.25 10.53 4310.0 10.55 11.03 1.86 111891.25 7.41 7.82 9.18 6.43 69.16 3.66 4.02 1.54e+07 23338.61 POLYGON ((-76.57...
{% endraw %} {% raw %}
# Lets check what datatype our geometry column is before we try to convert it!
merged_banks.geometry.dtype
<geopandas.array.GeometryDtype at 0x7fdb49c56320>
{% endraw %} {% raw %}
# Process the dataframe as a geodataframe with a known CRS and geom column
# Since the geometry column is already being interpreted as a geopandas dtype,
# we should readily be able to convert the dataframe without fuss.
banksCountgdf = GeoDataFrame(merged_banks, crs=2248, geometry='geometry')
{% endraw %} {% raw %}
# If you'd like, drop duplicate columns like so.
# merged_df = merged_df.drop('CSA', axis=1)
{% endraw %} {% raw %}
# In order for this choropleth to work, the total number of banks in each csa must be tallied.
# This can be done programmatically, but i havent added the code. 
# The column needs to be changed from CSA to whatever this new tallied column is named.
banksCountgdf.plot( column='banksCount', legend=True)
<matplotlib.axes._subplots.AxesSubplot at 0x7fdb4dda9160>
{% endraw %} {% raw %}
dxp.bar(x='CSA2010', y='banksCount', data=banksCountgdf)
findfont: Font family ['Helvetica'] not found. Falling back to DejaVu Sans.
{% endraw %}

Go ahead and update the form and run the cell to explore the data visually!

{% raw %}
tempBanksNoGeom = banksCountgdf.drop('geometry', axis=1)
{% endraw %} {% raw %}
#@title Form fields to view spatial data
#@markdown Forms support many types of fields.

legendOn = "True"  #@param ['True', 'False']
displayCol = "banksCount" #@param ["banksCount", 'tpop10', 'male10', 'female10', 'paa17', 'pwhite17', 'pasi17', 'p2more17', 'ppac17', 'phisp17', 'racdiv17', 'age5_17', 'age18_17', 'age24_17', 'age64_17', 'age65_17', 'hhs10', 'femhhs17', 'fam17', 'hhsize10', 'mhhi17', 'hh25inc17', 'hh40inc17', 'hh60inc17', 'hh75inc17', 'hhm7517', 'hhpov17', 'hhchpov17"] {allow-input: true}
#@markdown ---

banksCountgdf.plot( column=displayCol, legend=legendOn)
dxp.bar(x='CSA2010', 
        y=displayCol, 
        data=banksCountgdf)
<matplotlib.axes._subplots.AxesSubplot at 0x7fdb3625ec88>
{% endraw %} {% raw %}
banksCountgdf.dropna().CSA2010.values
array(['Allendale/Irvington/S. Hilton', 'Beechfield/Ten Hills/West Hills',
       'Belair-Edison', 'Brooklyn/Curtis Bay/Hawkins Point', 'Canton',
       'Cedonia/Frankford', 'Cherry Hill', 'Chinquapin Park/Belvedere',
       'Claremont/Armistead', 'Clifton-Berea', 'Cross-Country/Cheswolde',
       'Dickeyville/Franklintown', 'Dorchester/Ashburton',
       'Downtown/Seton Hill', 'Edmondson Village', 'Fells Point',
       'Forest Park/Walbrook', 'Glen-Fallstaff',
       'Greater Charles Village/Barclay', 'Greater Govans',
       'Greater Mondawmin', 'Greater Roland Park/Poplar Hill',
       'Greater Rosemont', 'Greenmount East', 'Hamilton',
       'Harbor East/Little Italy', 'Harford/Echodale', 'Highlandtown',
       'Howard Park/West Arlington', 'Inner Harbor/Federal Hill',
       'Lauraville', 'Loch Raven', 'Madison/East End',
       'Medfield/Hampden/Woodberry/Remington', 'Midtown',
       'Midway/Coldstream', 'Morrell Park/Violetville',
       'Mount Washington/Coldspring', 'North Baltimore/Guilford/Homeland',
       'Northwood', 'Oldtown/Middle East',
       'Orangeville/East Highlandtown', 'Patterson Park North & East',
       'Penn North/Reservoir Hill', 'Pimlico/Arlington/Hilltop',
       'Poppleton/The Terraces/Hollins Market',
       'Sandtown-Winchester/Harlem Park', 'South Baltimore',
       'Southeastern', 'Southern Park Heights', 'Southwest Baltimore',
       'The Waverlies', 'Upton/Druid Heights',
       'Washington Village/Pigtown', 'Westport/Mount Winans/Lakeland'],
      dtype=object)
{% endraw %} {% raw %}
#@title Now lets compare some indicators
#@markdown Forms support many types of fields.

legendOn = "True"  #@param ['True', 'False']
displayColx = "banksCount" #@param ["banksCount", 'tpop10', 'male10', 'female10', 'paa17', 'pwhite17', 'pasi17', 'p2more17', 'ppac17', 'phisp17', 'racdiv17', 'age5_17', 'age18_17', 'age24_17', 'age64_17', 'age65_17', 'hhs10', 'femhhs17', 'fam17', 'hhsize10', 'mhhi17', 'hh25inc17', 'hh40inc17', 'hh60inc17', 'hh75inc17', 'hhm7517', 'hhpov17', 'hhchpov17"] {allow-input: true}
displayColy = "female10" #@param ["banksCount", 'tpop10', 'male10', 'female10', 'paa17', 'pwhite17', 'pasi17', 'p2more17', 'ppac17', 'phisp17', 'racdiv17', 'age5_17', 'age18_17', 'age24_17', 'age64_17', 'age65_17', 'hhs10', 'femhhs17', 'fam17', 'hhsize10', 'mhhi17', 'hh25inc17', 'hh40inc17', 'hh60inc17', 'hh75inc17', 'hhm7517', 'hhpov17', 'hhchpov17"] {allow-input: true}
# displayColm = "Westport/Mount Winans/Lakeland" #@param ['Allendale/Irvington/S. Hilton', 'Beechfield/Ten Hills/West Hills', 'Belair-Edison', 'Brooklyn/Curtis Bay/Hawkins Point', 'Canton', 'Cedonia/Frankford', 'Cherry Hill', 'Chinquapin Park/Belvedere', 'Claremont/Armistead', 'Clifton-Berea', 'Cross-Country/Cheswolde', 'Dickeyville/Franklintown', 'Dorchester/Ashburton', 'Downtown/Seton Hill', 'Edmondson Village', 'Fells Point', 'Forest Park/Walbrook', 'Glen-Fallstaff', 'Greater Charles Village/Barclay', 'Greater Govans', 'Greater Mondawmin', 'Greater Roland Park/Poplar Hill', 'Greater Rosemont', 'Greenmount East', 'Hamilton', 'Harbor East/Little Italy', 'Harford/Echodale', 'Highlandtown', 'Howard Park/West Arlington', 'Inner Harbor/Federal Hill', 'Lauraville', 'Loch Raven', 'Madison/East End', 'Medfield/Hampden/Woodberry/Remington', 'Midtown', 'Midway/Coldstream', 'Morrell Park/Violetville', 'Mount Washington/Coldspring', 'North Baltimore/Guilford/Homeland', 'Northwood', 'Oldtown/Middle East', 'Orangeville/East Highlandtown', 'Patterson Park North & East', 'Penn North/Reservoir Hill', 'Pimlico/Arlington/Hilltop', 'Poppleton/The Terraces/Hollins Market', 'Sandtown-Winchester/Harlem Park', 'South Baltimore', 'Southeastern', 'Southern Park Heights', 'Southwest Baltimore', 'The Waverlies', 'Upton/Druid Heights', 'Washington Village/Pigtown', 'Westport/Mount Winans/Lakeland'] {allow-input: true}
#@markdown ---

# print('Comparing', displayColm, "'s Indicators ", displayColx, " and " , displayColy )
dxp.kde(x=displayColx, y=displayColy, data= banksCountgdf.dropna() )
{% endraw %} {% raw %}
# dxp.kde(x=displayColx, y=displayColy, data= banksCountgdf[ banksCountgdf.CSA2010 != displayColm ].dropna() )
{% endraw %}

Approach 3: Method 1: Convert Column(s) to Coordinate

Later in this tutorial we will show how you may find geometric bounds that coorespond with a points location ( points in polygons ). This can be a useful trick when we we want to create a heatmap of point data with specified boundaries.

If a user wanted to create a heatmap of this data...

they would first have to perform an aggregation of their columns onto unique geometry columns

Possible Path: (points in polygons -> crosswalk -> merge GeoJson).

It would be the aggregate of of a column that gets colorized on the heatmap.

Aggregation operations can easily be performed using a pivot table in XL.

I hope to embed support for this functionality in the future.

Heatmaps are covered in the next tutorial.

Pre-Aggregated information is required to continue onto the next tutorial.

Approach 3: Method 1: Example 1:

This is the generic example but it wont work since no URL is given.

{% raw %}
# More Information: https://geopandas.readthedocs.io/en/latest/gallery/create_geopandas_from_pandas.html#from-longitudes-and-latitudes

# If your data has coordinates in two columns run this cell
# It will create a geometry column from the two.
# A public dataset is not provided for this example and will not run.

# Load DF HERE. Accidently deleted the link. Need to refind. 
# Just rely on example 2 for now. 
exe_df = ''

exe_df['x'] = pd.to_numeric(exe_df['x'], errors='coerce')
exe_df['y'] = pd.to_numeric(exe_df['y'], errors='coerce')
# exe_df = exe_df.replace(np.nan, 0, regex=True)

# An example of loading in an internal BNIA file
crs = {'init' :'epsg:2248'} 
geometry=[Point(xy) for xy in zip(exe_df.x, exe_df.y)]
exe_gdf = gpd.GeoDataFrame( exe_df.drop(['x', 'y'], axis=1), crs=in_crs, geometry=geometry)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-31-524fd9aa5e67> in <module>()
      9 exe_df = ''
     10 
---> 11 exe_df['x'] = pd.to_numeric(exe_df['x'], errors='coerce')
     12 exe_df['y'] = pd.to_numeric(exe_df['y'], errors='coerce')
     13 # exe_df = exe_df.replace(np.nan, 0, regex=True)

TypeError: string indices must be integers
{% endraw %}

Approach 3: Method 1: Example 2: FOOD BANK PANTRIES

{% raw %}
# Alternate Primary Table
# Table: Food Bank And Pantry Sites, 
# XLSX Sheet: Baltimore City Pantries Tracts
# Columns: X	Y	OBJECTID	Name	Address	City_1	State	Zip	# in Zip	FIPS
url = 'https://docs.google.com/spreadsheets/d/e/2PACX-1vT3lG0n542sIGE2O-C8fiXx-qUZG2WDO6ezRGcNsS4z8MM30XocVZ90P1UQOIXO2w/pub?gid=1152681223&single=true&output=csv'

# Read in the dataframe
food_df = pd.read_csv(url)
{% endraw %} {% raw %}
food_df['X'] = pd.to_numeric(food_df['X'], errors='coerce')
food_df['Y'] = pd.to_numeric(food_df['Y'], errors='coerce')
# df = df.replace(np.nan, 0, regex=True)

# An example of loading in an internal BNIA file
crs = {'init' :'epsg:2248'} 
geometry=[Point(xy) for xy in zip(food_df['X'], food_df['Y'])]
food_gdf = gpd.GeoDataFrame( food_df.drop(['X', 'Y'], axis=1), crs=in_crs, geometry=geometry)
{% endraw %} {% raw %}
food_gdf.head()
OBJECTID Name Address City_1 State Zip # in Zip FIPS geometry
0 1 Victory Forest -... 10000 Brunswick ... Silver Spring MD 20817 NaN NaN POINT (-77.05673...
1 2 Glassmanor Eleme... 1011 Marcy Avenue Oxon Hill MD 20745 NaN NaN POINT (-76.99036...
2 3 Apple Blossoms 1013 Cresthaven ... Silver Spring MD 20903 NaN NaN POINT (-76.99155...
3 4 Lakeview Apartme... 10250 West Lake Dr. Bethesda MD 20817 NaN NaN POINT (-77.14929...
4 5 Central Gardens 103 Cindy Lane Capitol Heights MD NaN NaN POINT (-76.88974...
{% endraw %} {% raw %}
food_gdf.plot()
<matplotlib.axes._subplots.AxesSubplot at 0x7fdb3473c3c8>
{% endraw %}

Approach 3: Method 2: Geocoding Addresses to Coordinates

This function takes a while. The less columns/data/records the faster it execues.

{% raw %}
# More information vist: https://geopy.readthedocs.io/en/stable/#module-geopy.geocoders

# In this example we retrieve and map a dataset with no lat/lng but containing an address

# The url listed below is public.
url = 'https://docs.google.com/spreadsheets/d/e/2PACX-1vTMgdqWykZeIsMwCFllPuG1cd4gGDB6BUqaAOM0Lx9VGdCo2JJy9v_CR8ZaEDWO3Q/pub?gid=290715815&single=true&output=csv'
df = pd.read_csv(url)

# In this example our data is stored in the 'STREET' attribute
addrCol = 'STREET'
geometry = []
geolocator = Nominatim(user_agent="specify_your_app_name_here")

for index, row in df.iterrows():
  # We will try and return an address for each Street Name
  try: 
      # retrieve the geocoded information of our street address
      geol = geolocator.geocode(row[addrCol], timeout=None)
      
      print('Geocoding: ', location.address) 
      # print(location.raw)
      
      # create a mappable coordinate point from the response object's lat/lang values.
      pnt = Point(geol.longitude, geol.latitude)
      
      # Append this value to the list of geometries
      geometry.append(pnt)
      
  except: 
      # If no street name was found decide what to do here.
      # df.loc[index]['geom'] = Point(0,0) # Alternate method
      geometry.append(Point(0,0))
      
# Finally, we stuff the geometry data we created back into the dataframe
df['geometry'] = geometry

# Convert the dataframe into a geodataframe and map it!
gdf = gpd.GeoDataFrame( df, geometry=geometry)
{% endraw %} {% raw %}
food_gdf2.head()
{% endraw %}

Approach 4: Connecting to a PostGIS database

In the following example pulls point geodata from a Postgres database.

We will pull the postgres point data in two manners.

  • SQL query where an SQL query uses ST_Transform(the_geom,4326) to transform the_geom's CRS from a DATABASE Binary encoding into standard Lat Long's
  • Using a plan SQL query and performing the conversion using gpd.io.sql.read_postgis() to pull the data in as 2248 and convert the CRS using .to_crs(epsg=4326)
  • These examples will not work in colabs as their is no local database to connect to and has been commented out for that reason
{% raw %}
# This Notebook can be downloaded to connect to a database
'''
conn = psycopg2.connect(host='', dbname='', user='', password='', port='')

# DB Import Method One
sql1 = 'SELECT the_geom, gid, geogcode, ooi, address, addrtyp, city, block, lot, desclu, existing FROM housing.mdprop_2017v2 limit 100;'
pointData = gpd.io.sql.read_postgis(sql1, conn, geom_col='the_geom', crs=2248)
pointData = pointData.to_crs(epsg=4326)

# DB Import Method Two
sql2 = 'SELECT ST_Transform(the_geom,4326) as the_geom, ooi, desclu, address FROM housing.mdprop_2017v2;'
pointData = gpd.GeoDataFrame.from_postgis(sql2, conn, geom_col='the_geom', crs=4326)
pointData.head()
pointData.plot()
'''
{% endraw %}

Basics Operations

Inspection

{% raw %}
def geomSummary(gdf): return type(gdf), gdf.crs, gdf.columns;
# for p in df['Tract'].sort_values(): print(p)
geomSummary(csa_gdf)
(geopandas.geodataframe.GeoDataFrame, <Geographic 2D CRS: EPSG:4326>
 Name: WGS 84
 Axis Info [ellipsoidal]:
 - Lat[north]: Geodetic latitude (degree)
 - Lon[east]: Geodetic longitude (degree)
 Area of Use:
 - name: World
 - bounds: (-180.0, -90.0, 180.0, 90.0)
 Datum: World Geodetic System 1984
 - Ellipsoid: WGS 84
 - Prime Meridian: Greenwich, Index(['OBJECTID', 'CSA2010', 'tpop10', 'male10', 'female10', 'paa17',
        'pwhite17', 'pasi17', 'p2more17', 'ppac17', 'phisp17', 'racdiv17',
        'age5_17', 'age18_17', 'age24_17', 'age64_17', 'age65_17', 'hhs10',
        'femhhs17', 'fam17', 'hhsize10', 'mhhi17', 'hh25inc17', 'hh40inc17',
        'hh60inc17', 'hh75inc17', 'hhm7517', 'hhpov17', 'hhchpov17',
        'Shape__Area', 'Shape__Length', 'geometry'],
       dtype='object'))
{% endraw %}

Converting CRS

{% raw %}
# Convert the CRS of the dataset into one you desire
# The gdf must be loaded with a known crs in order for the to_crs conversion to work
# We use this often to converting BNIAs custom CRS to the common type 
out_crs = 4326
csa_gdf = csa_gdf.to_crs(epsg=out_crs)
{% endraw %}

Saving

{% raw %}
# Here is code to comit a simple save
filename = 'TEST_FILE_NAME'
csa_gdf.to_file(f"{filename}.geojson", driver='GeoJSON')
{% endraw %} {% raw %}
# Here is code to save this new projection as a geojson file and read it back in
csa_gdf = csa_gdf.to_crs(epsg=2248) #just making sure
csa_gdf.to_file(filename+'.shp', driver='ESRI Shapefile')
csa_gdf = gpd.read_file(filename+'.shp')
{% endraw %}

Geometric Manipulations

Draw Tool

{% raw %}
import folium
from folium.plugins import Draw
# Draw tool. Create and export your own boundaries
m = folium.Map()
draw = Draw()
draw.add_to(m)
m = folium.Map(location=[-27.23, -48.36], zoom_start=12)
draw = Draw(export=True)
draw.add_to(m)
# m.save(os.path.join('results', 'Draw1.html'))
m
<folium.plugins.draw.Draw at 0x7fdb346b7470>
<folium.plugins.draw.Draw at 0x7fdb4de43518>
Make this Notebook Trusted to load map: File -> Trust Notebook
{% endraw %}

Boundary

{% raw %}
newcsa = csa_gdf.copy()
newcsa['geometry'] = csa_gdf.boundary
newcsa.plot(column='CSA2010' )
<matplotlib.axes._subplots.AxesSubplot at 0x7fdb340a5668>
{% endraw %}

envelope

{% raw %}
newcsa = csa_gdf.copy()
newcsa['geometry'] = csa_gdf.envelope
newcsa.plot(column='CSA2010' )
<matplotlib.axes._subplots.AxesSubplot at 0x7fdb3402b748>
{% endraw %}

convex_hull

{% raw %}
newcsa = csa_gdf.copy()
newcsa['geometry'] = csa_gdf.convex_hull
newcsa.plot(column='CSA2010' )
# , cmap='OrRd', scheme='quantiles'
# newcsa.boundary.plot(  )
<matplotlib.axes._subplots.AxesSubplot at 0x7fdb33fa22e8>
{% endraw %}

simplify

{% raw %}
newcsa = csa_gdf.copy()
newcsa['geometry'] = csa_gdf.simplify(30)
newcsa.plot(column='CSA2010' )
<matplotlib.axes._subplots.AxesSubplot at 0x7fdb33f86358>
{% endraw %}

buffer

{% raw %}
newcsa = csa_gdf.copy()
newcsa['geometry'] = csa_gdf.buffer(0.01)
newcsa.plot(column='CSA2010' )
<matplotlib.axes._subplots.AxesSubplot at 0x7fdb33edceb8>
{% endraw %}

rotate

{% raw %}
newcsa = csa_gdf.copy()
newcsa['geometry'] = csa_gdf.rotate(30)
newcsa.plot(column='CSA2010' )
<matplotlib.axes._subplots.AxesSubplot at 0x7fdb33ec5c18>
{% endraw %}

scale

{% raw %}
newcsa = csa_gdf.copy()
newcsa['geometry'] = csa_gdf.scale(3, 2)
newcsa.plot(column='CSA2010' )
<matplotlib.axes._subplots.AxesSubplot at 0x7fdb33e34ac8>
{% endraw %}

skew

{% raw %}
newcsa = csa_gdf.copy()
newcsa['geometry'] = csa_gdf.skew(1, 10)
newcsa.plot(column='CSA2010' )
<matplotlib.axes._subplots.AxesSubplot at 0x7fdb33f86b38>
{% endraw %}

Advanced

Operations:

  • Reading in data (points/ geoms) -- Convert lat/lng columns to point coordinates -- Geocoding address to coordinates -- Changing coordinate reference systems -- Connecting to PostGisDB's
  • Basic Operations
  • Saving shape data
  • Get Polygon Centroids
  • Working with Points and Polygons -- Map Points and Polygons -- Get Points in Polygons

Input(s):

  • Dataset (points/ bounds) url
  • Points/ bounds geometry column(s)
  • Points/ bounds crs's
  • Points/ bounds mapping color(s)
  • New filename

Output: File

Create Geospatial Functions

This function will handle common geo spatial exploratory methods. It covers everything discussed in the basic operations and more!

{% raw %}
{% endraw %} {% raw %}

workWithGeometryData[source]

workWithGeometryData(method=False, df=False, polys=False, ptsCoordCol=False, polygonsCoordCol=False, polyColorCol=False, polygonsLabel='polyOnPoint', pntsClr='red', polysClr='white')

{% endraw %} {% raw %}
{% endraw %} {% raw %}

map_points[source]

map_points(df, lat_col='latitude', lon_col='longitude', zoom_start=11, plot_points=False, pt_radius=15, draw_heatmap=False, heat_map_weights_col=None, heat_map_weights_normalize=True, heat_map_radius=15)

Creates a map given a dataframe of points. Can also produce a heatmap overlay

Arg: df: dataframe containing points to maps lat_col: Column containing latitude (string) lon_col: Column containing longitude (string) zoom_start: Integer representing the initial zoom of the map plot_points: Add points to map (boolean) pt_radius: Size of each point draw_heatmap: Add heatmap to map (boolean) heat_map_weights_col: Column containing heatmap weights heat_map_weights_normalize: Normalize heatmap weights (boolean) heat_map_radius: Size of heatmap point

Returns: folium map object

{% endraw %}

Processing Geometry is tedius enough to merit its own handler

{% raw %}
{% endraw %} {% raw %}

readInGeometryData[source]

readInGeometryData(url=False, porg=False, geom=False, lat=False, lng=False, revgeocode=False, save=False, in_crs=4326, out_crs=False)

{% endraw %}

As you can see we have a lot of points. Lets see if there is any better way to visualize this.

Example: Using the advanced Functions

Simple Examples

Example 0: Loading describing and plotting a simple shapefile.

{% raw %}
# Example 0: Loading describing and plotting a simple shapefile.
url = 'https://docs.google.com/spreadsheets/d/e/2PACX-1vQ8xXdUaT17jkdK0MWTJpg3GOy6jMWeaXTlguXNjCSb8Vr_FanSZQRaTU-m811fQz4kyMFK5wcahMNY/pub?gid=886223646&single=true&output=csv'

# Lets create a dataframe using the library
geom = readInGeometryData(url=url, geom='geometry', in_crs=2248, out_crs=2248)

# And now lets do it again just for fun using the dataframe we just created from the URL
geomi = readInGeometryData(url=geom, geom='geometry', in_crs=2248, out_crs=2248)
workWithGeometryData('summary', geomi) 
geomi.plot()
RECIEVED url: https://docs.google.com/spreadsheets/d/e/2PACX-1vQ8xXdUaT17jkdK0MWTJpg3GOy6jMWeaXTlguXNjCSb8Vr_FanSZQRaTU-m811fQz4kyMFK5wcahMNY/pub?gid=886223646&single=true&output=csv, 
 porg: g, 
 geom: geometry, 
 lat: False, 
 lng: False, 
 revgeocode: False, 
 in_crs: 2248, 
 out_crs: 2248
Index(['TRACTCE10', 'GEOID10', 'NAME10', 'CSA', 'Tract', 'geometry'], dtype='object')
isGeoDataframe
RECIEVED url:      TRACTCE10  ...                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            geometry
0    151000     ...  POLYGON ((-76.67796 39.33993, -76.67805 39.33983, -76.67813 39.33974, -76.67822 39.33964, -76.67846 39.33928, -76.67854 39.33916, -76.67868 39.33896, -76.67895 39.33861, -76.67912 39.33836, -76.67917 39.33828, -76.67919 39.33823, -76.67919 39.33817, -76.67917 39.33789, -76.67908 39.33704, -76.67898 39.33622, -76.67897 39.33613, -76.67889 39.33531, -76.67880 39.33451, -76.67874 39.33397, -76.67868 39.33344, -76.67863 39.33291, -76.67859 39.33247, -76.67856 39.33216, -76.67856 39.33207, -76.67857 39.33202, -76.67858 39.33199, -76.67861 39.33192, -76.67867 39.33176, -76.67916 39.33077, -76.67966 39.32978, -76.68016 39.32879, -76.68059 39.32793, -76.68107 39.32695, -76.68128 39.32649, -76.68150 39.32603, -76.68173 39.32556, -76.68194 39.32515, -76.68198 39.32511, -76.68200 39.32509, -76.68205 39.32506, -76.68206 39.32505, -76.68215 39.32501, -76.68261 39.32583, -76.68316 39.32566, -76.68337 39.32560, -76.68359 39.32553, -76.68419 39.32535, -76.68439 39.32529, -76.68568 39.32490, -76.68659 39.32463, -76.68707 39.32450, -76.68714 39.32448, -76.68821 39.32416, -76.68863 39.32403, -76.69007 39.32359, -76.68995 39.32369, -76.68960 39.32445, -76.68913 39.32539, -76.68884 39.32596, -76.68881 39.32602, -76.68858 39.32647, -76.68855 39.32653, -76.68828 39.32711, -76.68800 39.32770, -76.68771 39.32826, -76.68742 39.32884, -76.68858 39.32918, -76.68859 39.32951, -76.68864 39.33043, -76.68977 39.33073, -76.69086 39.33101, -76.69064 39.33146, -76.69051 39.33173, -76.69036 39.33202, -76.69007 39.33258, -76.68967 39.33339, -76.68950 39.33372, -76.68936 39.33398, -76.68936 39.33508, -76.68935 39.33563, -76.68935 39.33570, -76.68935 39.33613, -76.68935 39.33620, -76.68934 39.33663, -76.68935 39.33716, -76.68936 39.33768, -76.68936 39.33818, -76.68936 39.33870, -76.68937 39.33922, -76.68937 39.33973, -76.68937 39.34024, -76.68937 39.34037, -76.68938 39.34075, -76.68706 39.34076, -76.68547 39.34076, -76.68448 39.34078, -76.68417 39.34078, -76.68404 39.34078, -76.68347 39.34079, -76.68331 39.34098, -76.68310 39.34122, -76.68305 39.34128, -76.68298 39.34136, -76.68291 39.34144, -76.67934 39.34035, -76.67796 39.33993))                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
1    80700      ...  POLYGON ((-76.58850 39.30868, -76.58841 39.30773, -76.58838 39.30721, -76.58834 39.30669, -76.58832 39.30616, -76.58828 39.30564, -76.58947 39.30561, -76.59002 39.30560, -76.59068 39.30557, -76.59133 39.30555, -76.59252 39.30545, -76.59456 39.30537, -76.59485 39.30536, -76.59523 39.30534, -76.59664 39.30530, -76.59736 39.30526, -76.59807 39.30522, -76.59809 39.30564, -76.59811 39.30623, -76.59812 39.30629, -76.59812 39.30640, -76.59818 39.30736, -76.59828 39.30843, -76.59754 39.30846, -76.59686 39.30850, -76.59595 39.30854, -76.59502 39.30857, -76.59478 39.30858, -76.59395 39.30861, -76.59284 39.30864, -76.59219 39.30866, -76.59153 39.30868, -76.59089 39.30871, -76.59023 39.30875, -76.58934 39.30880, -76.58917 39.30880, -76.58848 39.30881, -76.58850 39.30868))                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
2    80500      ...  POLYGON ((-76.58988 39.31612, -76.59015 39.31586, -76.59031 39.31571, -76.59044 39.31555, -76.59052 39.31544, -76.59056 39.31535, -76.59060 39.31524, -76.59062 39.31507, -76.59061 39.31500, -76.59057 39.31431, -76.58925 39.31434, -76.58881 39.31434, -76.58877 39.31364, -76.58874 39.31331, -76.58869 39.31242, -76.58907 39.31240, -76.58926 39.31239, -76.58947 39.31238, -76.59043 39.31234, -76.59110 39.31231, -76.59175 39.31229, -76.59241 39.31224, -76.59309 39.31222, -76.59420 39.31219, -76.59501 39.31217, -76.59523 39.31216, -76.59692 39.31209, -76.59707 39.31208, -76.59775 39.31204, -76.59847 39.31199, -76.59937 39.31194, -76.59905 39.31296, -76.59897 39.31311, -76.59887 39.31325, -76.59872 39.31344, -76.59852 39.31366, -76.59825 39.31395, -76.59803 39.31418, -76.59794 39.31426, -76.59716 39.31505, -76.59695 39.31526, -76.59645 39.31573, -76.59633 39.31586, -76.59593 39.31626, -76.59548 39.31668, -76.59517 39.31696, -76.59488 39.31724, -76.59204 39.31658, -76.59096 39.31635, -76.58988 39.31612))                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
3    150500     ...  POLYGON ((-76.65756 39.32543, -76.65751 39.32537, -76.65741 39.32521, -76.65727 39.32503, -76.65710 39.32479, -76.65684 39.32444, -76.65622 39.32363, -76.65554 39.32271, -76.65526 39.32234, -76.65491 39.32185, -76.65429 39.32097, -76.65361 39.32006, -76.65247 39.31855, -76.65239 39.31846, -76.65227 39.31831, -76.65215 39.31819, -76.65201 39.31805, -76.65185 39.31791, -76.65165 39.31775, -76.65145 39.31758, -76.65061 39.31682, -76.65049 39.31671, -76.65120 39.31616, -76.65196 39.31575, -76.65223 39.31564, -76.65245 39.31557, -76.65267 39.31552, -76.65290 39.31547, -76.65314 39.31545, -76.65337 39.31543, -76.65362 39.31542, -76.65365 39.31542, -76.65382 39.31542, -76.65401 39.31542, -76.65404 39.31543, -76.65419 39.31543, -76.65441 39.31546, -76.65463 39.31550, -76.65485 39.31554, -76.65495 39.31556, -76.65513 39.31561, -76.65531 39.31566, -76.65555 39.31573, -76.65574 39.31577, -76.65599 39.31581, -76.65639 39.31586, -76.65672 39.31587, -76.65692 39.31587, -76.65709 39.31587, -76.65729 39.31586, -76.65750 39.31584, -76.65760 39.31583, -76.65770 39.31581, -76.65791 39.31577, -76.65824 39.31569, -76.65863 39.31557, -76.65893 39.31549, -76.65978 39.31523, -76.65985 39.31540, -76.66036 39.31625, -76.66046 39.31638, -76.66080 39.31675, -76.66109 39.31707, -76.66118 39.31717, -76.66150 39.31750, -76.66247 39.31848, -76.66319 39.31919, -76.66382 39.31982, -76.66406 39.32008, -76.66421 39.32029, -76.66431 39.32045, -76.66441 39.32063, -76.66450 39.32081, -76.66461 39.32108, -76.66470 39.32143, -76.66475 39.32170, -76.66477 39.32196, -76.66477 39.32207, -76.66477 39.32219, -76.66478 39.32235, -76.66477 39.32248, -76.66473 39.32271, -76.66451 39.32379, -76.66425 39.32528, -76.66418 39.32569, -76.66414 39.32597, -76.66414 39.32625, -76.66413 39.32645, -76.66415 39.32660, -76.66417 39.32678, -76.66419 39.32699, -76.66422 39.32715, -76.66430 39.32751, -76.66397 39.32743, -76.66384 39.32739, -76.66333 39.32726, -76.66159 39.32692, -76.66075 39.32668, -76.65979 39.32662, -76.65972 39.32660, -76.65961 39.32652, -76.65877 39.32582, -76.65791 39.32555, -76.65791 39.32558, -76.65791 39.32563, -76.65790 39.32568, -76.65789 39.32573, -76.65786 39.32577, -76.65759 39.32581, -76.65759 39.32575, -76.65758 39.32569, -76.65756 39.32543))                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        
4    120100     ...  POLYGON ((-76.60954 39.33979, -76.60955 39.33939, -76.60953 39.33867, -76.60953 39.33731, -76.60952 39.33682, -76.60951 39.33640, -76.60951 39.33534, -76.60951 39.33446, -76.60950 39.33370, -76.60949 39.33335, -76.60949 39.33318, -76.60947 39.33262, -76.60947 39.33252, -76.60947 39.33097, -76.60982 39.33096, -76.61000 39.33097, -76.61013 39.33098, -76.61034 39.33101, -76.61045 39.33103, -76.61056 39.33105, -76.61110 39.33118, -76.61134 39.33125, -76.61175 39.33139, -76.61204 39.33149, -76.61246 39.33169, -76.61282 39.33187, -76.61300 39.33196, -76.61346 39.33224, -76.61363 39.33235, -76.61375 39.33244, -76.61384 39.33250, -76.61392 39.33256, -76.61396 39.33261, -76.61400 39.33269, -76.61424 39.33266, -76.61445 39.33265, -76.61470 39.33265, -76.61498 39.33267, -76.61515 39.33252, -76.61570 39.33203, -76.61583 39.33191, -76.61589 39.33184, -76.61622 39.33153, -76.61627 39.33147, -76.61718 39.33205, -76.61776 39.33241, -76.61798 39.33255, -76.61815 39.33266, -76.61878 39.33306, -76.61931 39.33340, -76.61939 39.33345, -76.62009 39.33389, -76.62063 39.33424, -76.62096 39.33446, -76.62212 39.33524, -76.62263 39.33557, -76.62296 39.33579, -76.62317 39.33593, -76.62325 39.33598, -76.62345 39.33611, -76.62371 39.33628, -76.62402 39.33648, -76.62403 39.33660, -76.62487 39.33807, -76.62509 39.33851, -76.62534 39.33892, -76.62556 39.33926, -76.62574 39.33961, -76.62576 39.34054, -76.62577 39.34146, -76.62578 39.34159, -76.62532 39.34160, -76.62530 39.34160, -76.62510 39.34170, -76.62500 39.34176, -76.62490 39.34182, -76.62484 39.34187, -76.62467 39.34200, -76.62443 39.34187, -76.62431 39.34180, -76.62427 39.34176, -76.62401 39.34147, -76.62388 39.34133, -76.62379 39.34122, -76.62373 39.34112, -76.62368 39.34105, -76.62357 39.34092, -76.62352 39.34085, -76.62349 39.34082, -76.62346 39.34078, -76.62342 39.34075, -76.62337 39.34072, -76.62333 39.34070, -76.62326 39.34068, -76.62316 39.34065, -76.62308 39.34064, -76.62298 39.34064, -76.62292 39.34064, -76.62285 39.34066, -76.62277 39.34069, -76.62267 39.34073, -76.62257 39.34078, -76.62243 39.34085, -76.62222 39.34093, -76.62213 39.34097, -76.62203 39.34100, -76.62190 39.34104, -76.62179 39.34107, -76.62171 39.34108, -76.62161 39.34109, -76.62151 39.34109, -76.62143 39.34109, -76.62138 39.34107, -76.62132 39.34104, -76.62094 39.34080, -76.62042 39.34047, -76.61984 39.34011, -76.61953 39.33992, -76.61937 39.33982, -76.61917 39.33969, -76.61862 39.33938, -76.61840 39.33926, -76.61837 39.33863, -76.61836 39.33853, -76.61810 39.33856, -76.61784 39.33858, -76.61767 39.33861, -76.61756 39.33862, -76.61722 39.33868, -76.61697 39.33874, -76.61675 39.33880, -76.61666 39.33882, -76.61657 39.33884, -76.61638 39.33890, -76.61599 39.33902, -76.61513 39.33933, -76.61494 39.33941, -76.61466 39.33954, -76.61435 39.33969, -76.61398 39.33989, -76.61348 39.34015, -76.61342 39.34019, -76.61327 39.34028, -76.61310 39.34040, -76.61292 39.34051, -76.61269 39.34068, -76.61250 39.34081, -76.61229 39.34096, -76.61214 39.34083, -76.61199 39.34071, -76.61182 39.34058, -76.61163 39.34045, -76.61141 39.34032, -76.61128 39.34026, -76.61098 39.34013, -76.61077 39.34004, -76.61056 39.33998, -76.61038 39.33994, -76.61021 39.33990, -76.61001 39.33986, -76.60979 39.33983, -76.60954 39.33979))                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
..      ...     ...                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 ...                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
195  80800      ...  POLYGON ((-76.58813 39.30322, -76.58809 39.30245, -76.58808 39.30214, -76.58835 39.30213, -76.58892 39.30211, -76.58981 39.30208, -76.59048 39.30206, -76.59114 39.30203, -76.59179 39.30200, -76.59246 39.30197, -76.59350 39.30193, -76.59435 39.30189, -76.59466 39.30188, -76.59644 39.30182, -76.59732 39.30177, -76.59783 39.30175, -76.59784 39.30209, -76.59787 39.30240, -76.59792 39.30311, -76.59800 39.30418, -76.59807 39.30522, -76.59736 39.30526, -76.59664 39.30530, -76.59523 39.30534, -76.59485 39.30536, -76.59456 39.30537, -76.59252 39.30545, -76.59133 39.30555, -76.59068 39.30557, -76.59002 39.30560, -76.58947 39.30561, -76.58828 39.30564, -76.58825 39.30511, -76.58823 39.30485, -76.58822 39.30455, -76.58814 39.30349, -76.58813 39.30322))                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    
196  160500     ...  POLYGON ((-76.65160 39.30335, -76.65075 39.30325, -76.65040 39.30320, -76.65010 39.30315, -76.64984 39.30310, -76.64929 39.30303, -76.64906 39.30299, -76.64865 39.30292, -76.64870 39.30272, -76.64875 39.30267, -76.64879 39.30264, -76.64904 39.30254, -76.64915 39.30247, -76.64927 39.30237, -76.64939 39.30225, -76.64956 39.30203, -76.65017 39.30107, -76.65046 39.30078, -76.65063 39.30057, -76.65077 39.30037, -76.65090 39.30017, -76.65104 39.29977, -76.65134 39.29895, -76.65135 39.29864, -76.65105 39.29865, -76.65060 39.29866, -76.65060 39.29813, -76.65056 39.29751, -76.65057 39.29726, -76.65056 39.29693, -76.65054 39.29659, -76.65052 39.29635, -76.65047 39.29507, -76.65038 39.29372, -76.65086 39.29370, -76.65107 39.29369, -76.65126 39.29368, -76.65180 39.29365, -76.65241 39.29362, -76.65299 39.29359, -76.65322 39.29358, -76.65401 39.29356, -76.65474 39.29354, -76.65608 39.29348, -76.65627 39.29348, -76.65703 39.29344, -76.65780 39.29341, -76.65810 39.29340, -76.65858 39.29338, -76.65935 39.29335, -76.65937 39.29393, -76.65941 39.29463, -76.65943 39.29498, -76.65945 39.29541, -76.65947 39.29576, -76.65948 39.29594, -76.65949 39.29609, -76.65952 39.29647, -76.65953 39.29678, -76.65959 39.29834, -76.65966 39.29870, -76.65968 39.29875, -76.65970 39.29880, -76.65976 39.29891, -76.65985 39.29904, -76.65995 39.29915, -76.66023 39.29942, -76.66047 39.29964, -76.66060 39.29980, -76.66067 39.29990, -76.66073 39.30002, -76.66077 39.30015, -76.66080 39.30027, -76.66082 39.30045, -76.66087 39.30154, -76.66088 39.30177, -76.66087 39.30196, -76.66085 39.30213, -76.66074 39.30251, -76.66072 39.30262, -76.66074 39.30279, -76.66076 39.30287, -76.66080 39.30296, -76.66083 39.30308, -76.66083 39.30327, -76.66082 39.30358, -76.66080 39.30367, -76.66076 39.30376, -76.66078 39.30422, -76.66080 39.30453, -76.66082 39.30482, -76.66090 39.30614, -76.66017 39.30618, -76.65920 39.30624, -76.65904 39.30625, -76.65886 39.30607, -76.65858 39.30583, -76.65850 39.30577, -76.65842 39.30571, -76.65835 39.30566, -76.65793 39.30537, -76.65766 39.30522, -76.65738 39.30510, -76.65702 39.30495, -76.65693 39.30493, -76.65566 39.30450, -76.65379 39.30391, -76.65331 39.30378, -76.65274 39.30362, -76.65160 39.30335))                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
197  90100      ...  POLYGON ((-76.60021 39.34007, -76.60022 39.33982, -76.60020 39.33965, -76.60019 39.33960, -76.60016 39.33942, -76.60013 39.33924, -76.60002 39.33896, -76.59997 39.33887, -76.59990 39.33872, -76.59983 39.33859, -76.59973 39.33840, -76.59964 39.33826, -76.59955 39.33814, -76.59944 39.33797, -76.59934 39.33784, -76.59919 39.33765, -76.59883 39.33725, -76.59874 39.33716, -76.59859 39.33702, -76.59834 39.33682, -76.59795 39.33649, -76.59748 39.33610, -76.59699 39.33570, -76.59643 39.33524, -76.59614 39.33498, -76.59599 39.33483, -76.59585 39.33467, -76.59560 39.33431, -76.59545 39.33405, -76.59579 39.33401, -76.59638 39.33402, -76.59645 39.33402, -76.59695 39.33403, -76.59741 39.33404, -76.59788 39.33404, -76.59835 39.33404, -76.59882 39.33404, -76.59928 39.33403, -76.59978 39.33403, -76.60019 39.33404, -76.60066 39.33404, -76.60108 39.33404, -76.60154 39.33403, -76.60197 39.33403, -76.60284 39.33404, -76.60325 39.33404, -76.60448 39.33404, -76.60463 39.33433, -76.60512 39.33427, -76.60588 39.33419, -76.60631 39.33414, -76.60676 39.33409, -76.60711 39.33407, -76.60714 39.33406, -76.60755 39.33405, -76.60753 39.33448, -76.60951 39.33446, -76.60951 39.33534, -76.60951 39.33640, -76.60952 39.33682, -76.60953 39.33731, -76.60953 39.33867, -76.60955 39.33939, -76.60954 39.33979, -76.60955 39.34024, -76.60770 39.34023, -76.60766 39.33968, -76.60639 39.33971, -76.60641 39.34010, -76.60640 39.34013, -76.60637 39.34017, -76.60632 39.34020, -76.60601 39.34030, -76.60577 39.34038, -76.60559 39.34043, -76.60551 39.34045, -76.60541 39.34046, -76.60528 39.34046, -76.60496 39.34046, -76.60423 39.34048, -76.60368 39.34049, -76.60313 39.34052, -76.60248 39.34055, -76.60177 39.34056, -76.60158 39.34056, -76.60143 39.34055, -76.60130 39.34054, -76.60116 39.34051, -76.60086 39.34042, -76.60076 39.34030, -76.60066 39.34023, -76.60050 39.34017, -76.60037 39.34012, -76.60021 39.34007))                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
198  260402     ...  POLYGON ((-76.54312 39.30735, -76.54333 39.30741, -76.54359 39.30743, -76.54388 39.30748, -76.54406 39.30756, -76.54416 39.30761, -76.54429 39.30772, -76.54440 39.30785, -76.54451 39.30801, -76.54473 39.30827, -76.54518 39.30866, -76.54542 39.30887, -76.54547 39.30892, -76.54556 39.30897, -76.54574 39.30905, -76.54619 39.30922, -76.54680 39.30954, -76.54690 39.30961, -76.54699 39.30968, -76.54712 39.30981, -76.54719 39.30990, -76.54724 39.31000, -76.54735 39.31016, -76.54758 39.31068, -76.54776 39.31101, -76.54785 39.31115, -76.54788 39.31118, -76.54792 39.31120, -76.54799 39.31122, -76.54806 39.31123, -76.54816 39.31124, -76.54826 39.31123, -76.54837 39.31119, -76.54845 39.31119, -76.54853 39.31119, -76.54860 39.31122, -76.54872 39.31126, -76.54884 39.31128, -76.54897 39.31129, -76.54909 39.31130, -76.54951 39.31132, -76.54966 39.31132, -76.54975 39.31131, -76.54980 39.31129, -76.54986 39.31129, -76.54993 39.31130, -76.55009 39.31131, -76.55038 39.31132, -76.55032 39.31222, -76.55026 39.31251, -76.55021 39.31261, -76.55012 39.31273, -76.54997 39.31286, -76.54983 39.31304, -76.54964 39.31326, -76.54949 39.31343, -76.54937 39.31362, -76.54936 39.31365, -76.54935 39.31368, -76.54935 39.31374, -76.54939 39.31388, -76.54941 39.31396, -76.54941 39.31403, -76.54941 39.31410, -76.54940 39.31418, -76.54937 39.31426, -76.54934 39.31435, -76.54932 39.31442, -76.54931 39.31447, -76.54931 39.31455, -76.54930 39.31463, -76.54927 39.31471, -76.54923 39.31477, -76.54919 39.31495, -76.54910 39.31518, -76.54894 39.31589, -76.54813 39.31626, -76.54746 39.31666, -76.54584 39.31762, -76.54574 39.31769, -76.54524 39.31802, -76.54352 39.31917, -76.54235 39.31994, -76.54223 39.32002, -76.54046 39.32122, -76.54032 39.32138, -76.53980 39.32200, -76.53833 39.32377, -76.53687 39.32560, -76.53648 39.32611, -76.53602 39.32671, -76.53410 39.32830, -76.53380 39.32779, -76.53369 39.32763, -76.53366 39.32757, -76.53362 39.32750, -76.53362 39.32743, -76.53364 39.32732, -76.53365 39.32727, -76.53365 39.32721, -76.53366 39.32715, -76.53369 39.32709, -76.53370 39.32703, -76.53368 39.32694, -76.53364 39.32685, -76.53359 39.32676, -76.53356 39.32666, -76.53354 39.32652, -76.53352 39.32641, -76.53351 39.32635, -76.53343 39.32626, -76.53336 39.32616, -76.53328 39.32606, -76.53302 39.32563, -76.53266 39.32503, -76.53255 39.32486, -76.53246 39.32472, -76.53241 39.32457, -76.53239 39.32447, -76.53238 39.32439, -76.53238 39.32427, -76.53239 39.32420, -76.53241 39.32410, -76.53245 39.32404, -76.53255 39.32394, -76.53263 39.32390, -76.53276 39.32390, -76.53288 39.32391, -76.53298 39.32393, -76.53307 39.32393, -76.53319 39.32391, -76.53321 39.32389, -76.53323 39.32382, -76.53322 39.32372, -76.53321 39.32366, -76.53322 39.32362, -76.53332 39.32351, -76.53335 39.32346, -76.53340 39.32341, -76.53346 39.32338, -76.53354 39.32333, -76.53359 39.32327, -76.53359 39.32325, -76.53360 39.32320, -76.53362 39.32313, -76.53366 39.32300, -76.53369 39.32295, -76.53371 39.32290, -76.53373 39.32287, -76.53383 39.32279, -76.53395 39.32269, -76.53410 39.32261, -76.53417 39.32257, -76.53433 39.32241, -76.53441 39.32231, -76.53448 39.32218, -76.53454 39.32206, -76.53463 39.32181, -76.53467 39.32164, -76.53466 39.32146, -76.53464 39.32140, -76.53461 39.32136, -76.53457 39.32133, -76.53452 39.32129, -76.53449 39.32127, -76.53447 39.32123, -76.53446 39.32118, -76.53446 39.32113, -76.53447 39.32099, -76.53452 39.32074, -76.53454 39.32059, -76.53457 39.32052, -76.53463 39.32046, -76.53468 39.32042, -76.53471 39.32039, -76.53474 39.32017, -76.53478 39.32002, -76.53482 39.31996, -76.53488 39.31991, -76.53496 39.31987, -76.53551 39.31959, -76.53573 39.31946, -76.53578 39.31940, -76.53582 39.31935, -76.53584 39.31932, -76.53586 39.31929, -76.53587 39.31926, -76.53587 39.31923, -76.53587 39.31920, -76.53587 39.31917, -76.53586 39.31914, -76.53584 39.31912, -76.53582 39.31909, -76.53579 39.31907, -76.53576 39.31905, -76.53572 39.31903, -76.53569 39.31901, -76.53567 39.31898, -76.53564 39.31893, -76.53563 39.31887, -76.53562 39.31882, -76.53562 39.31876, -76.53564 39.31857, -76.53570 39.31829, -76.53571 39.31818, -76.53571 39.31818, -76.53570 39.31808, -76.53565 39.31800, -76.53545 39.31782, -76.53522 39.31762, -76.53584 39.31698, -76.53677 39.31608, -76.53781 39.31506, -76.53870 39.31421, -76.54107 39.31089, -76.54123 39.31059, -76.54126 39.31055, -76.54136 39.31039, -76.54175 39.30967, -76.54224 39.30878, -76.54280 39.30783, -76.54312 39.30735))
199  260700     ...  POLYGON ((-76.55464 39.28229, -76.55480 39.28212, -76.55490 39.28200, -76.55501 39.28186, -76.55509 39.28174, -76.55517 39.28160, -76.55524 39.28145, -76.55528 39.28134, -76.55535 39.28096, -76.55542 39.28095, -76.55638 39.28091, -76.55732 39.28088, -76.55739 39.28217, -76.55831 39.28162, -76.55826 39.28095, -76.55824 39.28083, -76.55926 39.28079, -76.56099 39.28074, -76.56102 39.28124, -76.56107 39.28177, -76.56181 39.28185, -76.56209 39.28143, -76.56212 39.28175, -76.56215 39.28233, -76.56218 39.28274, -76.56225 39.28378, -76.56230 39.28464, -76.56231 39.28479, -76.56239 39.28581, -76.56241 39.28632, -76.56244 39.28688, -76.56247 39.28731, -76.56251 39.28783, -76.56254 39.28835, -76.56258 39.28882, -76.56266 39.29046, -76.56274 39.29160, -76.56232 39.29161, -76.56167 39.29164, -76.56087 39.29167, -76.55989 39.29171, -76.55891 39.29174, -76.55741 39.29180, -76.55705 39.29182, -76.55616 39.29187, -76.55578 39.29188, -76.55548 39.29190, -76.55499 39.29191, -76.55558 39.29182, -76.55559 39.29181, -76.55578 39.29178, -76.55571 39.29145, -76.55568 39.29111, -76.55566 39.29096, -76.55563 39.29082, -76.55553 39.29062, -76.55549 39.29054, -76.55543 39.29043, -76.55434 39.28910, -76.55492 39.28758, -76.55498 39.28710, -76.55492 39.28608, -76.55486 39.28506, -76.55485 39.28492, -76.55480 39.28405, -76.55474 39.28304, -76.55464 39.28229))                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            

[200 rows x 6 columns], 
 porg: False, 
 geom: geometry, 
 lat: False, 
 lng: False, 
 revgeocode: False, 
 in_crs: 2248, 
 out_crs: 2248
Index(['TRACTCE10', 'GEOID10', 'NAME10', 'CSA', 'Tract', 'geometry'], dtype='object')
(geopandas.geodataframe.GeoDataFrame, <Projected CRS: EPSG:2248>
 Name: NAD83 / Maryland (ftUS)
 Axis Info [cartesian]:
 - X[east]: Easting (US survey foot)
 - Y[north]: Northing (US survey foot)
 Area of Use:
 - name: USA - Maryland
 - bounds: (-79.49, 37.97, -74.97, 39.73)
 Coordinate Operation:
 - name: SPCS83 Maryland zone (US Survey feet)
 - method: Lambert Conic Conformal (2SP)
 Datum: North American Datum 1983
 - Ellipsoid: GRS 1980
 - Prime Meridian: Greenwich, Index(['TRACTCE10', 'GEOID10', 'NAME10', 'CSA', 'Tract', 'geometry'], dtype='object'))
<matplotlib.axes._subplots.AxesSubplot at 0x7f43a589c860>
{% endraw %}

Example Points 1: Loading Tax data from addresses. Get sent to coordinates.

{% raw %}
url = 'https://docs.google.com/spreadsheets/d/e/2PACX-1vTMgdqWykZeIsMwCFllPuG1cd4gGDB6BUqaAOM0Lx9VGdCo2JJy9v_CR8ZaEDWO3Q/pub?gid=290715815&single=true&output=csv'
# points = readInGeometryData(url=url, revgeocode='y', lat='STREET_')
# workWithGeometryData('summary', points) 
{% endraw %}

Example Points 2: Loading Food Pantries as Points

{% raw %}
url = 'https://docs.google.com/spreadsheets/d/e/2PACX-1vT3lG0n542sIGE2O-C8fiXx-qUZG2WDO6ezRGcNsS4z8MM30XocVZ90P1UQOIXO2w/pub?gid=1152681223&single=true&output=csv'
crs = {'init' :'epsg:2248'} 
foodPantryLocations = readInGeometryData(url=url, porg='p', geom=False, lat='Y', lng='X', revgeocode=False,  save=False, in_crs=crs, out_crs=crs)
foodPantryLocations.plot()
RECIEVED url: https://docs.google.com/spreadsheets/d/e/2PACX-1vT3lG0n542sIGE2O-C8fiXx-qUZG2WDO6ezRGcNsS4z8MM30XocVZ90P1UQOIXO2w/pub?gid=1152681223&single=true&output=csv, 
 porg: p, 
 geom: False, 
 lat: Y, 
 lng: X, 
 revgeocode: False, 
 in_crs: {'init': 'epsg:2248'}, 
 out_crs: {'init': 'epsg:2248'}
Index(['X', 'Y', 'OBJECTID', 'Name', 'Address', 'City_1', 'State', 'Zip',
       '# in Zip', 'FIPS'],
      dtype='object')
/usr/local/lib/python3.6/dist-packages/pyproj/crs/crs.py:53: FutureWarning: '+init=<authority>:<code>' syntax is deprecated. '<authority>:<code>' is the preferred initialization method. When making the change, be mindful of axis order changes: https://pyproj4.github.io/pyproj/stable/gotchas.html#axis-order-changes-in-proj-6
  return _prepare_from_string(" ".join(pjargs))
<matplotlib.axes._subplots.AxesSubplot at 0x7fdb30800da0>
{% endraw %} {% raw %}
foodPantryLocations.head()
X Y OBJECTID Name Address City_1 State Zip # in Zip FIPS geometry
0 -77.06 39.02 1 Victory Forest -... 10000 Brunswick ... Silver Spring MD 20817 NaN NaN POINT (-77.05673...
1 -76.99 38.82 2 Glassmanor Eleme... 1011 Marcy Avenue Oxon Hill MD 20745 NaN NaN POINT (-76.99036...
2 -76.99 39.02 3 Apple Blossoms 1013 Cresthaven ... Silver Spring MD 20903 NaN NaN POINT (-76.99155...
3 -77.15 39.02 4 Lakeview Apartme... 10250 West Lake Dr. Bethesda MD 20817 NaN NaN POINT (-77.14929...
4 -76.89 38.89 5 Central Gardens 103 Cindy Lane Capitol Heights MD NaN NaN POINT (-76.88974...
{% endraw %}

Lets see how our map looks when we have points atop polygons

{% raw %}
panp = workWithGeometryData( 'pandp', foodPantryLocations[ foodPantryLocations.City_1 == 'Baltimore' ], csa_gdf, pntsClr='red', polysClr='white')
mapPointsandPolygons
{% endraw %}

Looking good! But the red dots are a bit too noisy. Lets create a choropleth instead!

We can start of by finding which points are inside of which polygons!

A choropleth map will be created at the bottom of the output once the the code below this cell is exectued for our Food Pantries Data.

{% raw %}
# https://stackoverflow.com/questions/27606924/count-number-of-points-in-multipolygon-shapefile-using-python
ptsCoordCol = 'geometry'
polygonsCoordCol = 'geometry'

pointsInPolys = workWithGeometryData('pinp', foodPantryLocations[ foodPantryLocations.City_1 == 'Baltimore' ], csa_gdf, 'geometry' , 'geometry')
pointsInPolys.plot(column='number of points', legend=True)
Total Points:  220.0
Total Points in Polygons:  199
Prcnt Points in Polygons:  0.9045454545454545
/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:74: FutureWarning:     You are passing non-geometry data to the GeoSeries constructor. Currently,
    it falls back to returning a pandas Series. But in the future, we will start
    to raise a TypeError instead.
<matplotlib.axes._subplots.AxesSubplot at 0x7fdb30c85438>
{% endraw %} {% raw %}
pointsInPolys.head()
OBJECTID CSA2010 tpop10 male10 female10 paa17 pwhite17 pasi17 p2more17 ppac17 phisp17 racdiv17 age5_17 age18_17 age24_17 age64_17 age65_17 hhs10 femhhs17 fam17 hhsize10 mhhi17 hh25inc17 hh40inc17 hh60inc17 hh75inc17 hhm7517 hhpov17 hhchpov17 Shape__Are Shape__Len geometry number of points pointsinpolygon
0 1 Allendale/Irving... 16726 7657 9069 90.28 6.53 0.11 1.00 0.00 2.06 20.12 6.61 17.37 9.00 53.29 13.72 6098 71.13 35.20 2.64 39495.63 32.99 17.72 19.91 11.95 17.43 20.70 32.77 6.38e+07 38770.17 POLYGON ((-76.65... 7 7
1 2 Beechfield/Ten H... 13391 5985 7406 75.32 18.86 0.42 3.31 0.31 1.78 41.02 7.93 14.58 9.71 55.37 12.42 5076 55.19 26.14 2.40 57572.50 20.42 13.90 18.18 10.87 36.64 10.47 23.92 4.79e+07 37524.95 POLYGON ((-76.69... 2 2
2 3 Belair-Edison 17380 7297 10083 85.65 10.03 0.57 1.70 0.81 1.24 27.26 5.42 22.81 8.41 54.45 8.91 6174 77.53 38.27 2.90 39624.48 34.10 16.28 20.07 8.11 21.45 20.27 34.56 4.50e+07 31307.31 POLYGON ((-76.56... 0 0
3 4 Brooklyn/Curtis ... 12900 5746 7154 37.96 39.68 2.53 3.61 1.31 14.91 73.93 10.91 16.09 8.25 57.45 7.30 5204 43.39 32.32 2.61 40275.28 31.40 18.32 18.52 9.08 22.68 24.21 46.41 1.76e+08 150987.70 MULTIPOLYGON (((... 4 4
4 5 Canton 8326 4094 4232 3.94 85.58 4.38 2.45 0.56 3.09 26.31 5.25 3.12 5.85 75.25 10.53 4310 10.55 11.03 1.86 111891.25 7.41 7.82 9.18 6.43 69.16 3.66 4.02 1.54e+07 23338.61 POLYGON ((-76.57... 1 1
{% endraw %} {% raw %}
# And now that we have that settled, lets map it!
panp = workWithGeometryData( 'pandp', foodPantryLocations[ foodPantryLocations.City_1 == 'Baltimore' ], pointsInPolys, polyColorCol='number of points')
mapPointsandPolygons
{% endraw %}

In that last example, we got a count of points in the polygon dataset.

If we wanted the Polygon each Point is on, you'd do it like this!

{% raw %}
panp = getPolygonOnPoints(foodPantryLocations[ foodPantryLocations.City_1 == 'Baltimore' ], csa_gdf, 'geometry', 'geometry', 'CSA2010')
panp.head()
panp.plot()
Total Points:  220.0
Total Points in Polygons:  199
Prcnt Points in Polygons:  0.9045454545454545
X Y OBJECTID Name Address City_1 State Zip # in Zip FIPS geometry CSA2010
310 -76.61 39.29 317 2-1-1 MD@UWCM 100 S. Charles S... Baltimore MD 21201 5.0 40100.0 POINT (-76.61499... Downtown/Seton Hill
313 -76.62 39.31 502 University of Ba... 21 W. Mount Roya... Baltimore MD 21201 NaN 110200.0 POINT (-76.61716... Midtown
315 -76.68 39.28 626 Project PLASE Em... 3601 Old Frederi... Baltimore MD 21201 NaN 200800.0 POINT (-76.67662... Allendale/Irving...
316 -76.63 39.30 699 Historic Samuel ... 507 W Preston St... Baltimore MD 21201 NaN 170200.0 POINT (-76.62645... Upton/Druid Heights
321 -76.62 39.30 715 Westminster Hous... 524 North Charle... Baltimore MD 21201 NaN 110200.0 POINT (-76.61560... Midtown
<matplotlib.axes._subplots.AxesSubplot at 0x7fdb3073bf98>
{% endraw %}

Alternately, we could map the centroid of boundaries within another boundary to find boundaries within boundaries

{% raw %}
map_points(food_df, lat_col='Y', lon_col='X', zoom_start=11, plot_points=True, pt_radius=15, draw_heatmap=True, heat_map_weights_col=None, heat_map_weights_normalize=True, heat_map_radius=15)
/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:51: FutureWarning: Method `add_children` is deprecated. Please use `add_child` instead.
Make this Notebook Trusted to load map: File -> Trust Notebook
{% endraw %}