About

Parsons, named after Lucy Parsons, is a Python package that contains a growing list of connectors and integrations to move data between various tools. Parsons is focused on integrations and connectors for tools utilized by the progressive community.

Parsons was built out of a belief that progressive organizations spend far too much time building the same integrations, over and over and over again, while they should be engaged in more important and impactful work. It was built and is mantained by The Movement Cooperative.

The Movement Cooperative

The Movement Cooperative is a member led organization focused on providing data, tools and strategic support for the progressive community. Our mission is to break down technological barriers for organizations that fight for social justice.

License and Usage

Usage of Parsons is governed by the TMC Parsons License, which allows for unlimited non-commercial usage, provided that individuals and organizations adhere to our broad values statement.

Design Goals

The goal of Parsons is to make the movement of data between systems as easy and straightforward as possible. Simply put, we seek to reduce the lines of code that are written by the progressive community. Not only is this a waste of time, but we rarely have the capacity and resources to fully unittest our scripts.

_images/parsons_diagram.png

Parsons seeks to be flexible from a data ingestion and output perspective, while providing ETL tools that recognize that our data is always messy. Central to this concept is the Parsons Table the table-like object that most methods return.

QuickStart

# VAN - Download activist codes to a CSV

from parsons import VAN
van = VAN(db='MyVoters')
ac = van.get_activist_codes()
ac.to_csv('my_activist_codes.csv')

# Redshift - Create a table from a CSV

from parsons import Table
tbl = Table.from_csv('my_table.csv')
tbl.to_redshift('my_schema.my_table')

# Redshift - Export from a query to CSV

from parsons import Redshift
sql = 'select * from my_schema.my_table'
rs = Redshift()
tbl = rs.query(sql)
tbl.to_csv('my_table.csv')

# Upload a file to S3

from parsons import S3
s3 = S3()
s3.put_file('my_bucket','my_table.csv')

# TargetSmart - Append data to a record

from parsons import TargetSmart
ts = TargetSmart(api_key='MY_KEY')
record = ts.data_enhance(231231231, state='DC')

Sources

Installation

You can install the most recent release by running: pip install parsons

Logging

Parsons uses the native python logging system. By default, log output will go to the console and look like:

parsons.modulename LOGLEVEL the specific log message

In your scripts that use Parsons, if you want to override the default Parsons logging behavior, just grab the “parsons” logger and tweak it:

import logging
parsons_logger = logging.getLogger('parsons')
# parsons_logger.setLevel('DEBUG')
# parsons_logger.addHandler(...)
# parsons_logger.setFormatter(...)

Indices and tables