Skip to content

Commit

Permalink
Update package docs
Browse files Browse the repository at this point in the history
  • Loading branch information
rosericazondekon committed May 1, 2023
1 parent 172c4a6 commit fd59b31
Show file tree
Hide file tree
Showing 4 changed files with 148 additions and 59 deletions.
100 changes: 43 additions & 57 deletions docs/articles/intro.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Get started
# Getting started

## Introduction

Expand All @@ -18,23 +18,21 @@ In this vignette, we explained how to create an NSSP user profile, and provide v

## Creating an NSSP user profile

We start by loading the `pynssp` package
We start by loading the `pandas` and the `pynssp` packages.

```python
# Loading useful R packages...
import pandas as pd
from pynssp.utils import *
>>> import pandas as pd
>>> from pynssp.utils import *
```

The next step is to create an NSSP user profile by creating an object of the class `Credentials`.
Here, we use the `create_profile()` function to create a user profile

```python
# Creating an ESSENCE user profile
myProfile = create_profile()
>>> myProfile = create_profile()

# Save profile object to file for future use
myProfile.pickle()
>>> myProfile.pickle()
```

The above code needs to be executed only once. Upon execution, it prompts the user to provide his username and password.
Expand All @@ -48,57 +46,53 @@ In the following sections, we show how to pull data from ESSENCE using the seven
## Time Series Data Table

```python
# URL from ESSENCE JSON Time Series Data Table API
url = "https://essence.syndromicsurveillance.org/nssp_essence/api/timeSeries?endDate=9Feb2021&medicalGrouping=injury&percentParam=noPercent&geographySystem=hospitaldhhsregion&datasource=va_hospdreg&detector=probrepswitch&startDate=11Nov2020&timeResolution=daily&medicalGroupingSystem=essencesyndromes&userId=455&aqtTarget=TimeSeries"
>>> url = "https://essence.syndromicsurveillance.org/nssp_essence/api/timeSeries?endDate=9Feb2021&medicalGrouping=injury&percentParam=noPercent&geographySystem=hospitaldhhsregion&datasource=va_hospdreg&detector=probrepswitch&startDate=11Nov2020&timeResolution=daily&medicalGroupingSystem=essencesyndromes&userId=455&aqtTarget=TimeSeries"

# Pull time series data
api_data_ts = get_api_data(url, profile=myProfile) # or api_data_ts = myProfile.get_api_data(url)
>>> api_data_ts = get_api_data(url, profile=myProfile) # or api_data_ts = myProfile.get_api_data(url)

api_data_ts.columns
>>> api_data_ts.columns

# Extracting embedded dataframe
api_data_ts = pd.json_normalize(api_data_ts["timeSeriesData"][0])
>>> api_data_ts = pd.json_normalize(api_data_ts["timeSeriesData"][0])

# Preview data
api_data_ts.head()
>>> api_data_ts.head()
```

Alternatively, the example below with the `get_essence_data()` function achieves the same outcome directly extracting the embedded dataframe when needed

```python
# Pull time series data
api_data_ts = get_essence_data(url, profile=myProfile)
>>> api_data_ts = get_essence_data(url, profile=myProfile)

# Preview data
api_data_ts.head()
>>> api_data_ts.head()
```

## Time Series Graph from ESSENCE

The example below shows how to retrieve the Time Series Graph from ESSENCE and insert it in a Jupyter notebook.

```python
# URL from ESSENCE Time Series Graph from ESSENCE API
url = "https://essence.syndromicsurveillance.org/nssp_essence/api/timeSeries/graph?endDate=9Feb2021&medicalGrouping=injury&percentParam=noPercent&geographySystem=hospitaldhhsregion&datasource=va_hospdreg&detector=probrepswitch&startDate=11Nov2020&timeResolution=daily&medicalGroupingSystem=essencesyndromes&userId=455&aqtTarget=TimeSeries&graphTitle=National%20-%20Injury%20Syndrome%20Daily%20Counts&xAxisLabel=Date&yAxisLabel=Count"
>>> url = "https://essence.syndromicsurveillance.org/nssp_essence/api/timeSeries/graph?endDate=9Feb2021&medicalGrouping=injury&percentParam=noPercent&geographySystem=hospitaldhhsregion&datasource=va_hospdreg&detector=probrepswitch&startDate=11Nov2020&timeResolution=daily&medicalGroupingSystem=essencesyndromes&userId=455&aqtTarget=TimeSeries&graphTitle=National%20-%20Injury%20Syndrome%20Daily%20Counts&xAxisLabel=Date&yAxisLabel=Count"

# Data pull from ESSENCE
api_data_graph = get_api_graph(url, profile=myProfile)
>>> api_data_graph = get_api_graph(url, profile=myProfile)

# Check the type of api_data_graph
type(api_data_graph)
>>> type(api_data_graph)

# Print image file location
print(api_data_graph)
>>> print(api_data_graph)

# Insert it into a Jupyter notebook
api_data_graph.plot()
>>> api_data_graph.plot()
```

From the example above, the variable `api_data_graph` is an `APIGraph` object. In an interactive mode from the Python console, the `.show()` method can be called on the `APIGraph` object to preview the image it contains.

```python
# Preview API graph
api_data_graph.show()
>>> api_data_graph.show()
```

For an exaustive list of the methods that may be called on an `APIGraph` object, please check the `pynssp` documentation.
Expand All @@ -110,29 +104,27 @@ The CSV option of the Table Builder Results API pulls in data in the tabular for
### CSV option

```python
# URL from CSV ESSENCE Table Builder results API
url = "https://essence.syndromicsurveillance.org/nssp_essence/api/tableBuilder/csv?endDate=31Dec2020&ccddCategory=cdc%20opioid%20overdose%20v3&percentParam=noPercent&geographySystem=hospitaldhhsregion&datasource=va_hospdreg&detector=nodetectordetector&startDate=1Oct2020&ageNCHS=11-14&ageNCHS=15-24&ageNCHS=25-34&ageNCHS=35-44&ageNCHS=45-54&ageNCHS=55-64&ageNCHS=65-74&ageNCHS=75-84&ageNCHS=85-1000&ageNCHS=unknown&timeResolution=monthly&hasBeenE=1&medicalGroupingSystem=essencesyndromes&userId=455&aqtTarget=TableBuilder&rowFields=timeResolution&rowFields=geographyhospitaldhhsregion&columnField=ageNCHS"
>>> url = "https://essence.syndromicsurveillance.org/nssp_essence/api/tableBuilder/csv?endDate=31Dec2020&ccddCategory=cdc%20opioid%20overdose%20v3&percentParam=noPercent&geographySystem=hospitaldhhsregion&datasource=va_hospdreg&detector=nodetectordetector&startDate=1Oct2020&ageNCHS=11-14&ageNCHS=15-24&ageNCHS=25-34&ageNCHS=35-44&ageNCHS=45-54&ageNCHS=55-64&ageNCHS=65-74&ageNCHS=75-84&ageNCHS=85-1000&ageNCHS=unknown&timeResolution=monthly&hasBeenE=1&medicalGroupingSystem=essencesyndromes&userId=455&aqtTarget=TableBuilder&rowFields=timeResolution&rowFields=geographyhospitaldhhsregion&columnField=ageNCHS"

# Data Pull from ESSENCE
api_data_tb_csv = get_api_data(url, fromCSV=True, profile=myProfile)
>>> api_data_tb_csv = get_api_data(url, fromCSV=True, profile=myProfile)

# Preview data
api_data_tb_csv.head()
>>> api_data_tb_csv.head()
```

### JSON option

While the `get_api_data()` function can equally be used to pull from ESSENCE API URL retruning JSON objects, the `get_essence_data()` function has the advantage of not requiring extra JSON parsing as described in the second section of this tutorial.

```python
# URL from JSON ESSENCE Table Builder results API
url = "https://essence2.syndromicsurveillance.org/nssp_essence/api/tableBuilder?endDate=31Dec2020&ccddCategory=cdc%20opioid%20overdose%20v3&percentParam=noPercent&geographySystem=hospitaldhhsregion&datasource=va_hospdreg&detector=nodetectordetector&startDate=1Oct2020&ageNCHS=11-14&ageNCHS=15-24&ageNCHS=25-34&ageNCHS=35-44&ageNCHS=45-54&ageNCHS=55-64&ageNCHS=65-74&ageNCHS=75-84&ageNCHS=85-1000&ageNCHS=unknown&timeResolution=monthly&hasBeenE=1&medicalGroupingSystem=essencesyndromes&userId=2362&aqtTarget=TableBuilder&rowFields=timeResolution&rowFields=geographyhospitaldhhsregion&columnField=ageNCHS"
>>> url = "https://essence2.syndromicsurveillance.org/nssp_essence/api/tableBuilder?endDate=31Dec2020&ccddCategory=cdc%20opioid%20overdose%20v3&percentParam=noPercent&geographySystem=hospitaldhhsregion&datasource=va_hospdreg&detector=nodetectordetector&startDate=1Oct2020&ageNCHS=11-14&ageNCHS=15-24&ageNCHS=25-34&ageNCHS=35-44&ageNCHS=45-54&ageNCHS=55-64&ageNCHS=65-74&ageNCHS=75-84&ageNCHS=85-1000&ageNCHS=unknown&timeResolution=monthly&hasBeenE=1&medicalGroupingSystem=essencesyndromes&userId=2362&aqtTarget=TableBuilder&rowFields=timeResolution&rowFields=geographyhospitaldhhsregion&columnField=ageNCHS"

# Data Pull from ESSENCE
api_data_tb_json = get_essence_data(url, profile=myProfile)
>>> api_data_tb_json = get_essence_data(url, profile=myProfile)

# Preview data
api_data_tb_json.head()
>>> api_data_tb_json.head()
```
## Data Details (line level)

Expand All @@ -141,40 +133,37 @@ Similarly to the Table builder Results API, the Data Details (line level) provid
### CSV option

```python
# URL from CSV ESSENCE Data Details (line level) API
url = "https://essence.syndromicsurveillance.org/nssp_essence/api/dataDetails/csv?medicalGrouping=injury&geography=region%20i&percentParam=noPercent&geographySystem=hospitaldhhsregion&datasource=va_hospdreg&detector=probrepswitch&timeResolution=daily&medicalGroupingSystem=essencesyndromes&userId=455&aqtTarget=TimeSeries&startDate=31Jan2021&endDate=31Jan2021"
>>> url = "https://essence.syndromicsurveillance.org/nssp_essence/api/dataDetails/csv?medicalGrouping=injury&geography=region%20i&percentParam=noPercent&geographySystem=hospitaldhhsregion&datasource=va_hospdreg&detector=probrepswitch&timeResolution=daily&medicalGroupingSystem=essencesyndromes&userId=455&aqtTarget=TimeSeries&startDate=31Jan2021&endDate=31Jan2021"

# Data Pull from ESSENCE
api_data_dd_csv = get_api_data(url, fromCSV=True, profile=myProfile)
>>> api_data_dd_csv = get_api_data(url, fromCSV=True, profile=myProfile)

# Preview data
api_data_dd_csv.head()
>>> api_data_dd_csv.head()
```

### JSON option

```python
# URL from JSON ESSENCE Data Details (line level) API
url = "https://essence.syndromicsurveillance.org/nssp_essence/api/dataDetails?endDate=31Jan2021&medicalGrouping=injury&percentParam=noPercent&geographySystem=hospitaldhhsregion&datasource=va_hospdreg&detector=probrepswitch&startDate=31Jan2021&timeResolution=daily&medicalGroupingSystem=essencesyndromes&userId=455&aqtTarget=DataDetails"
>>> url = "https://essence.syndromicsurveillance.org/nssp_essence/api/dataDetails?endDate=31Jan2021&medicalGrouping=injury&percentParam=noPercent&geographySystem=hospitaldhhsregion&datasource=va_hospdreg&detector=probrepswitch&startDate=31Jan2021&timeResolution=daily&medicalGroupingSystem=essencesyndromes&userId=455&aqtTarget=DataDetails"

# Data Pull from ESSENCE
api_data_dd_json = get_essence_data(url, profile=myProfile)
>>> api_data_dd_json = get_essence_data(url, profile=myProfile)

# Preview data
api_data_dd_json.head()
>>> api_data_dd_json.head()
```

## Summary Stats

```python
# URL from JSON ESSENCE Summary Stats API
url = "https://essence.syndromicsurveillance.org/nssp_essence/api/summaryData?endDate=31Jan2021&medicalGrouping=injury&geography=region%20i&percentParam=noPercent&geographySystem=hospitaldhhsregion&datasource=va_hosp&detector=probrepswitch&startDate=29Jan2021&timeResolution=daily&medicalGroupingSystem=essencesyndromes&userId=455&aqtTarget=TimeSeries"
>>> url = "https://essence.syndromicsurveillance.org/nssp_essence/api/summaryData?endDate=31Jan2021&medicalGrouping=injury&geography=region%20i&percentParam=noPercent&geographySystem=hospitaldhhsregion&datasource=va_hosp&detector=probrepswitch&startDate=29Jan2021&timeResolution=daily&medicalGroupingSystem=essencesyndromes&userId=455&aqtTarget=TimeSeries"

# Data Pull from ESSENCE
api_data_ss = get_essence_data(url, profile=myProfile)
>>> api_data_ss = get_essence_data(url, profile=myProfile)

# Preview data
api_data_ss.head()
>>> api_data_ss.head()
```

## Alert List Detection Table
Expand All @@ -184,40 +173,37 @@ Since the Alert List API provides programmatic access to the Alert List table on
### Alert List Detection Table by Patient Region

```python
# URL from JSON ESSENCE Alert List Detection Table API
url = "https://essence.syndromicsurveillance.org/nssp_essence/api/alerts/regionSyndromeAlerts?end_date=31Jan2021&start_date=29Jan2021"
>>> url = "https://essence.syndromicsurveillance.org/nssp_essence/api/alerts/regionSyndromeAlerts?end_date=31Jan2021&start_date=29Jan2021"

# Data Pull from ESSENCE
api_data_alr = get_essence_data(url, profile=myProfile)
>>> api_data_alr = get_essence_data(url, profile=myProfile)

# Preview data
api_data_alr.head()
>>> api_data_alr.head()
```

### Alert List Detection Table by Hospital Region

```python
# URL from JSON ESSENCE Alert List Detection Table API
url = "https://essence.syndromicsurveillance.org/nssp_essence/api/alerts/hospitalSyndromeAlerts?end_date=31Jan2021&start_date=29Jan2021"
>>> url = "https://essence.syndromicsurveillance.org/nssp_essence/api/alerts/hospitalSyndromeAlerts?end_date=31Jan2021&start_date=29Jan2021"

# Data Pull from ESSENCE
api_data_alh = get_api_data(url, profile=myProfile)
>>> api_data_alh = get_api_data(url, profile=myProfile)

# Preview data
api_data_alh.head()
>>> api_data_alh.head()
```

## Time series data table with stratified, historical alerts (from ESSENCE2)

This functionality as of February 10, 2023 is available from ESSENCE2. Therefore, if your ESSENCE 2 credentials are different from the one you define for ESSENCE above, you will have to recreate another profile object for ESSENCE 2 and use it to run the code below. In this example, it is assumed that the same user profile has been used for both ESSENCE and ESSENCE 2.

```python
# URL from JSON ESSENCE Time series data table with stratified, historical alerts API
url = "https://essence2.syndromicsurveillance.org/nssp_essence/api/timeSeries?endDate=9Feb2021&ccddCategory=cdc%20pneumonia%20ccdd%20v1&ccddCategory=cdc%20coronavirus-dd%20v1&ccddCategory=cli%20cc%20with%20cli%20dd%20and%20coronavirus%20dd%20v2&percentParam=ccddCategory&geographySystem=hospitaldhhsregion&datasource=va_hospdreg&detector=probrepswitch&startDate=11Nov2020&timeResolution=daily&hasBeenE=1&medicalGroupingSystem=essencesyndromes&userId=2362&aqtTarget=TimeSeries&stratVal=ccddCategory&multiStratVal=geography&graphOnly=true&numSeries=3&graphOptions=multipleSmall&seriesPerYear=false&nonZeroComposite=false&removeZeroSeries=true&startMonth=January&stratVal=ccddCategory&multiStratVal=geography&graphOnly=true&numSeries=3&graphOptions=multipleSmall&seriesPerYear=false&startMonth=January&nonZeroComposite=false"
>>> url = "https://essence2.syndromicsurveillance.org/nssp_essence/api/timeSeries?endDate=9Feb2021&ccddCategory=cdc%20pneumonia%20ccdd%20v1&ccddCategory=cdc%20coronavirus-dd%20v1&ccddCategory=cli%20cc%20with%20cli%20dd%20and%20coronavirus%20dd%20v2&percentParam=ccddCategory&geographySystem=hospitaldhhsregion&datasource=va_hospdreg&detector=probrepswitch&startDate=11Nov2020&timeResolution=daily&hasBeenE=1&medicalGroupingSystem=essencesyndromes&userId=2362&aqtTarget=TimeSeries&stratVal=ccddCategory&multiStratVal=geography&graphOnly=true&numSeries=3&graphOptions=multipleSmall&seriesPerYear=false&nonZeroComposite=false&removeZeroSeries=true&startMonth=January&stratVal=ccddCategory&multiStratVal=geography&graphOnly=true&numSeries=3&graphOptions=multipleSmall&seriesPerYear=false&startMonth=January&nonZeroComposite=false"

# Data Pull from ESSENCE
api_data_tssh = get_essence_data(url, profile=myProfile)
>>> api_data_tssh = get_essence_data(url, profile=myProfile)

# Preview data
api_data_tssh.head()
>>> api_data_tssh.head()
```
2 changes: 1 addition & 1 deletion docs/installation.rst
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
.. highlight:: shell

============
Installation
Installing pynssp
============


Expand Down
104 changes: 104 additions & 0 deletions docs/readme.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
# About pynssp
[![pynssp Pipeline](https://github.com/cdcent/pynssp/actions/workflows/check-standard.yaml/badge.svg)](https://github.com/cdcent/pynssp/actions/workflows/check-standard.yaml)

## Overview
`pynssp` is a Python package for the National Syndromic Surveillance Program (NSSP) and its Community of Practice. A collection of classes and methods to advance the practice of Syndromic Surveillance. It serves as a Python alternative to the [Rnssp R package](https://cdcgov.github.io/Rnssp).

## Installation

You can install the development version of `pynssp` from Github:

```console
$ pip install git+https://github.com/CDCgov/pynssp.git
```

## Usage
```python
>>> import pandas as pd
>>> from pynssp.utils import *
>>> from datetime import date, timedelta


## Creating a user profile (token)
>>> myProfile = create_token_profile()

## Creating a user profile (username and password)
>>> myProfile = create_profile()

## JSON URL from NSSP-ESSENCE API
>>> url = "https://essence.syndromicsurveillance.org/nssp_essence/api/alerts/regionSyndromeAlerts?end_date=31Jan2021&start_date=29Jan2021"

## Update Start and End dates in NSSP-ESSENCE API URL
>>> startDate = date.today() - timedelta(days=30)
>>> endDate = date.today()

>>> url = change_dates(url, start_date = startDate, end_date = endDate)

## Pull Time Series Data from NSSP-ESSENCE
>>> api_data = get_api_data(url, profile=myProfile)

## Inspect data object structure
>>> api_data.columns

## Extract table of interest
>>> api_data = pd.json_normalize(api_data["regionSyndromeAlerts"][0])

## Get a glimpse of the pulled dataset
>>> api_data.head()
```

## Contributing to this project
Should you want to contribute to this project, submit a push request to this Github repository and consider submitting a request to be added as a developer to [email protected].

## Getting Help
If you encounter a clear bug, please consider emailing the author at [email protected] and/or file an issue with a minimal reproducible example.

## Public Domain Standard Notice
This repository constitutes a work of the United States Government and is not
subject to domestic copyright protection under 17 USC ? 105. This repository is in
the public domain within the United States, and copyright and related rights in
the work worldwide are waived through the [CC0 1.0 Universal public domain dedication](https://creativecommons.org/publicdomain/zero/1.0/).
All contributions to this repository will be released under the CC0 dedication. By
submitting a pull request you are agreeing to comply with this waiver of
copyright interest.

## License Standard Notice
The repository utilizes code licensed under the terms of the Apache Software
License and therefore is licensed under ASL v2 or later.

This source code in this repository is free: you can redistribute it and/or modify it under
the terms of the Apache Software License version 2, or (at your option) any
later version.

This source code in this repository is distributed in the hope that it will be useful, but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE. See the Apache Software License for more details.

You should have received a copy of the Apache Software License along with this
program. If not, see http://www.apache.org/licenses/LICENSE-2.0.html

The source code forked from other open source projects will inherit its license.

## Privacy Standard Notice
This repository contains only non-sensitive, publicly available data and
information. All material and community participation is covered by the
[Disclaimer](https://github.com/CDCgov/template/blob/master/DISCLAIMER.md)
and [Code of Conduct](https://github.com/CDCgov/template/blob/master/code-of-conduct.md).
For more information about CDC's privacy policy, please visit [http://www.cdc.gov/other/privacy.html](https://www.cdc.gov/other/privacy.html).

## Contributing Standard Notice
Anyone is encouraged to contribute to the repository by [forking](https://help.github.com/articles/fork-a-repo)
and submitting a pull request. (If you are new to GitHub, you might start with a
[basic tutorial](https://help.github.com/articles/set-up-git).) By contributing
to this project, you grant a world-wide, royalty-free, perpetual, irrevocable,
non-exclusive, transferable license to all users under the terms of the
[Apache Software License v2](http://www.apache.org/licenses/LICENSE-2.0.html) or
later.

All comments, messages, pull requests, and other submissions received through
CDC including this GitHub page may be subject to applicable federal law, including but not limited to the Federal Records Act, and may be archived. Learn more at [http://www.cdc.gov/other/privacy.html](http://www.cdc.gov/other/privacy.html).

## Records Management Standard Notice
This repository is not a source of government records, but is a copy to increase
collaboration and collaborative potential. All government records will be
published through the [CDC web site](http://www.cdc.gov).
1 change: 0 additions & 1 deletion docs/readme.rst

This file was deleted.

0 comments on commit fd59b31

Please sign in to comment.