Getting Started

ProsperDatareader is a group of libraries designed to help get common data. Additional data-sources should be easy to extend as long as the result data can be returned by Pandas. Inspiration taken from pandas-datareader

Using ProsperDatareader

pip install ProsperDatareader

It is encouraged to import the smallest scope required.

1
2
3
import prosper.datareader.stocks as stocks

todays_quote = stocks.prices.get_quote_rh('MU')

Contributing Data Sources

The purpose of this project is to provide a common framework to fetch/parse data and put appropriate testing scaffolding around feeds. New data sources are always encouraged!

The template for adding new functions is:

Consider Top-Level Needs

If you are extending an existing framework, think of the common questions/data returned. More stock functions should go with prosper.datareader.stocks.

Adding a wild new segment? A new top-level module should be considered. EX: adding Albion data parsing should be added to prosper.datareader.albion.

All functions in top-level scopes should return pandas.DataFrame.

Consider Data Sources

Though functionality might be mixed between scopes, sources should be split by API provider. prosper.datareader.cryptocompare contains all the API parsing required to interface with CryptoCompare. These individual REST interfaces should then be imported into their useage scope like prosper.datareader.coins to return the data users expect.

Most data-source feeds should probably have jsonschema tests.

Developing ProsperDatareader

Support for additional data sources is greatly encouraged. Please feel free to submit a pull request on GitHub to help us support the feeds you use!

Requirements for Code

  1. Please follow pylint guidelines.
  2. All functions need Napoleon Style docstrings.
  3. 95% test coverage expected.
    • Please add jsonschema files to test new endpoints.
    • logger.error() messages encouraged where coverage is infeasible
  4. Docs for new features.
  5. Top-level functions should return pandas.DataFrame objects.

Testing

python setup.py test

Py.Test is automatically integrated into the project. Test requirements remain separate, but can be expanded with tests_require in setup.py.

The most important part of our testing suite is jsonschema coverage for data sources. Using Travis-CI + jsonschema we can stay ahead of breaking API changes. This also helps us guarantee reliability for everyone who uses our tools!

Coverage requirements are in place to offload worry. Though #pragma: no cover should be used sparingly, there are situations we cannot design for. Please use logger.error() messaging in sticky spaces so ProsperCommon users can be alerted when something unexpected fails.

Docs

sphinx-build -b html docs/ webpage/

We use Sphinx to generate both pages and auto-documentation for Napoleon Style docstrings. This allows us to upload to readthedocs.io.

Documentation gives us a chance to show users how our utilities work. Though we do not expect every function to have a paragraph in docs, we do expect top-level user functions to describe their use with ..code-block: python samples where appropriate.

WARNING: Due to dependencies, auto-documentation is currently broken

Release

Release is handled by tagging + Travis-CI. Tagged versions are automatically pushed to PyPI.

Release message should include useful update notes, and versions should follow Semantic Versioning standard.