Advices and tips for contributors

If you want to become active as developer, we provide all important information here to make the start as easy as possible. The code you produce should be seamlessly integrable into PyDynamic by aligning your work with the established workflows. This guide should work on all platforms and provide everything needed to start developing for PyDynamic. Please open an issue or ideally contribute to this guide as a start, if problems or questions arise.

Guiding principles

The PyDynamic development process is based on the following guiding principles:

Get started developing

Get the code on GitHub and locally

For collaboration, we recommend forking the repository as described here. Simply apply the changes to your fork and open a Pull Request on GitHub as described here. For small changes it will be sufficient to just apply your changes on GitHub and send the PR right away. For more comprehensive work, you should clone your fork and read on carefully.

Initial development setup

This guide assumes you already have a valid runtime environment for PyDynamic as described in the README. To start developing, install the required dependencies for your specific Python version. To find it, activate the desired virtual environment and execute:

(PyDynamic_venv) $ python --version
Python 3.8.8

Then upgrade/install pip and pip-tools which we use to pin our dependencies to specific versions:

(PyDynamic_venv) $ pip install --upgrade pip pip-tools

You can then initially install or at any later time update all dependencies to the versions we use. From the repository root run pip-tools’ command sync e.g., for Python 3.9:

(PyDynamic_venv) $ python -m piptools sync requirements/dev-requirements-py39.txt requirements/requirements-py39.txt

Advised toolset

We use black to implement our coding style, Sphinx for automated generation of our documentation on ReadTheDocs. We use pytest managed by tox as testing framework backed by hypothesis and coverage. For automated releases we use python-semantic-release in our pipeline on CircleCI . All requirements for contributions are derived from this. If you followed the steps for the initial development setup you have everything at your hands.

Coding style

As long as the readability of mathematical formulations is not impaired, our code should follow PEP8. For automating this uniform formatting task we use the Python package black. It is easy to handle and integrable into most common IDEs, such that it is automatically applied.

Commit messages

PyDynamic commit messages follow some conventions to be easily human and machine-readable.

Commit message structure

Conventional commit messages are required for the following:

Parts of the commit messages and links appear in the changelogs of subsequent releases as a result. We use the following types:

  • feat: for commits that introduce new features (this correlates with MINOR in semantic versioning)

  • docs: for commits that contribute significantly to documentation

  • fix: commits in which bugs are fixed (this correlates with PATCH in semantic versioning)

  • build: Commits that affect packaging

  • ci: Commits that affect the CI pipeline

  • test: Commits that apply significant changes to tests

  • chore: Commits that affect other non-PyDynamic components (e.g., ReadTheDocs, Git , … )

  • revert: commits, which undo previous commits using git revert

  • refactor: commits that merely reformulate, rename or similar

  • style: commits, which essentially make changes to line breaks and whitespace

  • wip: Commits which are not recognizable as one of the above-mentioned types until later, usually during a PR merge. The merge commit is then marked as the corresponding type.

Of the types mentioned above, the following appear in separate sections of the changelog:

  • Feature: feat

  • Documentation: docs

  • Fix: fix

  • Test: test

Commit message styling

Based on conventional commit messages, the so-called should complete the following sentence:

If this commit is applied, it will…

The first line of a commit message should not exceed 100 characters.

BREAKING CHANGEs

If a commit changes parts of PyDynamic’s public interface so that previously written code may no longer be executable, it is considered a BREAKING CHANGE (correlating with MAJOR in semantic versioning). This has to be marked by putting BREAKING CHANGE in the footer of the message as described in the conventional commit messages .

Examples

For examples please checkout the Git Log.

Testing

We strive to increase our code coverage with every change introduced. This requires that every new feature and every change to existing features is accompanied by appropriate pytest testing. We test the basic components for correctness and, if necessary, the integration into the big picture. It is usually sufficient to create appropriately named methods in one of the existing modules in the subfolder test. If necessary, add a new module that is appropriately named.

Workflow for adding completely new functionality

In case you add a new feature you generally follow the pattern:

  • read through and follow this contribution advices and tips, especially regarding the advised tool set and coding style

  • open an according issue to submit a feature request and get in touch with other PyDynamic developers and users

  • fork the repository or update the master branch of your fork and create an arbitrary named feature branch from master

  • decide which package and module your feature should be integrated into

  • if there is no suitable package or module, create a new one and a corresponding module in the test subdirectory with the same name prefixed by test_

  • after adding your functionality add it to all higher-level __all__ variables in the module itself and in the higher-level __init__.pys

  • if new dependencies are introduced, add them to setup.py or dev-requirements.in

  • during development write tests in alignment with existing test modules, for example test_interpolate or test_propagate_filter

  • write docstrings in the NumPy docstring format

  • as early as possible create a draft pull request onto the upstream’s master branch

  • once you think your changes are ready to merge, request a review from the PTB-M4D/pydynamic-devs (you will find them in the according drop-down) and mark your PR as ready for review

  • at the latest now you will have the opportunity to review the documentation automatically generated from the docstrings on ReadTheDocs after your reviewers will set up everything

  • resolve the conversations and have your pull request merged

Documentation

The documentation of PyDynamic consists of three parts. Every adaptation of an existing feature and every new feature requires adjustments on all three levels:

  • user documentation on ReadTheDocs

  • examples in the form of Jupyter notebooks for extensive features and Python scripts for features which can be comprehensively described with few lines of commented code

  • developer documentation in the form of comments in the code

User documentation

To locally generate a preview of what ReadTheDocs will generate from your docstrings, you can simply execute after activating your virtual environment:

(PyDynamic_venv) $ sphinx-build docs/ docs/_build
Sphinx v3.1.1 in Verwendung
making output directory...
[...]
build abgeschlossen.

The HTML pages are in docs/_build.

After that, you can open the file ./docs/_build/index.html relative to the project’s root with your favourite browser. Simply re-execute the above command after each change to the docstrings to update your local version of the documentation.

Examples

We want to provide extensive sample material for all PyDynamic features in order to simplify the use or even make it possible in the first place. We collect the examples in a separate repository PyDynamic_tutorials separate from PyDynamic to better distinguish the core functionality from additional material. Please refer to the corresponding README for more information about the setup and create a pull request accompanying the pull request in PyDynamic according to the same procedure we describe here.

Comments in the code

Regarding comments in the code we recommend investing 45 minutes for the PyCon DE 2019 Talk of Stefan Schwarzer, a 20+-years Python developer: Commenting code - beyond common wisdom.

Manage dependencies

As stated in the README and above we use pip-tools for dependency management. The requirements’ subdirectory contains a requirements.txt and a dev-requirements.txt for all supported Python versions, with a suffix naming the version, for example requirements-py36.txt To keep them up to date semi-automatically we use the bash script requirements/upgrade_dependencies.sh. It contains extensive comments on its use. pip-tools’ command pip-compile finds the right versions from the dependencies listed in setup.py and the dev-requirements.in.

Licensing

All contributions are released under PyDynamic’s GNU Lesser General Public License v3.0.