Contribute to VDAT¶
How To¶
The suggested workflow for implementing bug fixes and/or new features is the following:
Identify or, if necessary, add to our redmine issue tracker one or more issues to tackle. Multiple issues can be addressed together if they belong together. Assign the issues to yourself.
Create a new branch from the trunk with a name either referring to the topic or the issue to solve. E.g. if you need to add a new executable, tracked by issue #1111
do_something
:svn cp ^/trunk ^/branches/do_something_1111\ -m 'create branch to solve issue #1111'
Switch to the branch:
svn switch ^/branches/do_something_1111
Implement the required changes and don’t forget to track your progress on redmine. If the feature/bug fix requires a large amount of time, we suggest, when possible, to avoid one big commit at the end in favour of smaller commits. In this way, in case of breakages, is easier to traverse the branch history and find the offending code. For each commit you should add an entry in the
Changelog
file.If you work on multiple issues on the same branch, close one issue before proceeding to the next. When closing one issue is good habit to add in the description on the redmine the revision that resolves it.
Every function or class added or modified should be adequately documented as described in Coding style.
Documentation is essential both for users and for your fellow developers to understand the scope and signature of functions and classes. If a new module is added, it should be also added to the documentation in the appropriate place. See the existing documentation for examples.
Each executable should be documented and its description should contain enough information and examples to allow users to easily run it.
Every functionality should be thoroughly tested for python 2.7 and 3.4 or 3.5 in order to ensure that the code behaves as expected and that future modifications will not break existing functionalities. When fixing bugs, add tests to ensure that the bug will not repeat. For more information see Testing.
Once the issue(s) are solved and the branch is ready, merge any pending change from the trunk:
svn merge ^/trunk
While doing the merge, you might be asked to manually resolve one or more conflicts. Once all the conflicts have been solved, commit the changes with a meaningful commit message, e.g.:
merge ^/trunk into ^/branches/do_something_1111
. Then rerun the test suite to make sure your changes do not break functionalities implemented while you were working on your branch.Then contact the maintainer of
vdat
and ask to merge your branch back to the trunk.
Information about branching and merging can be found in the svn book. For any questions or if you need support do not hesitate to contact the maintainer or the other developers.
Coding style¶
All the code should be compliant with the official python style guidelines described in PEP 8. To help you keep the code in spec, we suggest to install plugins that check the code for you, like Synstastic for vim or flycheck for Emacs.
The code should also be thoroughly documented using the numpy style. See the existing documentation for examples.
Testing¶
Note
Every part of the code should be tested and should run at least under python 2.7 and the latest two or three python 3.x releases (e.g. 3.4, 3.5 and 3.6)
vdat
uses the testing framework provided by the pytest package. The tests should cover every
aspect of a function or method. If exceptions are explicitly raised, this should
also tested to ensure that the implementation behaves as expected.
The preferred way to run the tests is using tox, an automatised test help
package. If you have installed tox, with e.g. pip install tox
, you can run
it by typing:
tox
It will take care of creating virtual environments for every supported version
of python, if it exists on the system, install vdat
, its dependences and the
packages necessary to run the tests and runs py.test
You can run the tests for a specific python version using:
py.test
or:
python setup.py test
The latter command fetches all the needed dependences, among others pytest
itself, will be fetched and installed in a .eggs
directory. Then it will
run py.test
. This command might fail when running in a virtual environment.
If you get ImportError: No module named 'numpy'
while installing scipy
,
install numpy by hand pip install [--user] numpy
and rerun it again. Use
the option --addopts
to pass additional options to py.test
.
The VDAT test suites depends on some large file that is not shipped with the
main repository. You can get the file running the following command in the
parent directory of the vdat
root:
svn checkout svn://luna.mpe.mpg.de/vdat_test_data/trunk vdat_test_data
If for any reason you have the test data somewhere else, you can tell
pytest
where they are using the --extra-data-dir
option. If the
directory is not found, pytest
will abort with a useful error message.
You can run specific tests providing the file name(s) and, optionally the name of a test. E.g.:
py.test tests/test_libvdat/test_symlink.py # runs only the tests in one file
py.test tests/test_libvdat/test_symlink.py::test_no_raw # runs only one test function
Relevant command line options:
-v verbose output: print the names and parameters of the
tests
-s capture standard output: can cause weird interactions
with the logging module
--noremove Do not remove output created output files when
tearing down the tests. Useful to inspect the log
files created by vhc when updating the tests
--extra-data-dir=EXTRA_DATA_DIR
Directory containing extra test data.
A code coverage report is also created thanks to the pytest-cov plugin and can be visualized opening
into a browser cover/index.html
. If you want a recap of the coverage
directly in the terminal you can provide one of the following options when
running py.test
:
--cov-report term
--cov-report term-missing
Besides running the tests, the tox
command also builds, by default, the
documentation and collates the coverage tests from the various python
interpreters and can copy then to some directory. To do the latter create, if
necessary, the configuration file ~/.config/little_deploy.cfg
and add to it
a section called vdat
with either one or both of the following options:
[vdat]
# if given the deploys the documentation to the given dir
doc = /path/to/dir
# if given the deploys the coverage report to the given dir
cover = /path/to/other/dir
# it's also possible to insert the project name and the type of the document
# to deploy using the {project} and {type_} placeholders. E.g
# cover = /path/to/dir/{project}_{type_}
# will be expanded to /path/to/dir/vdat_cover
For more information about the configuration file check little_deploy.
For other command line arguments type:
py.test -h
For a list of available fixtures type:
py.test --fixtures tests/
Qt bindings¶
With version 0.9.0 and the use of qtpy
, VDAT can be run against different
Qt
bindings. Ideally we would like to run VDAT against all the possible.
When writing (2018/06/15) the tox
configuration file contains hints about
this ideal; I would have liked to run the following tests:
- py27:
PyQt4
- py34:
PySide
- py35:
PySide2
- py36:
PyQt5
Unfortunately this didn’t work out so the actual test matrix is:
- py27:
PyQt4
- py34:
PyQt4
- py35:
PyQt5
- py36:
PyQt5
If any of PySide
/PySide2
becomes supported, then the test matrix should
be reverted to
A note on PySide
¶
PySide
can be installed using pip
, however it needs python to be
compiled with the -fPIC
flag. If this is not the case, you might get the
following error while installing PySide
:
/usr/bin/ld:
/home/montefra/.pyenv/versions/3.4.6/lib/libpython3.4m.a(abstract.o):
relocation R_X86_64_32S against `_Py_NotImplementedStruct' can not be used
when making a shared object; recompile with -fPIC
/home/montefra/.pyenv/versions/3.4.6/lib/libpython3.4m.a: error adding
symbols: Bad value
collect2: error: ld returned 1 exit status
libshiboken/CMakeFiles/libshiboken.dir/build.make:381: recipe for
target 'libshiboken/libshiboken.cpython-34m.so.1.2.4' failed
make[2]: *** [libshiboken/libshiboken.cpython-34m.so.1.2.4]
Error 1
CMakeFiles/Makefile2:204: recipe for target
'libshiboken/CMakeFiles/libshiboken.dir/all' failed
make[1]: ***
[libshiboken/CMakeFiles/libshiboken.dir/all] Error 2
Makefile:127: recipe for target 'all' failed
make: *** [all] Error 2
error: Error compiling shiboken
If you are using pyenv
to handle your python versions, you can get a new
python version with the correct flag using:
PYTHON_CFLAGS=-fPIC pyenv install -v 3.4.8
Documentation¶
To build the documentation you need the additional dependences described in
Optional dependences. They can be installed by hand or during vdat
installation
by executing one of the following commands on a local copy:
pip install /path/to/vdat[qt5,doc]
pip install /path/to/vdat[qt5,livedoc]
The first install sip
, pyqt5
, sphinx
, the alabaster
theme and
the numpydoc
extension; the second also installs sphinx-autobuild
.
To build the documentation in html format go to the doc
directory and run:
make html
The output is saved in _doc/build/html
. For the full list of available
targets type make help
.
If you are updating the documentation and want avoid the
edit-compile-browser refresh
cycle, and you have installed
sphinx-autobuild
, type:
make livehtml
This command compiles the documentation and serves it on http://127.0.0.1:{port}, where {port} is an available port, and open the page on your default browser. The html documentation is automatically rebuilt after every change of the source and the browser reloaded.
Please make sure that every module in vdat
is present in the
Code Documentation.
Qt Documenation¶
We distribute this documentation also with VDAT, to allow consulting it offline.
The Qt documentation can be built by sphinx itself using the following command:
make qthelp
This command creates the documentation into the _build/qthelp
directory
together with these extra files:
- Qt Help Collection Project (
.qhcp
) - Qt Help Project (
.qhp
) - Qt Compressed Help (
.qch
) - Qt Help Collection (
.qhc
)
See the Qt Help documentation for more information about those files.
All those files can also be created running tox
, or tox -e qt-doc
. In
this case the files can be found in .tox/qt-doc/tmp/qthelp
To update the documentation copy the .qch
and .qhc
files into
vdat/gui/static
. If the files have a different names from the old ones,
remove the latter from the repository with svn rm
and add the new ones with
svn add
. Don’t forget to commit to propagate the changes.