Google Summer of Code 2020 Work Product Submission
August 31, 2020
Student
Simon Pelletier
Organization
INCF – International Neuroinformatics Coordinating Facility
Project Title
Improve LORIS API development and documentation
Mentors
Christine Rogers, Xavier Lecours-Boucher
Project Proposal
- Improvement of existing API endpoints and solving priority issues for API infrastructure
- Develop new workflow templates
- Creating new endpoints for modules of Loris
- Writing the documentation
- Provide comprehensive tutorials in Jupyter notebooks as examples of how to use the API new endpoints
Project Overview
Loris uses a LAMP structure (Linux, Apache, MySQL and PHP), so the endpoints use HTTP requests to communicate to the database. The API makes data acquisition from a scripting language, such as Python or R, easy and secure. It is an invaluable tool to develop workflows that can easily and rapidly be deployed.
During the summer, I improved the API by fixing bugs and writing an integration test suite that ensures future modifications of the codebase do not cause problems down the line in the API. The integration tests are run every time new code is pushed on a branch of Loris using Travis CI, so it will prevent bugs to manifest by blocking any modifications that breaks the API. Given the size and complexity of the Loris codebase, a bug that goes unnoticed can be long to fix and slow down future developments. This problem would undoubtedly grow as the codebase grows, thus the importance of writing integration tests early. These automatic tests will obviously help for testing during the code review process, as some tests can be time consuming, especially when they have to be repeated often when debugging.
In addition to improving the API, I wrote documentation in the form of Jupyter notebooks with examples on how to use each endpoint of the API with python. I also developed work flow tutorials in the form of Jupyter notebooks. These tutorials demonstrate how to use the API’s endpoints to download all images and electrophysiology recordings contained in a project, then pre-process the samples for subsequent analysis. The purpose of these notebooks is not to create additional modules to analyse the data or make an exhaustive list of possible subsequent analysis, but it is usually very helpful as a starting point for the users and mights how useful LORIS could be if used as part of neuroinformatics workflows.
Summer Timeline
May
The first month was spent installing Loris on my own Virtual Machine. I also spent time learning new skills (e.g. how to use the text editor Vim), getting to know the codebase of Loris and the team of developers.
June
Integration tests
In June, I started to write a brand new test suite for LORIS REST API endpoints.
New endpoints
I started writing a new endpoint to download all DICOM files available in a project.
July
Integration tests
Most of the work was done by the end of July, but it was merged on August 27th. More work needs to be done to cover all of the current endpoints, but testing of the API will now be more efficient and will make the development of new endpoints faster and safer not to break existing endpoints.
Jupyter notebooks
I worked on the development of two new Jupyter notebooks:
- The first notebook contains basic examples on how to use each endpoints.
- The second notebook contains an example of a workflow to pre-process all the images downloaded with the API for subsequent analysis by the researchers using Loris.
New endpoints
I continued the work started in June and started writing PUT and PATCH methods that were missing in one of the existing endpoints.
August
August was mostly dedicated to make sure all the work started in the previous months are finished and that most Pull Requests be merged by August 31st, or as close as possible to be merged.
What will happen to open PRs?
Open GSOC PRs should be merged over the next few weeks.
Summer project presentation (August 18th)
On August 18th, I presented the work I have done on the API during my Google Summer of Code.
Summary
Merged Prs
In the following link is a list of all Merged PRs that were initiated during the summer.
Open PRs
In the following link is a list of all Open PRs that were initiated during the summer.
Detailed list of PRs initiated over the summer
API improvements
PR #6508 Typo fix
PR #6672 API Bugs fix
PR #6828 Display error message when MINCToolsPath is not configured
Tests
PR #6671 Creation of a new integration test suite
- 44 endpoints tested (557 assertions)
- 16 endpoint tests skipped because of missing data in docker environment (Issue #6909)
PR #6601 API Test Plan
New endpoints and other API extensions
PR #6775 New endpoint for Projects Dicoms
PR #6780 PUT and PATCH methods added to endpoint candidates/{candid}/{visit}/instruments/{instrument}/flags
PR #6905 Add missing files paths to attributes list
PR #6899 New features for Dicoms and images endpoints
Jupyter Notebooks
PR #6854 Jupyter Notebooks to demonstrate how to use the API.
PR #6890 Creation of an imaging dataset