Note
These instructions describe installing CATMAID from source for long-term use. If you want to quickly try CATMAID for evaluation or demonstration purposes, consider using the docker image.
Basic Installation Instructions¶
These installation instructions have been tested on Ubuntu 20.04 LTS (Focal Fossa), so may need some minor changes for other Debian-based distributions. For installation on Mac OS X, first read these additional instructions.
Introduction¶
The most fundamental dependencies of CATMAID are:
PostgreSQL 12 and PostGIS 2.5
CPython 3.6, 3.7, 3.8, 3.9 or PyPy3.7 (CPython 3.8 is recommended)
To get the required PostgreSQL version for Debian-based systems, such as Ubuntu, you have to add the officical Postgres repository as an extra Apt repository (if you haven’t done so already):
PG_URL="http://apt.postgresql.org/pub/repos/apt/"
APT_LINE="deb ${PG_URL} $(lsb_release -cs)-pgdg main"
echo "${APT_LINE}" | sudo tee "/etc/apt/sources.list.d/pgdg.list"
sudo apt-get install wget ca-certificates
PG_KEY_URL="https://www.postgresql.org/media/keys/ACCC4CF8.asc"
wget --quiet -O - ${PG_KEY_URL} | sudo apt-key add -
sudo apt-get update
While other Python versions are supported, we recommend the use of Python 3.8. To be able to install it on Ubuntu 16.04 and earlier, the following needs to be done:
sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt-get update
The Django version we use also requires a GDAL version of at least 2.0. The
installed version can be checked using gdalinfo --version
. Should version 2
or newer not be available on your system, use the following PPA:
sudo add-apt-repository ppa:ubuntugis/ppa
sudo apt-get update
And then you can install these dependencies with:
sudo apt-get install python3.8 postgresql-12 postgresql-12-postgis-2.5 gdal-bin
CATMAID is based on the Django web framework. If you just wish to work on developing CATMAID, you can use Django’s built-in lightweight development server. However, for production you will need to configure your webserver to talk to the CATMAID web application using WSGI. We have successfully tested using either Apache with mod_wsgi, or Nginx with gevent, gunicorn or uSWGI. Particularly Nginx with uWSGI have been tested extensively. Setting up one of these web servers is described in later sections.
1. Clone the repository¶
The git repository is hosted at https://github.com/catmaid/CATMAID - clone this repository
somewhere outside your web root, e.g. in /home/alice
, so that
the source code is in /home/alice/catmaid
:
git clone https://github.com/catmaid/CATMAID.git catmaid
2. Install required Python packages¶
We recommend the use of Python 3.8 or newer for production use. With a few limitations PyPy3 can be used as well (no cropping, no back-end plotting, no synapse clustering, no ontology clustering).
We strongly recommend that you install all Python package dependencies into a virtualenv, so that they are isolated from the system-wide installed packages and can be upgraded easily. Some of these Python packages depend on system-wide libraries that you will need to install in advance, however. You can do this with the following command on Ubuntu (you may need additional PPAs, such as ubuntugis and postgresql):
sudo apt-get install postgresql-12 postgresql-12-postgis-2.5 gcc gfortran \ git python3.8-dev postgresql-common libpq-dev \ libhdf5-serial-dev libboost-dev virtualenvwrapper \ libboost-python-dev libxml2-dev libxslt1-dev \ libjpeg-dev libtiff-dev libblas-dev liblapack-dev \ pkg-config binutils libproj-dev gdal-bin
Virtualenv Wrapper needs to source your environment. Start a new terminal or if you are using the bash:
source ~/.bashrc
Please test if virtualenvwrapper
is set up correctly, by executing:
mkvirtualenv --version
If it gives you a version, everything is fine. Otherwise, e.g. if the command
mkvirtualenv
is not found, add the following line to your ~/.bashrc
file
and call source ~/.bashrc
again:
source /usr/share/virtualenvwrapper/virtualenvwrapper.sh
To create a new virtualenv for CATMAID’s Python dependencies, you can do:
mkvirtualenv --no-site-packages -p /usr/bin/python3.8 catmaid
That will create a virtualenv in ~/.virtualenvs/catmaid/
, and
while your virtualenv is activated, Python libraries will be
imported from (and installed to) there. After creating the
virtualenv as above, it will be activated for you, but in new
shells, for example, you will need to activate it by running:
workon catmaid
Note
Many distributions ship with an outdated version of Pip. This is the tool we use to install Python packages within the virtualenv, so let’s update it first:
python -m pip install -U pip
Note
It is possible to use PyPy as Python implementation, which can improve performance of back-end heavy endpoints. Most functionality is available, except for the following: Ontology clustering, Cropping, Synapse clustering, HDF 5 tiles and User analytics. To use PyPy, a new virtualenv using the PyPy executable has to be created:
mkvirtualenv --no-site-packages -p /usr/bin/pypy catmaid
Note
If you are using Python 3.6 or newer on Ubuntu 14.04 and 16.04, never uninstall Python 3.5, because it might break some parts of the system.
Install all of the required Python packages with:
cd /home/alice/catmaid/django
pip install -r requirements.txt
If that worked correctly, then the second-last line of output will begin
Successfully installed
, and list the Python packages that have just been
installed.
If you set up a production environment, please also install the
requirements-production.txt
file:
pip install -r requirements-production.txt
This will rebuild some dependencies from source. If asked whether existing packages should be replaced by the new version, answer with yes.
3. Install and configure PostgreSQL¶
If you are comfortable with creating a new PostgreSQL database for CATMAID, then
you should do that and continue to the next section. If you decide to do so,
please make sure to also install the postgis
extension and the pg_trgm
extension for the new CATMAID database. The advice here is a suggested approach
for people who are unsure what to do.
If you are uncomfortable with using the PostgreSQL interactive
terminal from the command line, you may wish to install an
alternative interface, such as pgAdmin (sudo apt-get install
pgadmin3
) or phpPgAdmin (sudo apt-get install phppgadmin
).
We suppose for the examples below that you want to create a
database called catmaid
and a database user called
catmaid_user
. Firstly, we need to reconfigure PostgreSQL to
allow password-based authentication for that user to that
database. To do that, edit the file
/etc/postgresql/12/main/pg_hba.conf
and add this line as the
first rule in that file:
local catmaid catmaid_user md5
After saving that file, you need to restart PostgreSQL with:
sudo service postgresql reload
You can generate the commands for creating the database and
database user with the scripts/createuser.sh
helper script.
This takes the database name, the database user and the user’s
password as arguments and outputs some commands that can be
interpreted by the PostgreSQL shell. These can be piped
directly to psql
, so you could create the database and the
user with, for example:
scripts/createuser.sh catmaid catmaid_user p4ssw0rd | sudo -u postgres psql
Besides creating the database and the database user, it will also enable a
required Postgres extension, called postgis
. You should now be able to
access the database and see that it is currently empty except for PostGIS
relations, e.g.:
psql -U catmaid_user catmaid
Password for user catmaid_user:
psql (12.9 (Ubuntu 12.9-2.pgdg20.04+1))
Type "help" for help.
catmaid=> \d
List of relations
Schema | Name | Type | Owner
--------+-------------------+-------+----------
public | geography_columns | view | postgres
public | geometry_columns | view | postgres
public | raster_columns | view | postgres
public | raster_overviews | view | postgres
public | spatial_ref_sys | table | postgres
4. Create the Django settings files¶
Now you should change into
/home/alice/catmaid/django/
and run:
cp configuration.py.example configuration.py
You should now edit configuration.py
and fill in all the
details requested. Then you should run:
./create_configuration.py
This will output some suggested Nginx and Apache configuration in the
terminal, and generate the files django.wsgi
and settings.py
in /home/alice/catmaid/django/projects/mysite
. An explanation of all
possible settings in the settings.py file can be found here.
5. Create the database tables¶
The commands in the following sections are all based on the Django site’s admin
script manage.py
, which would be in /home/alice/catmaid/django/projects
,
so these instructions assume that you’ve changed into that directory:
cd /home/alice/catmaid/django/projects
Now create all required tables and bring the database schema up to date for applications that mange changes to their tables with South:
./manage.py migrate
6. Prepare the static files¶
The static files (mostly Javascript, CSS and image files) that
CATMAID requires need to be collected together into
/home/alice/catmaid/django/static
before they will be
available. To do this, you need to run:
./manage.py collectstatic -l
(The -l
means to create symbolic links to the original
location of the files rather than copy them.)
7. Create an administrative user¶
In order to be able to log in to the CATMAID admin interface, you will need to create a “superuser” account to log in with. You can do this with:
./manage.py createsuperuser
8. Optionally add some example projects¶
If you want to have some example projects to try in your new CATMAID instance, you can create a couple with the following command:
./manage.py catmaid_insert_example_projects --user=1
(The superuser you just created should have the user ID 1
.)
9. Try running the Django development server¶
You can run the Django development server with:
./manage.py runserver
You should then be able to visit your instance of catmaid at http://localhost:8000. Note though that in its default configuration CATMAID
will prevent static files from being served with the runserver
command and
while the website should load it may not look like expected. To temporarily
allow this to test without enabling debug mode, set SERVE_STATIC = True
in
settings.py
. For a production setup, the webserver should take care of
serving static files.
10. Setting up a production webserver¶
You have various options for setting up CATMAID with a production webserver - you can choose from (at least) the following two:
Nginx and uWSGI (or alternatively with Gevent or Gunicorn)
Apache and mod_wsgi, in which case see Setting up Apache and mod_wsgi
We prefer to use Nginx because of a more straight-forward configuration, smaller memory footprint and better performance with available WSGI servers. In production setups we made good experience with uWSGI running behind Nginx, which is described in more detail in the Nginx and uWSGI section.
Note if the domain you are serving your image data from is different from where CATMAID is running, CORS headers have to be sent by the image server or some aspects of the web front-end won’t work as expected. For more details, have a look here. The same is true for CATMAID back-ends that should be accessed by clients originating not from the same domain. Check the CORS setup section for more details.
In general you want to fine-tune your setup to improve performance. Please have a look at our collection of advice for the various infrastructure parts (e.g. webserver, database, file system). This can really make a difference. An explanation of all possible settings in the settings.py file can be found here.
11. Using the admin interface¶
You should be able to log in to the CATMAID admin interface and
complete administration tasks by adding /admin/
after the
root URL of your CATMAID instance. For example, with the
development server, this would be:
http://localhost:8000/admin/
… or, to use the variables used in the configuration.py
(see step 4), the
URL would be:
http://<catmaid_servername>/<catmaid_subdirectory>/admin/
12. Creating tiles for new CATMAID stacks¶
You can generate the image tiles for a stack with the
scripts/tiles/tile_stack
script or by exporting from TrakEM2
with its “Export > Flat Images” option and selecting the “Export
for web” checkbox. Make the folder with the image pyramid
web-accessible and use the URL as image_base
URL for your
stack.
13. Making tools visible¶
CATMAID offers a growing set of tools. To not overload the user-interface, all tools which go beyond navigation are hidden by default. Which tools are visible is stored in a user profile for each user. You can adjust these settings at the bottom of the page while editing a user in the admin interface.