To get gitlab running in production, I use a customized compose file:
version: '3.3' services: web: image: 'gitlab/gitlab-ce:latest' restart: always hostname: 'gitlab.example.org' environment: GITLAB_OMNIBUS_CONFIG: | external_url 'https://gitlab.example.org' letsencrypt['enabled'] = false dns: - 18.104.22.168 - 22.214.171.124 ports: - '8180:80' - '8143:443' volumes: - '/srv/gitlab/config:/etc/gitlab' - '/srv/gitlab/logs:/var/log/gitlab' - '/srv/gitlab/data:/var/opt/gitlab' deploy: resources: limits: cpus: '2' memory: 4000M reservations: cpus: '0.1' memory: 500M
Start container using docker-compose in compatibility mode:
docker-compose --compatibility up -d
docker pull gitlab/gitlab-ce:latest docker-compose down docker image prune -f docker-compose --compatibility up -d
I recently upgraded my gitlab-ce docker container to the latest version 13.1.2, coming from 12.9.10.
Now unfortunately gitlabs PostgreSQL no longer works :(
The data directory was initialized by PostgreSQL version 10, which is not compatible with this version 11.7.
I found out, that it was possible to migrate the database from version 10 to 11 with the old gitlab version, but going back to this doesn´t work. All parts, except the database, were migrated to the new version. So, how to fix it?
Using docker-compose to start our container will start gitlab in it. Because of the database failure, the container restart every minute. We don´t want this for our migration work. So we start it manually and access the bash in it.
Please replace (at least) the hostname and the other options with your custom configuration!
docker run --rm \ --hostname gitlab.example.org \ --publish 8143:443 --publish 8180:80 \ --env GITLAB_OMNIBUS_CONFIG="external_url 'https://gitlab.example.org'; letsencrypt['enabled'] = false" \ --name gitlab_custom \ --volume /srv/gitlab/config:/etc/gitlab \ --volume /srv/gitlab/logs:/var/log/gitlab \ --volume /srv/gitlab/data:/var/opt/gitlab \ -it gitlab/gitlab-ce:latest /bin/bash
Now we are in the docker container.
First of all run the default entry script to check that it really doesn`t work!
If there is a failure in the
Recipe: gitlab::database_migrations section and at least the error message
Error executing action 'run' on resource 'bash[migrate gitlab-rails database]', continue to migrate the database.
We need to install the default (old and new) PostgreSQL versions.
apt update apt install postgresql-10 postgresql-11
Create the destination folder for the migrated database and update the directory owner to gitlab-psql. We assume, the 'old' (version 10) database data is located at /var/opt/gitlab/postgresql/data.
mkdir -p /tmp/11/data chown -R gitlab-psql /tmp/11 /var/opt/gitlab/postgresql/data
Now we need to initialize the new database structure.
sudo -u gitlab-psql /usr/lib/postgresql/11/bin/initdb -D /tmp/11/data
Take a look at /var/opt/gitlab/postgresql/data/postgresql.conf for the right port (and set it with -p).
Before running the real migration, we should check the access and schema of the database. In case of errors, this command can be run multiple times.
sudo -u gitlab-psql /usr/lib/postgresql/11/bin/pg_upgrade -D /tmp/11/data -d /var/opt/gitlab/postgresql/data/ -v -b /usr/lib/postgresql/10/bin -B /usr/lib/postgresql/11/bin/ -p 5432 --check
If there was an error during the check, you may need to update some other directory permissions.
If the check was successful, do the database migration with the following command. This can take some time, depending on the size of your gitlab instance.
sudo -u gitlab-psql /usr/lib/postgresql/11/bin/pg_upgrade -D /tmp/11/data -d /var/opt/gitlab/postgresql/data/ -v -b /usr/lib/postgresql/10/bin -B /usr/lib/postgresql/11/bin/ -p 5432
Stop all gitlab services, backup the old database and move the migrated data to the original path.
gitlab-ctl stop rm -f /var/opt/gitlab/postgresql/.s.PGSQL.5432* mv /var/opt/gitlab/postgresql/data /var/opt/gitlab/postgresql/data_old_10 mv /tmp/11/data /var/opt/gitlab/postgresql/data
Now run the default entry script to finish the built-in gitlab migration.
If it works (some log output appears after loading the gitlab parts and of course the website works), you can press Ctrl+C, and enter exit to leave and remove the current container.
If there were still database errors, ... oops.
If there is an error like this,
PG::ConnectionBad: could not connect to server: No such file or directory | Is the server running locally and accepting | connections on Unix domain socket "/var/opt/gitlab/postgresql/.s.PGSQL.5432"?
make sure no postgresql process is running, remove the socket file and try again.
After that, leave the container.
Start your gitlab container the way you start it by default, e.g. using docker-compose up like me:
docker-compose --compatibility up -d
I hope this short explanation helped someone who had a database problem similar to mine after a gitlab upgrade.