Particulars of the use of buildbot for ABINIT

Although the ETSF test farm might be accessed independently of buildbot, the usual mode of interaction with the test farm is through the buildbot system. In the following, we suppose that you are a well-identified developer of ABINIT, hence, you have your own git branches.

There are two ways to trigger the tests on the test farm:

• a merge request of your branch to trunk/develop will trigger nightly (the bunch starts around 10:30pm European time) the launch of automatic tests on all the “nightly” builders ;
• a general “on-demand” Web page on bbportal. You can contact Xavier if you think you are sufficiently expert to use it, provided (of course) you need it for your development.

You will receive by mail the results of the execution of the tests, concerning each nightly builder, one mail for each builder. Congratulation if your compilation succeeded, all the tests succeeded, etc !

Let's now analyze what to do in case of failure. The mail from buildbot is rather self-explanatory, and one is quickly lead, through hyperlinks (“further details”) to a list of the steps that were executed by buildbot, the failing one being indicated in red.

If it is step “10 make_core”, you have to click on the “10.2 make” or “10.3 stderr” hyperlink, and you will see the usual log/err file produced by the compiler. In case you would like to have direct access on the machine, this is possible, see below.

If it is step “12 testsuite”, you have to click on the “12.4 summary” hyperlink, to identify whether tests “passed” or “failed”, and to identify the test directories where such things happened. Remember : on the reference machine “abiref”, all tests must “succeed”, while on the other machines, no one test can “fail”, but a “pass” is OK. More execution information is found in the “12.2 xreport” hyperlink.

In order to correct the test in the other series, use “12.5 All Results” hyperlink to identify precisely the tests that must be corrected.

The links “12.1 stdio” and “12.3 testsbot” are not very often used in the debugging process by standard users, but more by the maintainers.

12.1 stdio     = the stdout of buildbot (Usually NOT useful for debugging)
12.2 xreport   = detailed output of execution
12.3 testbot   = output of the configuration use to drive // tests (Usually NOT useful for debugging)
12.4 summary   = a short table, summarizing the status (succeeded, passed, failed) of the tests, per directory
12.5 All Results = all results grouped by number of "cpu" (np= 1,2,4,10,...).

In order to correct the test in the “22. abirules” or “23. buildsys” series executed with abiref_gnu_5.3_debug, the “diff” hyperlink will likely be the most useful for you to correct your source code. Perhaps you will need also to complete the information from “22.4 log” by a look to the “20.2 make” hyperlink. Do not hesitate to contact Xavier Gonze to find some reasonably elegant way to get rid of the “unused variable” or “unused arguments” problems in delicate cases.

If you have git branches, you are entitled to have interactive access to each of the testfarm servers. Moreover, you can login as “buildbot” user, and place yourself as if you were the one who just made the run reported in the mail that you received. Hence, you are directly placed in an environment ready to debug.

1. Login on a gateway (well, the first time, contact Jean-Michel Beuken, to allow your machine to connect to the private network - the following instruction assumes your public key is ~/.ssh/id_rsa_abinit ) :
   create a new section in ~/.ssh/config
   

   Host abifarm
      Hostname hall.abinit.org
      Port XXXX
      User USERNAME
      IdentityFile ~/.ssh/id_rsa_abinit

   and use
   

   ssh abifarm

   This will give you access to the machine called "discourse".

2. To take the hand over buildbot, execute :
   sudo su - buildbot
   
3. Now, you need to access other machines of the testfarm, use the supplementary step
   ssh <name_of_the_machine>
   where <name_of_the_machine> might be eos, scope, alps, etc. e.g. ssh eos
   The first part of the name of the bot is the name_of_the_machine.
4. cd ABINIT (this is needed only on selected servers)
5. cd to the directory for the particular builder that you want to debug (the machine might support more than one builder) ( ex: cd alps_gnu_9.3_openmpi )
6. cd to the directory for the particular branch that you want to debug (the one bearing your name) ( ex : cd trunk_develop )
7. Use the command “module load NAME_OF_BUILDER” to select the compiler that you want to use (type “module avail” to see the list of options).
   Try to solve your problem … ! You can modify the files, recompile, etc …
8. The server can handle usual git commands. So you might make modifications, commit and push to your branch. Note however that by default, buildbot is in a “detached HEAD” state. You can check this by issuing git branch -a. You need to checkout your branch, with the command git checkout <name_of_your_branch>. Also, you need to checkout the file config/specs/testfarm.conf, with the command git checkout config/specs/testfarm.conf
9. Of course, after commit and push, it is also advised to test your modifications on your own machine, before using again the test farm, although this is not mandatory.

In 2026 (summer), most workers (=physical machines) of the ABINIT test farm will be physically migrated to the CISM, the High-Performance Computing (HPC) center of UCLouvain. The CISM will take care of the physical maintenance of the servers, the OS upgrades, installation of part of the software stack (compilers/MPI/NetCDF/NetCDF-Fortran/HDF5/libxc/linear algebra) while the ABINIT team will be responsible of the specific ABINIT software stack and related libraries (PSML, Wannier90, AtomPAW, Bigdft), and of course, managing the daily/nightly ABINIT tests. The CISM is a member of the Belgian (French side) Consortium of University HPC centers, with acronym CECI. Please, have a look at CISM and CECI Web sites ...

In order to have access to the CISM machines of the ABINIT test farm, e.g. for debugging purposes, the first step is to obtain a CECI login.

• The ABINIT developers working in a Belgian university (or being officially scientific collaborators in a Belgian university) can obtain such an account directly from the CECI login portal
.
• The other ABINIT developers can obtain a CECI guest login. An ABINIT developer with a permanent position at UCLouvain should be contacted. This person will be referred as CISM-ABINIT contact in what follows. For the time being (2026), contact Xavier. Needed: your first and last name and preferred ID. The CECI login portal allows the permanent UCLouvain ABINIT developers to ask for such CECI guest login for you. The procedure might take a few days.

If you have a UCLouvain CECI login (possibly a guest one), then login on "manneback". This might be as simple as

ssh manneback

This command allows you to access manneback through the gwceci.cism.ucl.ac.be gateway (but this step should be transparent to you).

If you have a ULiege CECI login, then you will be able to access manneback from lemaitre4.cism.ucl.ac.be (to which you have access from the ULiege gateway). However, your manneback account will need to be created by the CISM people first. Hence, contact also the CISM-ABINIT contact, who will relay to CISM.

Anyhow, when you will have an access to manneback, you will have a prompt like the following :

 [xgonze@mbackf1 ~]$ 

Then, send the file .ssh/id_ecdsa.pub that you have in your manneback account to the CISM-ABINIT contact. This public SSH key has been automatically created when your manneback account has been created. It has nothing to do with your CECI SSH public key. This manneback public SSH key will be placed in the `~/.ssh/authorized_keys` file of the abinitbot generic user of the mb-mil016 machine, part of manneback. When you will be told that this is done, impersonate as abinitbot on the specific machine mb-mil016 of the manneback cluster:

ssh abinitbot@mb-mil016

Now you have access to the CISM ABINIT testfarm working directories !

This finishes the initialisation of your access to the CISM part of the ABINIT test farm. For some shortcut, see the next section, though...

Then comes the usual (everytime that you want to debug) procedure. Log in as abinitbot@mb-mil016. To debug on the the MANN bot (manneback_gnu_14.2_hpc - the only one currently located at CISM):

1. cd /home/ucl/modl/abinitbot ; source modules.sh ; cd run
2. cd to the directory for the particular branch that you want to debug (the one bearing your name) ( ex : cd trunk_develop )
3. You can proceed now like for the other machines of the test farm, see the step 8 and step 9 of the previous section !

In order to make the abinitbot@mb-mil016 access more straightforward, e.g. from your own laptop (or your preferred machine in your institution), insert the following information in the `~/.ssh/config` of your laptop (changing CECI-USER-ID by your CECI user ID) in case you are from UCLouvain or have a guest UCLouvain CECI account

 Host gwceci
   HostName gwceci.cism.ucl.ac.be
   User CECI-USER-ID
   IdentityFile ~/.ssh/id_rsa.ceci
   IdentitiesOnly yes

 Host manneback
   HostName manneback.cism.ucl.ac.be
   User CECI-USER-ID
   ProxyJump gwceci
   IdentityFile ~/.ssh/id_rsa.ceci
   IdentitiesOnly yes

 Host mb-mil016
   user abinitbot
   ProxyJump manneback
   identityfile ~/.ssh/id_ecdsa_manneback
   IdentitiesOnly yes

If you are from ULiege, and can access Lemaitre4 through the ULiege gateway, insert the following information in the `~/.ssh/config` of your laptop (changing CECI-USER-ID by your CECI user ID).

 Host gwceci
   HostName gwceci.uliege.be
   User CECI-USER-ID
   IdentityFile ~/.ssh/id_rsa.ceci
   IdentitiesOnly yes

 Host lemaitre4
   HostName lemaitre4.cism.ucl.ac.be
   User CECI-USER-ID
   ProxyJump gwceci
   IdentityFile ~/.ssh/id_rsa.ceci
   IdentitiesOnly yes

 Host manneback
   HostName manneback.cism.ucl.ac.be
   User CECI-USER-ID
   ProxyJump lemaitre4
   IdentityFile ~/.ssh/id_rsa.ceci
   IdentitiesOnly yes

 Host mb-mil016
   HostName mb-mil016
   User abinitbot
   ProxyJump manneback
   IdentityFile ~/.ssh/id_ecdsa_manneback
   IdentitiesOnly yes

In both cases, copy the private SSH key `~/.ssh/id_ecdsa` of your Manneback account (not the abinitbot@mb-mil016 one) to your laptop, e.g. by issuing on your laptop

 scp manneback:.ssh/id_ecdsa ~/.ssh/id_ecdsa_manneback 

At this stage, the following command should also work from your laptop:

ssh abinitbot@mb-mil016