===============
CI integrations
===============

*calcite* provides different levels of integration for each CI provider (which can be different from your git project provider).
You will need one since *calcite* does not build/run the performance tests itself. We however are thinking of providing benchmarking environment at a later point in time.

In the meantime, please have a look at the :doc:`/performance/stability/index` chapter to understand how to best setup a benchmarking machine and testsuite.

We try to support all providers as best as we can within our capabilities, and do not have contracts or special arrangements with any CI provider.
That is why we will soon open source our command line interface so that any provider can be supported easily.

.. note::
  Even if your CI provider is not listed, it is still possible to use *calcite* !
  It simply means some information will not be detected automatically, but you can provide them through the CLI arguments.

The :ref:`calcite upload <usage/upload:Uploading with calcite-cli>` CLI will try to detect the CI provider/tool you are using based on environment variables, which makes upload configuration easier.

.. 
  The CI provider integration has multiple levels of support:
  - The CLI detects the providers and automatically fills in basic informations such as the git commit, branch, build ID.
  - The CLI detects pull requests, and the server can automatically delay :ref:`usage/triggers:Triggers`.
  
  .. @TODO@ Update pull requests support
  
  =================================== =================== ==================
        Provider                       Basic information   Delayed triggers
  =================================== =================== ==================
    `AppVeyor`_                               ✓                   ✗    
    `Azure Pipelines`_                        ✓                   ✗    
    `Circle CI `_                             ✓                   ✗    
    `Github Actions`_                         ✓                   ✗    
    `Gitlab CI`_                              ✓                   ✗    
    Jenkins_                                  ✓                   ✗    
    Travis_                                   ✓                   ✗    
  =================================== =================== ==================

=================================== ======== ======== ========= ==============
      Provider                       commit   branch   buildId   Pull request
=================================== ======== ======== ========= ==============
  `AppVeyor`_                           ✓        ✓       ✓          ✓        
  `Azure Pipelines`_                    ✓        ✓       ✓          ✓        
  `Circle CI`_                          ✓        ✓       ✓          ✓        
  `Github Actions`_                     ✓        ✓       ✓          ✓        
  `Gitlab CI`_                          ✓        ✓       ✓          ~ (*)        
  Jenkins_                              ✓        ✓       ✓          ✓        
  Travis_                               ✓        ✓       ✓          ✓        
=================================== ======== ======== ========= ==============

\* Can only detect merge requests with GitlabCI 11.6+ and only if ``only: [merge_requests]`` or ``rules`` syntax is used and the merge request is created. See ``CI_MERGE_REQUEST_ID`` `here <https://docs.gitlab.com/ce/ci/variables/predefined_variables.html>`_. We do not support Github pull requests yet.

.. _AppVeyor: https://appveyor.com/
.. _Azure Pipelines: https://azure.microsoft.com/en-us/services/devops/pipelines
.. _Circle CI: https://circleci.com
.. _Github Actions: https://github.com/features/actions
.. _Gitlab CI: https://about.gitlab.com/stages-devops-lifecycle/continuous-integration
.. _Jenkins: https://www.jenkins.io
.. _Travis: https://travis-ci.org