==============
Dataset upload
==============


calcite upload
==============

To upload a dataset to calcite, we recommand using our :doc:`/usage/cli`.

.. hint::
    ``npx @siliceum/calcite-cli [command arguments]`` let you run the command line directly.

.. @TODO@ Project Token (id) setup

Uploading with calcite-cli
--------------------------

Each project has a unique token, which is used when uploading a :term:`dataset <Dataset>`.
It can be found in the project details page, under the description of the project.

A dataset contains your :term:`datapoints <Datapoint>` and some metadata. It can either be provided in our json format or through a configuration file used to import the data.

.. @TODO@ Create this example !

.. hint::
    A ``fake_bench.json`` file is provided as part of the example, and this is what we will use for our first upload.

Setup your CI so that the ``CALCITE_TOKEN`` environment variables contains your project token, and use the following command to upload the ``./fake_bench.json`` dataset:

.. code-block:: bash

    calcite upload ./fake_bench.json

.. note::
    | This should automatically detect the git branch, commit, etc (see the supported :ref:`CI providers <usage/CI-integrations:CI integrations>`).
    | We however suggest overriding the default dataset :term:`config <Benchmark configuration>` name.


You can also specify more information via additional arguments:

calcite upload [options] <filePath>
    upload one dataset to a calcite server, by specifiying the config file, the gitRef and the commit id.

    -u, --url <server url>     Set the target server url, for preview and enterprise users. The `CALCITE_URL` environment variable can also be used, otherwise it will be set to the public server.
    -t, --token <token id>     Set the project token. By default the `CALCITE_TOKEN` environment variable is used.
    -C, --commit <sha>         Specify the full git commit sha.
    -R, --ref <gitref>         Specify the git ref.
    -BC, --benchconfig <sha>   Specify the benchmark config name.
    -BID, --buildId <id>       Specify the CI build number.
    -PR, --pullRequestId <id>  Specify the pull request number.
    --dry-run                  Do not upload, instead print the content of the payload to the console.
    -h, --help                 display help for command

Dataset formats
===============

To understand why the data is organized the following way, please have a look at the :ref:`usage/concepts:Concepts` documentation.

calcite JSon schema
-------------------

calcite uses its own json schema to describe a dataset.
Ultimately, this is what will be sent to the server, but you would usually create this file using an :ref:`importer <usage/upload:Importing from other formats>`.

A lot of importers are being added, and will be open-sourced. Do not hesitate to contact us if you created a new one!

The schema is available :download:`here <calcite-upload-schema.json>`.

.. image:: json-schema.png

.. @TODO@ intégrer https://github.com/jlblcc/json-schema-viewer/blob/master/templates/basic.html





.. jsonschema-siliceum:: calcite-upload-schema.json

Example
    .. code-block:: json
    
        {
          "testSuites": [
            {
              "name": "someTestSuite",
              "tests": [
                {
                  "name": "someTestName",
                  "dataPoints": [
                    {
                      "displayName": "",
                      "aggregationPolicy": "avg",
                      "diffPolicy": "sub",
                      "regressionPolicy": "lessIsBetter",
                      "regressionArgument": 0,
                      "name": "duration",
                      "values": [0.11,0.128,0.091,0.089,0.102,0.098,0.102,0.101,0.099,0.107],
                      "unit": "s"
                    }
                  ]
                }
              ]
            }
          ]
        }

Importing from other formats
----------------------------

Importing from other formats is best handled by specifying a ``.js`` file (usually ``calcite.config.js``) instead of a ``.json`` file to the ``calcite upload`` command.
The cli expects the configuration file to export a function returning a :ref:`calcite .json <usage/upload:calcite JSon schema>` file.
This way you can write or reuse any kind of importer.

Example
    .. code-block:: bash

        npx calcite upload ./calcite.config.js

``calcite.config.js``
    .. code-block:: js

        // Require modules, create functions, anything you want here... 
        const getTestDatapoints = (testname) => {
            // ...
        }

        // Export the function to be called by the cli
        // It must return a valid calcite JSon object.
        module.exports = async () => (
            testSuites: [
                {
                    name: 'fake testSuite',
                    tests: [
                        {
                            name: 'sometest',
                            datapoints: getTestDatapoints('sometest')
                        }, {
                            name: 'othertest',
                            datapoints: getTestDatapoints('othertest')
                        },
                    ]
                },
            ]
        });




Available importers
^^^^^^^^^^^^^^^^^^^

The following importers are available:

.. @TODO@: Add links to files/repo/npm package

C++
"""

- Google benchmark (WIP)
- nanobench (WIP, need to upstream)

Rust
""""

- Official microbenchmarking library (WIP)

Load testing
""""""""""""

- JMeter (WIP)
- Gatling (WIP)
- NeoLoad (WIP)

Web
"""

- Lighthouse CI (WIP)


Importer user example
^^^^^^^^^^^^^^^^^^^^^

@TODO@ How to use importers ? Via npm ? Copy the file directly ?