Writing a C++ Assignment Configuration ====================================== This is a quick-start guide to writing your own C++ Assignment Configuration YAML. Note that this document is by no means comprehensive; The Grader is much more extensible than what is advertised here. Template -------- A starting template can be found :download:`here `. The file will contain a one-liner description of which required option does. Alternatively, an unannotated version can be found :download:`here `. The following sections will all use the above template to briefly explain the meaning of each configuration option. Always refer to configuration-specific documents for more details on how to use each stage! Settings -------- :doc:`Full Documentation Here `. The ``_settings`` block is used to configure global grading options. .. code-block:: yaml _settings: lang: cpp/g++:8 use_template: FILENAMES template: - do_not_include.txt use_skeleton: true use_provided: true stage_wait_duration_secs: 10 cpus: 1.0 mem_gb: 2.0 enable_features: network: false Available languages and compilers are explained below. More info about other configuration keys are available in :doc:`Writing an Assignment Configuration`. ``lang`` ^^^^^^^^ This key determines the programming language which this grading pipeline is targeted for. Certain stages, such as :doc:`pipeline/docker/Compile` and :doc:`pipeline/docker/StdioTest`, contain custom language-specific logic. By setting this key, it allows those stages to adapt its execution for the specified language. The currently supported combinations are: - ``cpp/g++:`` - Supports any GCC version 8 or above - ```` can be the full version (``9.3.0``), major+minor version (``9.3``), or major version only (``9``) - ``cpp/clang:`` - Supports any clang version 8 or above Suggested Structure ------------------- Here is a minimum suggested structure for setting up the pipeline to grade a C++ assignment: .. code-block:: yaml fileStructureValidation: {} diffWithSkeleton: {} compile: # ... stdioTest: # ... score: # ... To include more than one compilation stage, see :ref:`multiple-stage`. Pre-Grading Stages ------------------ Pre-Grading Stages are pipeline stages which are performed before compilation and/or test execution. These stages are generally language-agnostic in nature. ``fileStructureValidation`` ^^^^^^^^^^^^^^^^^^^^^^^^^^^ :doc:`Full Documentation Here `. File Structure Validation is a stage which checks whether files submitted by the students matches the expected list of files that students should submit. If the student submission does not contain certain files, or the student submission contains additional files, this stage will emit a warning for students. Note that in order to use this stage, ``_settings.use_template`` must not be ``null``. ``diffWithSkeleton`` ^^^^^^^^^^^^^^^^^^^^ :doc:`Full Documentation Here `. Diff With Skeleton is a stage which checks whether students have inadvertently submitted certain files from the skeleton code. If there are any files in the student submission which are identical to those in the skeleton code, this stage will emit a warning for students. Note that in order to use this stage, ``_settings.use_skeleton`` must be ``true``. Grading Stages -------------- Grading Stages are pipeline stages which are performed to either prepare or execute grading tasks. Some of these stages have the additional ability to be score-generating, i.e. a score can be generated from the stage based on its correctness. Note that the following sections are not an exhaustive list of all supported stages. Refer to :doc:`this page ` for a comprehensive list of supported stages and their configuration. Necessary/Recommended ^^^^^^^^^^^^^^^^^^^^^ ``Compile`` """"""""""" :doc:`Full Documentation Here `. The Compile stage is a language-agnostic stage which can be used in any pipeline with a supported ``_settings.lang``. This allows you to only need to specify the ``input``, ``output``, and optionally the compilation flags to compile file(s) using the compiler. Here are the default flags that are used for C++: ["-std=c++11", "-pedantic", "-Wall", "-Wextra", "-g"] .. code-block:: yaml compile: input: "[String]" # full set of files needed for compilation output: String? # name of the compiled target executable flags: "[String]?" # flags used for compilation. These will be fed into specific compilation commands of the language. [default: ] additional_packages: "[String]?" # any additional package to install as dependency to the container [default: []] ``StdioTest`` """"""""""""" :doc:`Full Documentation Here `. The Stdio Test stage is a language-agnostic stage which is used to perform testing using Standard I/O (i.e. ``stdout`` and ``stderr``) streams. .. code-block:: yaml stdioTest: testCases: - file: String # file to run id: Int # id for this case args: "[String]?" # optional command line arguments stdin: String? # optional standard input file_stdin: String? # optional helper file for providing standard input expected: String? # optional expected output of the case file_expected: String? # optional helper file for providing expected output visibility: Visiblity # visibility of the result of this case score: Double? # scores for passing this case, if any (if not specified, scoring is disabled for this test case) valgrind: Valgrind? # test case-specific override for Valgrind [default: null (following the `valgrind` key)] diff_ignore_flags: "[DiffIgnoreFlag]?" # list of diff ignore flags to be used for comparing standard output against expected output C++ ^^^ ``Make`` """""""" :doc:`Full Documentation Here `. Ths Make stage is a stage which invokes GNU Make on ``Makefile`` for easier compilation. ``Valgrind`` """""""""""" :doc:`Full Documentation Here `. The Valgrind stage is a stage which invokes Valgrind on an executable. Note that the specification for Valgrind does not contain individual test cases. Instead, the Valgrind configuration is designed to be applied to all test cases within all ``stdioTest`` blocks. Fine-grained overrides can be configured using the ``stdioTest.testCases.valgrind`` key. ``GTest`` """"""""" :doc:`Full Documentation Here `. The GTest stage is a stage which compiles files containing GTest test cases and executes them. Post-Grading Stages ------------------- ``Score`` ^^^^^^^^^ :doc:`Full Documentation Here `. The Score stage performs aggregation on all score-generating stages preceding it. .. note:: If this stage is not specified, an overall score will not be generated for submissions.