# Cpp - Meta Stage for StdioTest on C++ code

Meta stage for creating compile and StdioTest stage for grading submissions against standard input/output using declared
test cases.

- Kind: Grading Stage
- Behaviour:
    - Construct a pipeline consist of compile stage and a stdiotest stage
    - Takes a list of input files and compiles to one output executable, then the submission is tested with the defined
      test cases.
    - See [`Compile`](./Compile.md) and [`StdioTest`](./StdioTest.md)
- Throws:
    - See [`Compile`](./Compile.md) and [`StdioTest`](./StdioTest.md)

## Config

```
cpp:
  input: [String]          # list of input files to be compiled
  output: String           # name of output executable
  testCases:
  - id: Int                # id for this case
    args: [String]?        # optional command line arguments
    stdin: String?         # optional standard input
    file_stdin: String?    # optional helper file for providing standard input
    expected: String?      # optional expected output of the case
    file_expected: String? # optional helper file for providing expected output
    visibility: Visiblity  # visibility of the result of this case
    score: Double?         # scores for passing this case, if any (if not specified, scoring is disabled for this test case)
    valgrind: Valgrind?    # test case-specific override for Valgrind [default: null (following the `valgrind` key)]
  diff_ignore_flags: [DiffIgnoreFlag]?  # list of diff ignore flags to be used for comparing standard output against expected output
```

- `Visiblity: ALWAYS_VISIBLE | ALWAYS_HIDDEN | VISIBLE_AFTER_GRADING | VISIBLE_AFTER_GRADING_IF_FAILED`
- `DiffIgnoreFlag: TRAILING_WHITESPACE | SPACE_CHANGE | ALL_SPACE | BLANK_LINES`
- `file_stdin` and `file_expected`: These files are expected to be within the `provided` directory. If these fields are
  set, specifying their in-file text counterpart will return error
- `score`: Caveat - If `valgrind.score` is set to `null` and `stdioTest.score` is set to `null`, the `Valgrind` stage
  will use the `score` value of the global `Valgrind` configuration. To disable this behavior, use `0` as the
  `stdioTest.score`.
- `Valgrind`: see the [config](Valgrind.md): of Valgrind stage

## Report

- See [`Compile`](./Compile.md) and [`StdioTest`](./StdioTest.md)

```
stdioTest:
  - file: String           # file to run
    id: Int                # id for this case
    args: String           # command line arguments concatenated
    stdin: [String]        # standard input, could be an empty list
    stdout: [String]       # the standard output of the submission upon execution
    stderr: [String]       # the standard errors generated during execution
    exitCode: Int          # the first non-zero exit code in the sequence of commands executed, or zero if no command 
                           # exited with a non-zero exit code
    expect: [String]       # expected output of the case
    visibility: Visiblity  # visibility of the result of this case
    diff: [String]         # the diff generated output between expect and stdout (in this order)
    hasTimedOut: Boolean   # whether the execution has timed out
    isCorrect: Boolean     # whether the case is considered correct
    score: Score?          # scores for passing this case and the total score
    exeExitCode: Int       # The exit code of the executable file
    diffExitCode: Int      # The exit code of diff
```

[Example](examples/stdio_report.yaml)

See [Scorable.md](../Scorable.md) for more information on `score`.