# Valgrind

Stage for memory checking via Valgrind.

- Kind: Grading Stage
- Required stages: `stdioTest`
- Behavior: Runs Valgrind memory check over all test cases list in `stdioTest`
- Throws:
  - [`FieldsIncorrectlyUsedError`](../../model/Config.md) if `_settings.lang` does not correspond to a distro

## Config

### Global Configuration

For configuring Valgrind globally (i.e. for all test cases):

```
valgrind:
  enabled: Boolean?               # explicit override of using Valgrind or not (especially useful in StdioTest.Config) [default: true]
  args: [String]?                 # Valgrind command line options [default: []]
  checksFilter: [String]?         # list of filters against the kind of Valgrind errors [default: ["*"]]
  visibility: Visibility?         # specifies how this test case should be displayed in report rendering. INHERIT means taken from the visibility of the same test case in StdioTest [default: INHERIT]
  compat:noMatchStdout: Boolean   # whether to compare the `stdout` of the executable when determining the correctness of the test case
  score: Double?                  # Score of this stage. If not defined, uses stdioTest.score first, then cascades to global 
                                  # valgrind.score if still null.
                                  # 0.0 may be used to indicate that this stage should not contribute to the final score.
```

- `Visiblity: ALWAYS_VISIBLE | ALWAYS_HIDDEN | VISIBLE_AFTER_GRADING | VISIBLE_AFTER_GRADING_IF_FAILED | INHERIT`

- `checksFilter` takes values: `*`, `Leak_*`, `Uninit*`, `*Free`
  - which maybe mapped to just 'all', 'leak', 'uninitialized', 'free' etc. in the UI
  - there may be hints in the ui to explain what these kinds/classes of bugs mean. e.g.
    - "Leak-class of bugs catches memory leaks within the application"
    - "Uninit-class of bugs catches usages of variables which were not initialized"
- `compat:noMatchStdout`: If set to true, the `stdout` of the executable (within Valgrind) will **not** be used in 
  determining the correctness of the test case.
  - In other words, the score will be solely determined by whether the executable emits errors in Valgrind.
  - The expected output and `diff` flags will be inherited from the corresponding `stdioTest` test case.
  - **Including `stdout` for determining the correctness of the test case will become the default in the next release of
    the ZINC Grader. To opt out of this behavior, set this field to `true`.**

### Local Configuration

For fine-grained override/control in each `testCase` **given that `valgrind` is defined at the top-level**:

```
stdioTest:
  testCases:
    # ...
    valgrind: Valgrind?    # that is, the above config is nested under the `valgrind` key
```

### Example

```yaml
stdioTest:
  testCases:
    - file: a.out
      id: 1
      args: [1]
      stdin: ~
      expected: ~
      visibility: ALWAYS_VISIBLE
      score: 50
      valgrind:
        enabled: false  # Disables valgrind for this case
    - file: a.out
      id: 2
      args: [2]
      expected: ~
      visibility: ALWAYS_HIDDEN
      score: 50
      valgrind:
        visibility: INHERIT # Enables valgrind for this case since this case is ALWAYS_VISIBLE
        score: 50           # Sets the score of Valgrind to 50
    - file: a.out
      id: 3
      args: [3]
      expected: ~
      visibility: ALWAYS_HIDDEN
      score: 50
      valgrind:
        visibility: INHERIT # Enables valgrind for this case since this case is ALWAYS_VISIBLE; 
                            # Score is inherited from the test case, so it is 50.
```

## Report

TODO
```yaml
stageReports:
  valgrind:
    - executable: String     # executable for this test case
      id: Int                # test case id
      args: [String]         # command line arguments of the test case
      isSuccess: Boolean     # Whether this stage executed without errors
      hasTimedOut: Boolean   # whether the execution has timed out
      exitCode: Int          # Valgrind exit code
      stdout: [String]       # All standard output messages from the executable (Valgrind does not output to stdout)
      stderr: [String]       # All standard error messages from Valgrind and the executable
      errors:                # list of valgrind errors
        - kind: String       # kind of error
          what:              # list of bugs as a trace: from call site to bottom-most (system) function
          - whatText: String # bug description
            stack:           # call stack of this bug
            - ip: Int        # instruction pointer
              obj: String?   # object file
              fn: String?    # function called
              dir: String?   # parent directory of the object file
              file: String?  # involved source file ('culprit')
              line: Int?     # line number in source file
            aux: Boolean     # indicates whether or not this error is primary: the root error will have aux=false
      visibility: Visibility # specifies how this test case should be displayed in report rendering after resolution
      exeExitCode: Int       # the exit code from Valgrind
      diffExitCode: Int?     # the exit code from diff, if `compat:noMatchStdout` is set to `false`
      score: Score?          # Score of this stage.
```

[Example](examples/valgrind_report.yaml)

- `kind` can take values from [this list](https://sourceware.org/git/?p=valgrind.git;a=blob;f=docs/internals/xml-output-protocol4.txt;h=3703bbd0250b806ba90c387c34a5260be6903719;hb=608cb11914e5f23d0fc12c61dad29c5c7952a1de#l494)
- `aux`'s [explanation](https://sourceware.org/git/?p=valgrind.git;a=blob;f=docs/internals/xml-output-protocol4.txt;h=3703bbd0250b806ba90c387c34a5260be6903719;hb=608cb11914e5f23d0fc12c61dad29c5c7952a1de#l448)
- See [Scorable.md](../Scorable.md) for more information on `score`.