Score

Stage which accumulates all scores from previous stages and emits its own “final score”.

  • Kind: Post-Grading Stage

  • Behavior: Accumulates all scores.

Config

score:
  normalizedTo: Double?   # The "maximum" score to normalize to. If null, use the original max value.
  minScore: Double?       # The lower bound of the score to clip to. If null, do not clip the score.
  maxScore: Double?       # The upper bound of the score to clip to. If null, do not clip the score.

  • normalizedTo refers to the maximum score to rescale the final score to

    • Usually used to normalize the score to 100-based.

Report

score:
  - score: Double         # Final score of the cumulated reports
    gradedTotal: Double   # Total score based on graded reports
    maxTotal: Double      # Total score based on all reports

The general score is computed by normalizing the score first, then clipping the scores within minScore and maxScore, seen as follows:

score.score = (pipelineStages.sum(score.score) / pipelineStages.sum(score.total) * normalizedTo).coerceIn(minScore, maxScore)

Note that the gradedTotal and maxTotal is affected by maxScore of the config, meaning that the total score(s) will also be clipped.

gradedTotal and maxTotal

The difference between gradedTotal and maxTotal is that gradedTotal accumulates the scores of all pipeline stages which are successfully graded, whereas maxTotal accumulates the scores of all pipeline stages, regardless of whether the grading is successful.

For example, consider a pipeline stage which uses unit tests for grading.

  • If the stage successfully executes

    • Stage will emit Score(score = 10.0, total = 10.0)

    • Score will emit Report(score = 10.0, gradedTotal = 10.0, maxTotal = 10.0)

  • If the stage executes with some test cases failing

    • Stage will emit Score(score = 5.0, total = 10.0)

    • Score will emit Report(score = 5.0, gradedTotal = 10.0, maxTotal = 10.0)

  • If the stage executes, but after execution the unit test report cannot be found by the Grader

    • Stage will emit Score(score = null, total = 10.0)

    • Score will emit Report(score = 0.0, gradedTotal = 0.0, maxTotal = 10.0)