Writing a Java Assignment Configuration
This is a quick-start guide to writing your own Java Assignment Configuration YAML. Note that this document is by no means comprehensive; The Grader is much more extensible than what is advertised here.
Template
A starting template can be found here. The file will contain a one-liner
description of which required option does. Alternatively, an unannotated version can be found
here.
The following sections will all use the above template to briefly explain the meaning of each configuration option. Always refer to configuration-specific documents for more details on how to use each stage!
Settings
The _settings block is used to configure global grading options.
_settings:
lang: java:11
use_template: FILENAMES
template:
- do_not_include.txt
use_skeleton: true
use_provided: true
stage_wait_duration_secs: 10
cpus: 1.0
mem_gb: 2.0
enable_features:
network: false
The most important configuration keys are explained below. More info about other configuration keys are available in Writing an Assignment Configuration.
lang
This key determines the programming language which this grading pipeline is targeted for.
Certain stages, such as Compile and StdioTest, contain custom language-specific logic. By setting this key, it allows those stages to adapt its execution for the specified language.
The currently supported languages are:
java:<java_version>Supports any Java version 7 or above
For Java versions 8 or below,
<java_version>can be the full version (8u282) or major version only (8)For Java versions 9 or above,
<java_version>can be the full version (11.0.10), major+minor version (11.0), or major version only (11)
Typical Config Structure
fileStructureValidation: {}
diffWithSkeleton: {}
compile:
# ...
stdioTest:
# ...
score:
# ...
To include more than one compilation stage, see Multiple Stages of Same Type.
Note
The Grader may perform some rudimentary reordering to ensure that no illogical pipelines are created. For example,
pipelines which execute stdioTest before compile may be reordered to ensure that compilation is always
performed before testing.
Pre-Grading Stages
Pre-Grading Stages are pipeline stages which are performed before compilation and/or test execution. These stages are generally language-agnostic in nature.
fileStructureValidation
File Structure Validation is a stage which checks whether files submitted by the students matches the expected list of files that students should submit. If the student submission does not contain certain files, or the student submission contains additional files, this stage will emit a warning for students.
Note that in order to use this stage, _settings.use_template must not be null.
diffWithSkeleton
Diff With Skeleton is a stage which checks whether students have inadvertently submitted certain files from the skeleton code. If there are any files in the student submission which are identical to those in the skeleton code, this stage will emit a warning for students.
Note that in order to use this stage, _settings.use_skeleton must be true.
Grading Stages
Grading Stages are pipeline stages which are performed to either prepare or execute grading tasks. Some of these stages have the additional ability to be score-generating, i.e. a score can be generated from the stage based on its correctness.
Note that the following sections are not an exhaustive list of all supported stages. Refer to this page for a comprehensive list of supported stages and their configuration.
Language-Agnostic
Compile
The Compile stage is a language-agnostic stage which can be used in any pipeline with a supported _settings.lang.
This allows you to only need to specify the input, output, and optionally the compilation flags to compile
file(s) using javac.
StdioTest
The Stdio Test stage is a language-agnostic stage which is used to perform testing using Standard I/O (i.e. stdout
and stderr) streams.
Java
JavaCompile
The JavaCompile stage is a stage which compiles class files specified in the stage to bytecodes.
javaCompile:
input: [PathSpec] # Input files and/or directories
classpath: "[String]?" # Classpath to pass to javac; Defaults to []
outDir: String? # Directory to output all compiled class files to; Defaults to `.`
flags: "[String]?" # Other options to provide to javac
experimental:optimize: Boolean? # Experimental: Optimize pipeline by creating as few pipeline stages as needed
additional_packages: "[String]?" # Additional packages to install as dependencies
PathSpec
PathSpec:
file: String?
dir: String?
Setting the value of either file or dir implies that the path you are specifying points to a file or a directory
respectively. Note that you should only specify one of them; Specifying both is an error.
You may use * as a wildcard in either keys.
In the context of this stage:
Specifying
filemeans to compile file(s) specified by the path.
For example, the following compiles the file in src/main/java/Main.java.
input:
file: src/main/java/Main.java
For example, the following compiles all files matching the wildcard *.java (the quotation marks are optional). Note
that this will not recursively descend into subdirectories.
input:
file: '*.java'
Specifying
dirmeans to compile all files with the file extension.javaunder the specified path (including subdirectories), treating the path as a directory.
For example, the following compiles all Java source files under src/main/java.
input:
dir: src/main/java
JUnit
The JUnit stage is a stage which compiles files containing JUnit test cases and executes them.
Note that this stage uses the JUnit Platform Runner provided by JUnit 5. While JUnit 5 test cases are natively supported, only part of JUnit 3 and JUnit 4’s feature set is supported.
Gradle
The Gradle stage is a stage which executes Gradle task(s) in a Gradle project.
Post-Grading Stages
Score
The Score stage performs aggregation on all score-generating stages preceding it.
Note
If this stage is not specified, an overall score will not be generated for submissions.