docker-runner/dev.ust.zinc.grader.pipeline.docker.copy/AbstractCopyStage

AbstractCopyStage

abstract class AbstractCopyStage(inType: KClass<out StageResult>, outType: KClass<out StageResult>, baseDistro: OSDockerfile, val fileToDest: Map<String, String> = mapOf("*" to ""), var hostPath: Path? = null, val mountPath: Path? = null) : DockerPipelineStage

Abstraction for all pipeline stages involving copy operations with volumes.

Parameters

inType

Input type for the stage.

outType

Output type for the stage.

baseDistro

OS Distribution image to use as the basis of the pipeline stage.

fileToDest

Map of files to copy, each filename can be mapped to a named destination subdirectory under '/out' (default to the root of EnvPath.COMPOSITE_OUT_PATH). Allows using '*' as Shell script expansions.

hostPath

Path to host to mount into the container for host-volume transfers.

mountPath

Path to host to mount into the container as /mnt (to mount additional files aside from hostPath)

Inheritors

CopyHostToVolume

CopyVolumeToHost

CopyVolumeToVolume

Constructors

AbstractCopyStage

constructor(inType: KClass<out StageResult>, outType: KClass<out StageResult>, baseDistro: OSDockerfile, fileToDest: Map<String, String> = mapOf("*" to ""), hostPath: Path? = null, mountPath: Path? = null)

Types

object Companion

Properties

protected lateinit var _runner: Runner

protected open val ~~allowNetwork~~: Boolean

Whether the container is allowed network access.

open val className: String

open val cmdLang: CmdLang

Command language used for constructing the command specification.

cmdOutputHandler

open val cmdOutputHandler: CmdOutputHandler

Handler describing how the stage performs necessary IO actions to retrieve stage execution outputs.

protected open override val cmdSpec: CmdBuilder

Command to execute during the pipeline stage.

open override val config: PipelineStage.EmptyConfig

val configKeySrc: String?

var container: DockerContainer?

protected val distro: BaseDockerfile

Base distro image for this pipeline stage.

protected lateinit var docker: DockerDaemon

Instance to the Docker daemon.

protected open val dockerfileSpec: DockerfileBuilder.() -> Unit

Additional specifications to add to the Dockerfile.

environmentVars

protected open val environmentVars: List<String>

Environment variables to provide to the runtime environment.

val CmdUtils.Var.exitCode: Int

Convenience property for retrieving exitcode of a variable.

protected val fileToDest: Map<String, String>

hasDynamicDockerfileSpec

protected open val hasDynamicDockerfileSpec: Boolean = false

Whether the dockerfileSpec contains directives which are evaluated at pipeline execution time.

protected var hasTimedOut: Boolean

protected var hostPath: Path?

Path to mount to the host system as part of the pipeline, if this stage accepts/emits a HostVolume.

protected val imageTag: String?

Image tag to use for caching pipeline stage images.

val inType: KClass<out StageResult>

private val isCacheImage: Boolean = true

Whether to cache this image in the local Docker registry.

protected val logger: Logger

var logVolume: DockerVolume?

protected open val mntMountMode: AccessMode

Mode to mount the mountPath as.

protected val mountPath: Path? = null

Path to mount to the host system for supplementary use, if any.

var outputLogger: ContainerExecLogger?

The logger output are stored in logs of outputLogger after stage execution

val outType: KClass<out StageResult>

protected var parserContext: ConfigParsable.Context?

val pipelinePath: String

requestedResources

protected open val requestedResources: EnvContext.RequestedSystemResources

The system resources which control what the container is allowed to access.

open override val runner: DockerRunner

val CmdUtils.Var.stderr: List<String>

Convenience property for retrieving stderr content of a variable line-by-line.

val CmdUtils.Var.stdout: List<String>

Convenience property for retrieving stdout content of a variable line-by-line.

open override val traceableState: AtomicReference<TraceableState<PipelineStage.ExecStateData, Unit>>

protected open val volMountMode: AccessMode

Mode to mount the in-place volume as.

protected open override val workingDir: String

Working directory of the container.

Functions

fun buildCmd(configure: CmdBuilder.() -> Unit): CmdBuilder

Builder method for CmdBuilder using the command language cmdLang of the stage.

fun buildCmdWith(cmd: CmdBuilder.() -> String): CmdBuilder

Builder method for CmdBuilder using the command language cmdLang of the stage.

configureContainer

protected open fun CreateContainerCmd.configureContainer(input: StageResult): CreateContainerCmd

Configures the container during execution of the pipeline.

contentOrDefault

protected fun <R> CmdUtils.Var.Result<R>.contentOrDefault(default: R): R

Returns the content of this result (stdout, stderr, or exitCode); Otherwise returns a default value generated by default.

copyFromContainer

protected suspend fun copyFromContainer(hostPath: Path)

Copies files from the container to the host using the Docker API.

copyToContainer

protected suspend fun copyToContainer(hostPath: Path)

Copies files from the host to the container using the Docker API.

protected suspend override fun execute(input: StageResult): StageResult

suspend fun execWithInput(input: StageResult): StageResult

private suspend fun getDockerfile(): Path

Base Dockerfile for all pipeline stage.

protected suspend fun getDockerImage(force: Boolean = false): DockerImage

Builds a Docker image based on the Dockerfile specification as defined by this stage, overwriting any previously cached image if exists.

private suspend fun getHostConfig(inVolume: DockerVolume?): Pair<HostConfig, DockerVolume?>

Determines the host configuration of the container.

installPackages

fun installPackages(additionalPackages: List<String> = emptyList(), vararg packages: String): DockerfileBuilder.() -> Unit

Dockerfile spec for installing packages

logContainerInfo

protected suspend fun logContainerInfo(image: DockerImage, hostConfig: HostConfig)

Logs the information used to create the container for this pipeline stage.

markStageAsTimedOut

protected fun markStageAsTimedOut()

protected open suspend override fun onExecComplete()

protected fun onExecFailed(tr: Throwable)

onExecFailedImpl

protected open override fun onExecFailedImpl(tr: Throwable)

onInitParserContext

protected open fun onInitParserContext(configContext: ConfigParsable.Context)

postExecCleanup

protected suspend fun postExecCleanup(startTime: Instant, success: Boolean)

Performs cleanup after the execution of the container.

putParserContext

fun putParserContext(configContext: ConfigParsable.Context)

fun putRunner(runner: Runner)

removeContainer

private suspend fun removeContainer()

Removes the container and temporary log volume created by this pipeline stage.

protected open suspend fun runContainer(input: StageResult, hostConfig: HostConfig, outVolume: DockerVolume?): DockerVolume?

Builds the Docker image backing this pipeline stage, and runs the container.

transformResult

private fun transformResult(outVolume: DockerVolume?): StageResult

Transforms the result of the stage into a StageResult.

fun Var(baseName: String): CmdUtils.Var

Variable builder inheriting the CmdLang of this stage.