Helper function to loop through list of TestCase (testCases) to validate configuration and files Returns the first type of errors per test case that it finds