Building a safe and secure embedded world Software Unit Verification in IEC 62304 Frank Büchner, Hitex GmbH, Karlsruhe
Hitex GmbH Founded 1976 in Karlsruhe, Germany Approx. 50 employees Subsidiary in UK (20 employees) Part of the Infineon Group since 2003 Tools for safety & security Test services Engineering, production, consulting AURIX preferred design house (PDH) Software Unit Verification, Frank Büchner, Nov. 2018 Copyright Hitex GmbH 2018. All rights reserved. 2
Motivation Inspiration by a look in non-medical standards IEC 61508 ISO 26262 DIN EN 50128 ISO 14971 ISO 13485 IEC 60601-1 IEC 61010-1 ISO/IEC 12207 Software Unit Verification, Frank Büchner, Nov. 2018 Copyright Hitex GmbH 2018. All rights reserved. 3
Contents 1. A Look in 62304 2. A Look in Other Standards 2.1 26262 and Code Coverage 2.2 50128 and Independent Testing 2.3 61508 and Software Complexity Control 2.4 26262 and Test Case Specification 2.5 26262 and Tool Qualification Software Unit Verification, Frank Büchner, Nov. 2018 Copyright Hitex GmbH 2018. All rights reserved. 4
A Look in 62304 Software Unit Verification IEC 62304:2006+AMD1:2015, p. 24 (Copyright der VDE VERLAG GmbH) DIN EN 62304:2018-06, p. 28 (Copyright der VDE VERLAG GmbH) Software Unit Verification, Frank Büchner, Nov. 2018 Copyright Hitex GmbH 2018. All rights reserved. 5
A Look in 62304 What is an unit? Three criteria 1. Not subdivided / not further decomposed 2. Separately testable 3. Defined by manufacturer Software item Software Unit Verification, Frank Büchner, Nov. 2018 Copyright Hitex GmbH 2018. All rights reserved. 6
A Look in 62304 What is an unit? Software Items: U U U Software System U U U Software Item U Software Unit Software Unit Verification, Frank Büchner, Nov. 2018 Copyright Hitex GmbH 2018. All rights reserved. 7
Conclusion What is an unit? Programming language C C++, Java, C#, Ada Unit Function Method Procedure / Function Software Unit Verification, Frank Büchner, Nov. 2018 Copyright Hitex GmbH 2018. All rights reserved. 8 Second term: verification
A Look in 62304 Verification IEC 62304:2006+AMD1:2015, p. 14 Software Unit Verification, Frank Büchner, Nov. 2018 Copyright Hitex GmbH 2018. All rights reserved. 9
A Look in 62304 Verification Where do the requirements come from? IEC 62304:2006+AMD1:2015, p. 23 Includes Requirements Decomposition + Risk Analysis Software Unit Verification, Frank Büchner, Nov. 2018 Copyright Hitex GmbH 2018. All rights reserved. 10
A Look in 62304 Verification Strategies, methods, and procedures IEC 62304:2006+AMD1:2015, p. 24 Software Unit Verification, Frank Büchner, Nov. 2018 Copyright Hitex GmbH 2018. All rights reserved. 11
Excursus Test static dynamic manual (by human) automated (by tool) automated (by tool) Software Unit Verification, Frank Büchner, Nov. 2018 Copyright Hitex GmbH 2018. All rights reserved. 12
A Look in 62304 Verification Acceptance Criteria IEC 62304:2006+AMD1:2015, p. 19 Software Unit Acceptance Criteria IEC 62304:2006+AMD1:2015, 5.5.3, p. 24 Software Unit Verification, Frank Büchner, Nov. 2018 Copyright Hitex GmbH 2018. All rights reserved. 13
Discussion Acceptance Criteria Requirements Link Matrix Software Unit Verification, Frank Büchner, Nov. 2018 Copyright Hitex GmbH 2018. All rights reserved. 14
Discussion Acceptance Criteria Interface Structure Software Unit Verification, Frank Büchner, Nov. 2018 Copyright Hitex GmbH 2018. All rights reserved. 15
Discussion Acceptance Criteria Coding standards (1/2) IEC 62304:2006+AMD1:2015, p. 49 Software Unit Verification, Frank Büchner, Nov. 2018 Copyright Hitex GmbH 2018. All rights reserved. 16
Discussion Acceptance Criteria Coding standards (2/2) Proprietary coding rules Ready-made, e.g. MISRA, CERT-C Checked by static analysis Preferrably checked by tool Software Unit Verification, Frank Büchner, Nov. 2018 Copyright Hitex GmbH 2018. All rights reserved. 17
A Look in 62304 Additional acceptance criteria IEC 62304:2006+AMD1:2015, p. 24 Software Unit Verification, Frank Büchner, Nov. 2018 Copyright Hitex GmbH 2018. All rights reserved. 18
Discussion Additional Acceptance Criteria Proper Event Sequence E.g. by checking the Call Trace Software Unit Verification, Frank Büchner, Nov. 2018 Copyright Hitex GmbH 2018. All rights reserved. 19
Discussion Additional Acceptance Criteria Data flow A variable can have 3 states: 1. d: defined (= value assigned) 2. r: referenced (= value used) 3. u: undefined (= not initialized) Three data flow anomalies: 1. ur 2. du 3. dd A data flow anomaly does not need to result in a failure Software Unit Verification, Frank Büchner, Nov. 2018 Copyright Hitex GmbH 2018. All rights reserved. 20
Discussion Additional Acceptance Criteria Control flow Example: Unreachable code Software Unit Verification, Frank Büchner, Nov. 2018 Copyright Hitex GmbH 2018. All rights reserved. 21
Discussion Additional Acceptance Criteria Fault handling Needs requirement Initialization of variables This is a data flow anomaly Self-diagnostic Needs requirement Boundary conditions Relates to test case specification [ [ Software Unit Verification, Frank Büchner, Nov. 2018 Copyright Hitex GmbH 2018. All rights reserved. 22
Contents 1. A Look in 62304 2. A Look in Other Standards 2.1 26262 and Code Coverage 2.2 50128 and Independent Testing 2.3 61508 and Software Complexity Control 2.4 26262 and Test Case Specification 2.5 26262 and Tool Qualification Software Unit Verification, Frank Büchner, Nov. 2018 Copyright Hitex GmbH 2018. All rights reserved. 23
26262 and Code Coverage 62304 mentions Coverage of requirements But not code coverage for unit verification Code coverage in ISO 26262:2011 for Unit Testing Part 6, Table 12 Software Unit Verification, Frank Büchner, Nov. 2018 Copyright Hitex GmbH 2018. All rights reserved. 24
26262 and Code Coverage Recommendation Safety Class 62304 A B C Coverage Measure Statement Coverage Branch Coverage Modified Condition / Decision Coverage (MC/DC) Software Unit Verification, Frank Büchner, Nov. 2018 Copyright Hitex GmbH 2018. All rights reserved. 25
Inhalt 1. A Look in 62304 2. A Look in Other Standards 2.1 26262 and Code Coverage 2.2 50128 and Independent Testing 2.3 61508 and Software Complexity Control 2.4 26262 and Test Case Specification 2.5 26262 and Tool Qualification Software Unit Verification, Frank Büchner, Nov. 2018 Copyright Hitex GmbH 2018. All rights reserved. 26
50128 and Independent Testing 62304 IEC 62304:2006+AMD1:2015, p. 8 IEC 62304:2006+AMD1:2015, p. 64 Software Unit Verification, Frank Büchner, Nov. 2018 Copyright Hitex GmbH 2018. All rights reserved. 27
50128 and Independent Testing 50128 Bahnanwendungen / Railway DIN EN 50128:2012-03 (Copyright der VDE VERLAG GmbH) At SIL 0: A person, who is implementer of a software component must not be tester of the same software component. (translation by the author) Software Unit Verification, Frank Büchner, Nov. 2018 Copyright Hitex GmbH 2018. All rights reserved. 28
Independent Testing Why is independent testing important? (1/3) Example of a specification A start value and a length define a range of values. Determine if a given value is within the defined range or not. The end of the range shall not be inside the range. Only integer numbers are to be considered. value outside inside outside start [ [ length Software Unit Verification, Frank Büchner, Nov. 2018 Copyright Hitex GmbH 2018. All rights reserved. 29
Independent Testing Why is independent testing important? (2/3) This case is simple 5 6 7 [ [ Value = 6 inside! Start = 5 Length= 2 Software Unit Verification, Frank Büchner, Nov. 2018 Copyright Hitex GmbH 2018. All rights reserved. 30
Independent Testing Why is independent testing important? (3/3) But this case? Value = -6??? -7-6 -5 ] ] Length = -2 Start = -5 Software Unit Verification, Frank Büchner, Nov. 2018 Copyright Hitex GmbH 2018. All rights reserved. 31
Contents 1. A Look in 62304 2. A Look in Other Standards 2.1 26262 and Code Coverage 2.2 50128 and Independent Testing 2.3 61508 and Software Complexity Control 2.4 26262 and Test Case Specification 2.5 26262 and Tool Qualification Software Unit Verification, Frank Büchner, Nov. 2018 Copyright Hitex GmbH 2018. All rights reserved. 32
61508 and Software Complexity Control IEC 61508 IEC 61508-3:2010, Table B.9 Software Unit Verification, Frank Büchner, Nov. 2018 Copyright Hitex GmbH 2018. All rights reserved. 33
61508 and Software Complexity Control Metrics for software complexity control Examples Cyclomatic complexity according to McCABE Volume according to Halstaed Tool support necessary! Software Unit Verification, Frank Büchner, Nov. 2018 Copyright Hitex GmbH 2018. All rights reserved. 34
61508 and Software Complexity Control Software module size limit 61508-7, section C.2.9: typically 2 to 4 screen sizes Metric Lines-of-code (LOC) Software Unit Verification, Frank Büchner, Nov. 2018 Copyright Hitex GmbH 2018. All rights reserved. 35
61508 and Software Complexity Control Parameter number limit Software Unit Verification, Frank Büchner, Nov. 2018 Copyright Hitex GmbH 2018. All rights reserved. 36
61508 and Software Complexity Control One entry / one exit Rule 15.5 from MISRA-C:2012 Only one return statement at the end Software Unit Verification, Frank Büchner, Nov. 2018 Copyright Hitex GmbH 2018. All rights reserved. 37
Contents 1. A Look in 62304 2. A Look in Other Standards 2.1 26262 and Code Coverage 2.2 50128 and Independent Testing 2.3 61508 and Software Complexity Control 2.4 26262 and Test Case Specification 2.5 26262 and Tool Qualification Software Unit Verification, Frank Büchner, Nov. 2018 Copyright Hitex GmbH 2018. All rights reserved. 38
26262 and Test Case Specification How to find test cases for black-box unit tests? ISO 26262:2011, part 6, table 11 Software Unit Verification, Frank Büchner, Nov. 2018 Copyright Hitex GmbH 2018. All rights reserved. 39
26262 and Test Case Specification Methods from ISO 26262 for deriving test cases Equivalence classes Software Unit Verification, Frank Büchner, Nov. 2018 Copyright Hitex GmbH 2018. All rights reserved. 40
Excursus Test case specification using the Classification Tree Method Software Unit Verification, Frank Büchner, Nov. 2018 Copyright Hitex GmbH 2018. All rights reserved. 41
26262 and Test Case Specification Methods from ISO 26262 for deriving test cases Error guessing aka intuitive testing aka experienced-based testing Software Unit Verification, Frank Büchner, Nov. 2018 Copyright Hitex GmbH 2018. All rights reserved. 42
Contents 1. A Look in 62304 2. A Look in Other Standards 2.1 26262 and Code Coverage 2.2 50128 and Independent Testing 2.3 61508 and Software Complexity Control 2.4 26262 and Test Case Specification 2.5 26262 and Tool Qualification Software Unit Verification, Frank Büchner, Nov. 2018 Copyright Hitex GmbH 2018. All rights reserved. 43
26262 and Tool Qualification Methods for tool qualification ISO 26262:2011, part 8, table 4 Software Unit Verification, Frank Büchner, Nov. 2018 Copyright Hitex GmbH 2018. All rights reserved. 44
Thank you for listening! Any questions? Unit test tool TESSY by Razorcat www.hitex.de/tessy Static analysis tool KLOCWORK by Roguewave www.hitex.de/klocwork Software Unit Verification, Frank Büchner, Nov. 2018 Copyright Hitex GmbH 2018. All rights reserved. 45
Contact & Additional Information Frank Büchner Dipl.-Inform. Principal Engineer Software Quality Hitex GmbH Greschbachstr. 12 Karlsruhe Germany Tel.: +49 / 721 / 9628 125 frank.buechner(at)hitex.de Software Unit Verification, Frank Büchner, Nov. 2018 Copyright Hitex GmbH 2018. All rights reserved. 46