Ensuring consistency of safety documentation

To ensure that different team (System, SW and HW) aligns to the same set of safety mechanisms (SM), Novotis uses a SM list. The SM list provides information about each safety mechanism is use, a unique identifier, the type of fault it addresses, and the design and diagnostic coverage.

In a typical scenario the safety analysis may start with a simple high-level requirements like “do no cause unwanted actuation”. Based on an initial block diagram of the architecture the safety analysis identifies potential failures that could violate higher level safety requirements, (1) in the picture.

Needed safety mechanisms to detect and handle the faults are selected and added to the SM list (2). Each SM is given an identifier, e.g. SM02. The diagnostic coverage from the SM list will be used by the safety analysis (3). By entering a reference to the SM in each analysis consistency between the different teams are ensured.

Once the iteration around architecture – analysis – needed SM is finished requirements on SM can be formulated (4) and related test cases defined (4).

This will ensure a consistent and complete set of safety requirements even in the case of few and high level initial requirements.

ISO 26262 – How To Training

Development teams have been asking for ISO 26262 training that focus more on how to develop a safe system. To meet this demand Novotis has developed a training with a heavy focus on practical exercices around the topics many teams find challenging:

  • Safety analysis (HW, SW, System)
  • Defining safety requirements
  • Designing safety test cases

This training has been given both at Tier 1 and OEM. It can be tailored to company specific needs by changing the the exercise modules included in the 2-day training.

The training assumes a basic understanding of ISO 26262, for instance through Novotis 2-day introduction to ISO 26262.

Novotis focus is in-house training tailored to customer needs. Read more about our training here or contact us to discuss training.

Defining an ISO 26262 process

Before defining a SPICE or ISO 26262 process, consider the the role of the processes in the architecture of the quality system. A useful quality system will consist of several items that will work together to provide rules and guidance for what to do. You should consider developing these items:

  • Process overview
  • Individual processes
  • Guided templates
  • Guidelines
  • Good examples

The process overview should provide and overview of all process, work product, roles and responsibilities. The overview should also define lifecycle and tailoring rules.

Processes should not be written to assume a certain lifecycle. Especially the engineering processes should be written to define activities need to achieve the expected outputs (work products) from defined inputs. If the process is applied in a waterfall approach or an agile (highly iterative) approach should be dealt with in the lifecycle part of the quality system.

Each process should have the following elements:

  • Process objectives
  • Input work products
  • Key activities
  • Output work products with responsibilities
  • Guidance
  • Training material

Novotis defines process objectives as a set of review criteria for the output work products. In this way the SPICE requirements for both process objectives and review criteria can be satisfied in a simple manner. These high level review criteria (5 – 10 points) can be replaced by more detailed checklists as needed.

The input and output work products provide the links between processes. A process should refine one or more key work products into new work products. Novotis tend to focus on the key work products of a process, e.g. requirements specification, design specification or test specification / report.

The list of work products in SPICE is very ambitious, Novotis have never seen any company use all these work products. The list of work products in ISO 26262 is more realistic but may not match what a SPICE compliant company already has in place.

Novotis recommends that before starting with writing the process, the company reviews the process and work product it is using (will use) and adapt the work product and process structure to fit company needs.

Novotis typically focus responsibilities on the work products, i.e. responsibility for creating, approving and reviewing work products.

Most of the variation in the processes are in the description of key activities. As the quality system will have guided templates Novotis believes it is most appropriate to have an activity focus in a process. At the same time a process may be used in different life cycles and therefore a flow chart type process may be too restrictive for most process, exceptions would be change control and problem resolution management.

Process should define the minimum needed activities to ensure quality outputs. Process should also be usable as a basis for quality assurance. This normally requires more activities than practices defined in SPICE. To cover SPICE and ISO 26262 requirements would require 20 – 30 activities in a process like requirements analysis.

Process should not be seen as training material for new developers. For that we may have guidance and training material. These can be more focused on ”how to” do while the process should focus more on ”what to do”.

Novotis normally uses a shorter guidance section in process where limited guidance (1 – 2 pages) is needed and will create a separate guideline for processes where more extended guidance is needed.

How to improve processes

When improving processes many companies start with the audit or assessment report and try to adress the weaknesses. That is a good approach in a mature company with a small gap to their target.

If the gap is larger the approach does not work well, try this approach instead:

Gather input on problems and improvement opportunities
Use multiple sources as input

  • Internal audits and assessments
  • External audits and assessments
  • Customer feedback
  • Problems databases
  • Objective metrics
  • Interviews
  • Project close down reviews

Different stakeholders can provide different perspectives and typically have different views. Make sure input from all relevant stakeholders is used.

Group related issues and improvement opportunities
Group related problems and improvement opportunities to possible long-term improvement projects. Projects should be roughly on the process level.
For example if there are many issues related to requirements changes and testing of requirements a first project could be to address “requirements analysis”. This groups related problems together and addresses a probable root cause more than a symptom.

Prioritize improvemenet projects
When prioritizing between different immprovement projects, consider:

  • Work from left to right in the engineering V-model
  • Work from top to bottom
  • Long lead time tasks first (typically tool based methods and changes requiring reorganization)
  • Time constraints of target projects

You can typically only handle 2 – 3 major improvement projects in paralell. Do not fall into teh trap of trying to everything at once.

Adress root causes and not the symptoms
In many cases your inputs to process improvement (like customer feedback or audit reports) are mostly symptoms. The best improvement projects typically adresses the root causes instead of symptoms. Use the “Why, why, why, why, why” approach to get down to root causes. Do not be afraid to challenge current approaches in this phase.

Identify short term tasks
Define the actions needed to establish the improved process / method. Actions should create the necessary output. This may include

  • Tools and methods
  • Templates
  • Guidelines
  • Processes
  • Training material
  • Good examples
  • Roles and responsibilities
  • Organizational proposals

Iteratively develop methods and QS assets
Typically use an iterative process in a small working group or a pilot project to develop the necessary methods and QS assets.

 

Unit testing in SPICE and ISO 26262

This is the first of a couple of posts dealing with testin in Automotive SPICE and ISO 26262 projects. In this post we will focus on unit testing.

Looking in the Automotive SPICE Assessment Model (PAM) we get only vague guidance on what is expected.

In the notes to this base practice it is stated that verification criteria must include test specification including unit test cases (if required by contract) and that coverage goals should be defined.

As an assessor I see three different approaches. All of which I regard as valid although they have different strengths and weaknesses. See the table below:

Test_SWOT

 

 

 

 

ISO 26262 is much more specific.

It states that requirements based and test should be performed (Part 6, table 10) and that the analysis for deriving test cases should cover requirements, equivalence classes and boundary values (table 11).

The standard further states that a purpose of the coverage criteria are to demonstrate that there is no unintended functionality (para 9.4.5).

The coverage criteria are:

– ASIL A: Statement coverage
– ASIL B – C: Branch coverage
– ASIL D: Modified condition / decision coverage

From the above there are some implications. First unit testing of safety mechanisms should be black box (on the unit level) and based on requirements. Secondly the coverage requirement will lead to longer test vectors especially for ASIL D, see the example below.

Test vector

 

 

 

 

 

 

 

 

Can you have the same unit test approach for a SPICE and ISO 26262 project? Yes, definetely but that would mean more effort on the SPICE “side” than what is typical in a non-safety project.

To get good coverage when black box testing units you should aim for a cohesive design. Each unit has a well defined task defined by one or a few requirements.

If your architectural pattern forces a separation of a safety mechanism into parts like:
– fault detection
– fault handing
– safety timing (typically on integration level)
You should consider splitting requirements into separate parts.

What is your experience with unit testing in ISO 26262. What questions are you facing?