PICASSOS

PICASSOS - A Collaborative Project

D-RisQ was invited to participate in a collaborative project run by Ricardo called ‘Proving Integrity of Complex Automotive Systems of Systems’.  The consortium included Jaguar Land Rover, Johnson Matthey Battery System, York Metrics, Warwick Manufacturing Group, Coven-try and Oxford Universities.  It was on this project that Modelworks® was evolved and the foundations of Kapture were laid.  The project was used to trial analysis techniques in order to reduce the time and cost impact of poor requirements and design.  This was conducted in the context of the automotive safety standard ISO26262.  Other activities such as coding and test were outside the scope of the trials documented here.

Project Aim

The aim was to not only develop these technologies but to also independently measure how effective they were in terms of error detection and the time taken to undertake verification when compared to other techniques.  At this stage, Kapture was not available and requirements had to be translated written in a semi-formal manner that required some training.  Modelworks also required some small amount of manual intervention.  Engineers in 2 companies were trained how to use the tools which only took about half a day.

The Trial

Warwick Manufacturing Group used an electric vehicle charging systems being developed for Jaguar Land Rover as the trial; this had 6 major areas of functionality.  There were 7 trials run on the various parts of the software 48 errors were seeded into either the requirements or the Simulink/Stateflow design.  One company did all the trials while another only did one; hence a total of 7 trials.  York Metrics had set the measurement processes for the activities undertaken by the 3 sets of software engineers who were to work in the trial.  The 1st set used the traditional review based techniques, the second used Simulink Design Verifier, while the third used the D-RisQ technologies.  Time for the various parts of the verification process were measured, which, for the D-RisQ process, also included the translation of the requirements into the semi-formal representation.  York Metrics monitored the trials and collated results.  Note that D-RisQ personnel were not involved in the trials.

The Results

The graph below show the time results.  The time allocated for the trial ran out hence the example PP had no results for D-RisQ (it was subsequently all proven).  All 3 processes were able to detect all 48 seeded errors (not including PP), but the D-RisQ process detected an additional unknown 49th error.  It can be seen that there is a consistent 60-80% saving through the use of D-RisQ prototype tools over traditional techniques and a broad range of savings over Simulink Design Verifier.  Possibly the most interesting result was the repeat of TA independently done by the 2nd company as TA2 gave almost the same results.  [Perhaps another result of interest was the 4 occasions where Simulink Design Verifier gave no savings.]