ISO 17025 - Right for Digital Forensics?

 


ISO 17025 is a mandatory standard for Digital Forensics laboratories in the United Kingdom (UK) as of October 2017.

All labs that are not ISO 17025 certified must disclose their ‘non-compliance’ on every report produced.

The following article is meant to provide information and open the discussion around this topic. 

This ISO 17025 accreditation will impact how Digital Forensic examinations are conducted in the future and by whom around the globe.

As it stands, views are mixed about the suitability of this standard for Digital Forensics. 

Certainly, some Digital Forensic Examiners (DFE’s) believe that using ISO 17025 for Digital Forensics is like placing a square peg into a round hole.

Is this belief based on fact or fear?

ISO 17025

ISO 17025 is also referred to as ISO/IEC 17025.

ISO – International Organization for Standardization
IEC – International Electrotechnical Commission

 

What is ISO 17025?

ISO 17025 was first published in 1999 to standardize labs around the world to ensure results from one lab would be accepted or repeated by other standardized labs.

This helps to break down international borders between countries when sharing forensic lab results.

To become ISO 17025 accredited, nationally recognized laboratory accreditation bodies assess the labs for conformity. 

These accreditation bodies must follow established “methods of evaluation” developed by the International Laboratory Accreditation Cooperation (ILAC).

The ISO/IEC 17025 standard is split into 5 distinctive categories:

  1. Scope
  2. Normative Resources
  3. Terms and Definitions
  4. Management Requirements
  5. Technical Requirements

Some areas that may be addressed within the above 5 categories include:

  • Testing and Calibration Standards
  • Staff Competence
  • Equipment Standards
  • Quality Management

The goal of ISO 17025 accreditation is to "consistently produce valid results"

Source: Wikipedia ISO/IEC 17025

Although ISO 17025 was written for testing and calibrating laboratories, many believe that it is the best fit for Digital Forensic Laboratories simply because no other international standard for digital forensics currently exists. 

This has been discussed in many articles including a conference paper on ResearchGate.

ResearchGage - ISO 17025:2005 closest relation to a management system for Digital Forensics

The belief that ISO 17025 is the best fit is shared by the Forensic Science Regulator in the UK, Dr. Gillian Tully who mandated that ISO 17025 would be mandatory for all Digital Forensic Labs in the UK by October 2017.

 

What is the FSR (Forensic Science Regular)?

The Forensic Science Regulator ensures that the provision of forensic science services across the criminal justice system is subject to an appropriate regime of scientific quality standards.

Source: Gov.uk

 

 

Who is the Forensic Science Regulator in the UK?

The post of FSR was established in 2007 and is currently held by Dr Gillian Tully. The Regulator is a public appointee, sponsored by the Home Office, who ensures that the provision of forensic science services across the Criminal Justice System (CJS) is subject to an appropriate regime of scientific quality standards.

Source: Forensic Science and Beyond

 

 

Update to ISO 17025:2005

The update to ISO 17025:2005 has been referred to as ISO 17025:2017 or ISO 17025:20xx.

The new standard is expected to be released by the end of 2017.  The original standard was produced in 1999 with only minor revisions in 2005.

The current ISO standard is referred to as ISO 17025:2005.

The present revision addresses the need to align it with the other more recent ISO 17000 series standards, as required by ISO CASCO and to modernise the standard, recognising advances in technology and business practices.

Source: United Kingdom Accreditation Service

 

ISO 17025 - Right Fit for Digital Forensics?    

There are others within the digital forensic community that do not believe that ISO 17025 is a good fit for Digital Forensics. 

Some of those concerns were captured in a recent UK survey, which is discussed below.

 

ISO 17025 UK Survey

A survey in the UK was conducted in early 2017 by Pat Beardmore, Geoff Fellows and Peter Sommer with results released in April 2017.

A total of 176 people responded to the survey.

Over 65% of those that responded stated that they were within Law Enforcement.

ISO 17025 UK Survey

The Cost of Accreditation

One of the main concerns often raised by practitioners is about the costs associated with ISO 17025 accreditation and whether smaller organizations can bear these extra costs.

 Interestingly, as the survey discovered, even the majority of those who went through the accreditation process were unaware of the actual costs. 

An additional 14% believed the cost was under £50,000 (approx. $66,000 USD) and 15% believed the cost was over that amount. 

With these survey results in mind, it is important to realize that those opposing ISO accreditation based on costs may be doing so due to fear of the unknown rather than hard facts based on research and past experiences.

ISO 17025 UK Survey - How much did it cost?

Understanding of ISO 17025

Another interesting result from the survey, was on the participants understanding of ISO 17025. 

Less than 25% believed they had a “Very good” or “High, ..” understanding of ISO 17025.

Can a person strongly oppose a requirement or movement towards accreditation, such as ISO 17025, if they only have a “Reasonably good” or less understanding of what it entails?

ISO 17025 UK Survey - Understanding of ISO 17025

To view the entire PDF Survey, click here.

 

Forum Discussions

Moving away from stats and charts, another good method to understand what Digital Forensic community really thinks on ISO 17025 is looking at what members have posted on forums. 

Probably the most well-known Digital Forensic forum is “Forensic Focus”. 

In a 2012 Forensic Focus discussion regarding ISO 17025, several key contributors to the forums provided the following comments.

Forensic Focus Discussion - ISO 17025 Right for Digital Forensics

Forensic Focus Discussion Thread - Is accreditation necessary

Although this “MindSmith” comment was posted in 2012, it’s still a valid question in most parts of the world where ISO 17025 is not yet mandated or even discussed.

Of course, in the UK, this appears to no longer be an option for discussion.

I believe the following comment by Jaclaz’s hits at the heart of what ISO 17025 is attempting to accomplish…

Consistency in Quality!

Forensic Focus Discussion - ISO 17025 is about consistency

ISO 17025 does not ensure higher quality work, but it at least sets minimum quality standards to be adhered to, to ensure all labs are at the very least starting on a level playing field.

ISO 17025 Accreditation create a level playing field?

 

Standards

When it comes to having high standards in digital forensic work, the voices from the community are loud and clear. 

Without standards or accreditations in place, the credibility of forensic examiners will likely be questioned in the future. 

“Credibility” would certainly endure increased scrutiny in the event of high-profile cases, especially where it is found that the examiner failed to have the proper training or knowledge to complete standard digital forensic examinations.

The lack of requirements for digital forensic practitioners to be certified in their discipline, be accountable to industry best practices and standards, or work out of accredited laboratories places the credibility of this forensic science in jeopardy.

~ Josh Moulin
Deputy Chief Information Officer
US Federal Government, National Security


Share Article with Digital Forensics Community


 

Impact of Inconsistent Standards

Josh Moulin backs up the above statement with the following information in his “Disheveled Digital Forensics: The Impact of Inconsistent Standards, Certifications, and Accreditation” 2014 Thesis Paper with the following comment.

Although digital forensics has been recognized as a legitimate forensic science and has been utilized in the criminal justice system for the same length of time that DNA has, the discipline is anything but disciplined. Within the United States, any law enforcement agency, business, or individual can open a forensic “laboratory” and begin providing services without having to demonstrate even foundational knowledge, skills, or abilities.

To further evidence this, within the law enforcement community alone there are only 67 digital forensic laboratories accredited to the ISO 17025:2005 standards for the nearly 18,000 law enforcement agencies in the country.

~ Josh Moulin
Deputy Chief Information Officer
US Federal Government, National Security

Although Josh Moulin does agree with the idea of accreditation, he is aware that it’s not the “be-all and end all”.

Having a laboratory accredited according to best practices such as ISO 17025 removes many questions about the quality assurance of the laboratory and the personnel performing work. Accreditation is not the be-all and end-all or a magic solution to issues plaguing the digital forensic discipline.

Accredited laboratories have been known to have issues with their findings as well, the only difference is that the laboratory accreditation standards generally help bring misconduct to light. For example, in 2014 the Oregon State Police quietly closed down their handwriting analysis unit after conducting an internal review of allegations involving bias, sloppy work, and dishonesty (Denson, 2014).

A report to the U.S. Congress said, “In the case of laboratories, accreditation does not mean that accredited laboratories do not make mistakes, nor does it mean that the laboratory utilizes best practices in every case, but rather, it means that the laboratory adheres to an established set of standards of quality and relies on acceptable practices within these requirements” (National Research Council, 2009).

~ Josh Moulin
Deputy Chief Information Officer
US Federal Government, National Security

If you haven’t had an opportunity to read his Thesis paper (84 pages), I highly recommend that you do as it includes a lot of great information on the subject of accreditation and why it is needed within the Digital Forensic Community.

He goes on to state the following, and I believe most forensic examiners would agree with it, especially if someone they cared about was being accused of a criminal act.

Much of the digital forensic community desires to have their evidence seen in court as forensically sound and bulletproof, yet do not want to go through the rigors that other traditional forensic sciences have done to prevent evidence spoliation and other mishandling and misinterpretations. …

If any digital forensic analyst ever found themselves in a position where digital evidence was being used in a legal proceeding against them, they would absolutely want that digital evidence processed in the best forensics lab with the most skilled analyst who meets certain standards.

~ Josh Moulin
Deputy Chief Information Officer
US Federal Government, National



 

Josh Moulin is not the only Digital Forensic Examiner who is worried about the current lack of standards and accreditation.  

Brett Shavers recently wrote a blog article titled “The last thing we want in DF/IR is the first thing we need in DF/IR (aka: regulations…)”.

Within this article, Brett states the following…

Brett Shavers : Digital forensics practitioner, author, and instructor.

The DF/IR field, as it stands today, is practically the Wild Wild West.  We have few regulations outside of obtaining a business license …  It’s freewheeling at the moment without any government intervention.

Brett Shavers
Digital Forensics Practitioner, Author, and Instructor

Author of "Placing the Suspect Behind the Keyboard", "Hiding Behind the Keyboard", and the "X-Ways Forensics Practitioner’s Guide". 
Brett Shavers Blog

Brett goes on to suggest that we need to start implementing our own regulations and standards before the government decides what is best for our profession.

For those of us in the United States or Canada, it appears that we still have time to guide this process towards an accreditation that fits Digital Forensics and isn’t too burdensome to implement.

But all it takes is one major court case and the government could quickly swoop in with regulations they deem necessary.

Let me get to the solution before getting into the issues.  Simply copy and modify what is being done in other professions to fit the DF/IR profession, and give our ideas to the respective government regulatory agencies to implement…Pick a profession, any profession, and get started.

~ Brett Shavers

 

Accreditation IS Useful – but is ISO 17025 the Solution?

Preston Coleman provides further insight into ISO 17025 accreditation as an examiner working within one of the few accredited labs in the United States.

Working in an ISO 17025 lab himself, he doesn’t disagree that there is a high cost and more work involved while working in an ISO 17025 lab, however he does say “accreditation as a concept should be useful and highly desired”.

Forensic Focus Discussion - Working in an ISO 17025 Lab in USA

~ Source: Forensic Focus Forum

Preston Coleman also mentions the need for “proper documentation”.

-- INSERT SHAMELESS PLUG --

Forensic Notes helps you to meet ISO 17025 Requirements

-- END SHAMELESS PLUG --

 

Forensic Science Regulator’s View - UK

Dr. Gillian Tully who is the UK’s Forensic Science Regulator recognizes the issues stating the following within the Forensic Science Regulator Annual Report (released January 2017).

A year on, it is clear that the single biggest challenge to achieving my aim is financial: the costs associated with complying with and being assessed against the standards.

~ Dr. Gillian Truly – Forensic Science Regulator

However, Dr. Gillian Tully goes on to state why she believes in ISO accreditation, stating…

To be clear, the standards are not some unachievable ‘gold-plated’ ideal; they are the minimum standards expected of any reliable forensic science organisation, drawing from general good scientific practice and also learning from errors and omissions of the past and of other industries. There have been enough examples of poor practice, lack of validation of methodology and ‘rogue’ laboratories in recent years (largely outside the UK) to make the case for a robust but proportionate quality system, with an assurance mechanism to check compliance.

Funding for forensic science across the board, and particularly, perhaps, for defence provision via legal aid, must be at a level that enables the standards to be met.

~ Dr. Gillian Truly – Forensic Science Regulator

Finishing with a powerful statement on why regulations must be put in place within Digital Forensics…

Otherwise we will face the costs, both in criminal justice terms and financially, of quality failures and loss of confidence in forensic science.

~ Dr. Gillian Truly – Forensic Science Regulator

 

In Conclusion

I believe the above statement really does summarize why accreditation is required within the Digital Forensic field.

Many of our reports are used to help convict or exonerate individuals.

We cannot forget that these individuals are fathers, mothers, sons, daughters, family members and friends of people we may know.

They deserve to have any evidence used in their trial to be treated and assessed to a rigorous and high standard.

We would not expect Forensic Labs handling DNA to NOT be accredited, so why would we want digital forensic labs to remain un-accredited?

How much would it cost your organization to lose a civil lawsuit if a report your organization produced resulted in the conviction of an individual who was later found to not be guilty?

Moving towards accredited Digital Forensic Labs is just part of the reality of progress within our field.

As an investigator, I would not want to use DNA results from a non-accredited office if I could get results from an internationally recognized lab which meets stringent regulations.

Accreditation is not unattainable or unbearable, as labs accredited in ISO 17025 have existed for many years as indicated within Josh Moulin’s Disheveled Thesis and the FSR Annual Report.

Forensic Science Regulator - Annual Report - Digital Forensics

Source: FSR – Annual Report

But many of these larger accredited organizations are losing contracts to smaller non-accredited companies.

Forensic Science Regulator - Annual Report - Larger Organizations

Source: FSR – Annual Report

This isn’t to say that small Digital Forensics labs shouldn’t exist, but they will need to raise their fees if they want to become accredited and compete for high profile criminal cases where accreditation becomes a requirement.

For many sole-proprietors, this may unfortunately push them out of business. 

The FSR states within their Annual Report that they are looking at ways to reduce the costs to sole-proprietors so there is hope that the costs can be reduced to allow them to remain competitive within the market.

The FSR Annual Report also recognizes that many organization will have failed to meet the requirements by the October 2017 deadline stating that “a substantial proportion of digital evidence produced after that date, disclosure of non-compliance will be required.”

Forensic Science Regulator - Declaration of Non-Compliance

Source: FSR – Annual Report

Once again, I am not saying that ‘ISO 17025’ is the best fit with Digital Forensics, but I do believe some sort of accreditation is required

We can’t ignore the fact that if accreditation is mandated in your country, there will be additional costs which could negatively impact some of the smaller digital forensic offices.

However, the goal is to have the overall consistency of digital forensic examinations increase to help ensure the evidence is presented fairly and accurately while reducing the chances of costly litigation due to incorrect or insufficient reports.

 

What are your thoughts regarding the information provided in this article?

My goal is keep the discussion going on forensic standards and accreditation, whether it is ISO 27025 or otherwise. 

I hope this article will help generate further debate amongst the digital forensic community as we all continue to look for ways to ensure excellence in our field.

Please post any comments below or at our favorite forum, Forensic Focus forums at: www.ForensicFocus.com

MEET ISO 17025 REQUIREMENTS FOR DOCUMENTATION
SIGN UP TODAY for a FREE 14-Day Full-Feature Trial
- no credit card or payment information required -


Share Article with Digital Forensics Community


Join the Twitter Discussion #17025

This article has spawned several good discussion on Social Media including the following one on Twitter (#17025).

Twitter Discussion on #17025 - p1

Twitter Discussion on #17025 - p2

It will take the efforts of members of their DFIR org to push their board to cooperate together with other DFIR orgs. Most efficient is board members communicating across organizations, rather than individuals trying to do this alone.

~ Brett Shavers (@Brett_Shavers)