Most of us in the Digital Forensic community agree we need some level of standards. But who should determine these standards? And which standards are truly relevant in ensuring the integrity of digital forensic work?
In 2020 the debate still continues on whether ISO 17025 is right for Digital Forensics. In some countries such as United Kingdom, it is now a mandatory standard.
Should this standard be adopted in the United States and Canada? If not, what standard should be digital forensic community be held to?
- What is the FSR (Forensic Science Regular)?
- Who is the Forensic Science Regulator in the UK?
- Update to ISO 17025:2005
ISO 17025 is a mandatory standard for Digital Forensics laboratories in the United Kingdom (UK) as of October 2017.
All labs that are not ISO 17025 certified must disclose their ‘non-compliance’ on every report produced.
The following article is meant to provide information and open the discussion around this topic.
This ISO 17025 accreditation will impact how Digital Forensic examinations are conducted in the future and by whom around the globe.
As it stands, views are mixed about the suitability of this standard for Digital Forensics.
Certainly, some Digital Forensic Examiners (DFE’s) believe that using ISO 17025 for Digital Forensics is like placing a square peg into a round hole.
Is this belief based on fact or fear?
ISO 17025
ISO 17025 is also referred to as ISO/IEC 17025.
ISO – International Organization for Standardization
IEC – International Electrotechnical Commission
What is ISO 17025?
ISO 17025 was first published in 1999 to standardize labs around the world to ensure results from one lab would be accepted or repeated by other standardized labs.
This helps to break down international borders between countries when sharing forensic lab results.
To become ISO 17025 accredited, nationally recognized laboratory accreditation bodies assess the labs for conformity.
These accreditation bodies must follow established “methods of evaluation” developed by the International Laboratory Accreditation Cooperation (ILAC).
The ISO/IEC 17025 standard is split into 5 distinctive categories:
- Scope
- Normative Resources
- Terms and Definitions
- Management Requirements
- Technical Requirements
Some areas that may be addressed within the above 5 categories include:
- Testing and Calibration Standards
- Staff Competence
- Equipment Standards
- Quality Management
Source: Wikipedia ISO/IEC 17025
Although ISO 17025 was written for testing and calibrating laboratories, many believe that it is the best fit for Digital Forensic Laboratories simply because no other international standard for digital forensics currently exists.
This has been discussed in many articles including a conference paper on ResearchGate.
The belief that ISO 17025 is the best fit is shared by the Forensic Science Regulator in the UK, Dr. Gillian Tully who mandated that ISO 17025 would be mandatory for all Digital Forensic Labs in the UK by October 2017.
What is the FSR (Forensic Science Regulator)?
Source: Gov.uk
Who is the Forensic Science Regulator in the UK?
Source: Forensic Science and Beyond
Update to ISO 17025:2005
The update to ISO 17025:2005 has been referred to as ISO 17025:2017 or ISO 17025:20xx.
The new standard was released near the end of 2017. The original standard was produced in 1999 with only minor revisions in 2005.
ISO 17025 – Right Fit for Digital Forensics?
There are others within the digital forensic community that do not believe that ISO 17025 is a good fit for Digital Forensics.
Some of those concerns were captured in a recent UK survey, which is discussed below.
Want to save your job? Your reputation?
Read Brett Shaver’s article “No job is finished until the paper work is done” on the importance of proper documentation!
ISO 17025 UK Survey
A survey in the UK was conducted in early 2017 by Pat Beardmore, Geoff Fellows and Peter Sommer with results released in April 2017.
A total of 176 people responded to the survey.
Over 65% of those that responded stated that they were within Law Enforcement.
The Cost of Accreditation
One of the main concerns often raised by practitioners is about the costs associated with ISO 17025 accreditation and whether smaller organizations can bear these extra costs.
Interestingly, as the survey discovered, even the majority of those who went through the accreditation process were unaware of the actual costs.
An additional 14% believed the cost was under £50,000 (approx. $66,000 USD) and 15% believed the cost was over that amount.
With these survey results in mind, it is important to realize that those opposing ISO accreditation based on costs may be doing so due to fear of the unknown rather than hard facts based on research and past experiences.
Understanding of ISO 17025
Another interesting result from the survey, was on the participants understanding of ISO 17025.
Less than 25% believed they had a “Very good” or “High, ..” understanding of ISO 17025.
Can a person strongly oppose a requirement or movement towards accreditation, such as ISO 17025, if they only have a “Reasonably good” or less understanding of what it entails?
To view the entire PDF Survey, click here.
Forum Discussions
Moving away from stats and charts, another good method to understand what Digital Forensic community really thinks on ISO 17025 is looking at what members have posted on forums.
Probably the most well-known Digital Forensic forum is “Forensic Focus”.
In a 2012 Forensic Focus discussion regarding ISO 17025, several key contributors to the forums provided the following comments.
Although this “MindSmith” comment was posted in 2012, it’s still a valid question in most parts of the world where ISO 17025 is not yet mandated or even discussed.
Of course, in the UK, this appears to no longer be an option for discussion.
I believe the following comment by Jaclaz’s hits at the heart of what ISO 17025 is attempting to accomplish…
Consistency in Quality!
The latest discussions on ISO 17025 and this article are taking place on Computer Forensics World Forum.
Join the discussion in the Forums!
ISO 17025 does not ensure higher quality work, but it at least sets minimum quality standards to be adhered to, to ensure all labs are at the very least starting on a level playing field.
Standards
When it comes to having high standards in digital forensic work, the voices from the community are loud and clear.
Without standards or accreditations in place, the credibility of forensic examiners will likely be questioned in the future.
“Credibility” would certainly endure increased scrutiny in the event of high-profile cases, especially where it is found that the examiner failed to have the proper training or knowledge to complete standard digital forensic examinations.
~ Josh Moulin
Deputy Chief Information Officer
US Federal Government, National Security
Impact of Inconsistent Standards
Josh Moulin backs up the above statement with the following information in his “Disheveled Digital Forensics: The Impact of Inconsistent Standards, Certifications, and Accreditation” 2014 Thesis Paper with the following comment.
Although digital forensics has been recognized as a legitimate forensic science and has been utilized in the criminal justice system for the same length of time that DNA has, the discipline is anything but disciplined. Within the United States, any law enforcement agency, business, or individual can open a forensic “laboratory” and begin providing services without having to demonstrate even foundational knowledge, skills, or abilities.
To further evidence this, within the law enforcement community alone there are only 67 digital forensic laboratories accredited to the ISO 17025:2005 standards for the nearly 18,000 law enforcement agencies in the country.
~ Josh Moulin
Deputy Chief Information Officer
US Federal Government, National Security
Although Josh Moulin does agree with the idea of accreditation, he is aware that it’s not the “be-all and end all”.
Having a laboratory accredited according to best practices such as ISO 17025 removes many questions about the quality assurance of the laboratory and the personnel performing work. Accreditation is not the be-all and end-all or a magic solution to issues plaguing the digital forensic discipline.
Accredited laboratories have been known to have issues with their findings as well, the only difference is that the laboratory accreditation standards generally help bring misconduct to light. For example, in 2014 the Oregon State Police quietly closed down their handwriting analysis unit after conducting an internal review of allegations involving bias, sloppy work, and dishonesty (Denson, 2014).
A report to the U.S. Congress said, “In the case of laboratories, accreditation does not mean that accredited laboratories do not make mistakes, nor does it mean that the laboratory utilizes best practices in every case, but rather, it means that the laboratory adheres to an established set of standards of quality and relies on acceptable practices within these requirements” (National Research Council, 2009).
~ Josh Moulin
Deputy Chief Information Officer
US Federal Government, National Security
If you haven’t had an opportunity to read his Thesis paper (84 pages), I highly recommend that you do as it includes a lot of great information on the subject of accreditation and why it is needed within the Digital Forensic Community.
He goes on to state the following, and I believe most forensic examiners would agree with it, especially if someone they cared about was being accused of a criminal act.
Much of the digital forensic community desires to have their evidence seen in court as forensically sound and bulletproof, yet do not want to go through the rigors that other traditional forensic sciences have done to prevent evidence spoliation and other mishandling and misinterpretations. …
If any digital forensic analyst ever found themselves in a position where digital evidence was being used in a legal proceeding against them, they would absolutely want that digital evidence processed in the best forensics lab with the most skilled analyst who meets certain standards.
~ Josh Moulin
Deputy Chief Information Officer
US Federal Government, National
Josh Moulin is not the only Digital Forensic Examiner who is worried about the current lack of standards and accreditation.
Brett Shavers recently wrote a blog article titled “The last thing we want in DF/IR is the first thing we need in DF/IR (aka: regulations…)”.
Within this article, Brett states the following…
Brett Shavers
Digital Forensics Practitioner, Author, and Instructor
Author of “Placing the Suspect Behind the Keyboard”, “Hiding Behind the Keyboard”, and the “X-Ways Forensics Practitioner’s Guide”.
Brett Shavers Blog
Brett goes on to suggest that we need to start implementing our own regulations and standards before the government decides what is best for our profession.
For those of us in the United States or Canada, it appears that we still have time to guide this process towards an accreditation that fits Digital Forensics and isn’t too burdensome to implement.
But all it takes is one major court case and the government could quickly swoop in with regulations they deem necessary.
~ Brett Shavers
Accreditation IS Useful – but is ISO 17025 the Solution?
Preston Coleman provides further insight into ISO 17025 accreditation as an examiner working within one of the few accredited labs in the United States.
Working in an ISO 17025 lab himself, he doesn’t disagree that there is a high cost and more work involved while working in an ISO 17025 lab, however he does say “accreditation as a concept should be useful and highly desired”.
~ Source: Forensic Focus Forum
Preston Coleman also mentions the need for “proper documentation”.
— INSERT SHAMELESS PLUG —
— END SHAMELESS PLUG —
Forensic Science Regulator’s View – UK
Dr. Gillian Tully who is the UK’s Forensic Science Regulator recognizes the issues stating the following within the Forensic Science Regulator Annual Report (released January 2017).
~ Dr. Gillian Truly – Forensic Science Regulator
However, Dr. Gillian Tully goes on to state why she believes in ISO accreditation, stating…
To be clear, the standards are not some unachievable ‘gold-plated’ ideal; they are the minimum standards expected of any reliable forensic science organisation, drawing from general good scientific practice and also learning from errors and omissions of the past and of other industries. There have been enough examples of poor practice, lack of validation of methodology and ‘rogue’ laboratories in recent years (largely outside the UK) to make the case for a robust but proportionate quality system, with an assurance mechanism to check compliance.
Funding for forensic science across the board, and particularly, perhaps, for defence provision via legal aid, must be at a level that enables the standards to be met.
~ Dr. Gillian Truly – Forensic Science Regulator
Finishing with a powerful statement on why regulations must be put in place within Digital Forensics…
~ Dr. Gillian Truly – Forensic Science Regulator
In Conclusion
I believe the above statement really does summarize why accreditation is required within the Digital Forensic field.
Many of our reports are used to help convict or exonerate individuals.
We cannot forget that these individuals are fathers, mothers, sons, daughters, family members and friends of people we may know.
They deserve to have any evidence used in their trial to be treated and assessed to a rigorous and high standard.
We would not expect Forensic Labs handling DNA to NOT be accredited, so why would we want digital forensic labs to remain un-accredited?
How much would it cost your organization to lose a civil lawsuit if a report your organization produced resulted in the conviction of an individual who was later found to not be guilty?
Moving towards accredited Digital Forensic Labs is just part of the reality of progress within our field.
As an investigator, I would not want to use DNA results from a non-accredited office if I could get results from an internationally recognized lab which meets stringent regulations.
Accreditation is not unattainable or unbearable, as labs accredited in ISO 17025 have existed for many years as indicated within Josh Moulin’s Disheveled Thesis and the FSR Annual Report.
~ Source: FSR – Annual Report
But many of these larger accredited organizations are losing contracts to smaller non-accredited companies.
~ Source: FSR – Annual Report
This isn’t to say that small Digital Forensics labs shouldn’t exist, but they will need to raise their fees if they want to become accredited and compete for high profile criminal cases where accreditation becomes a requirement.
For many sole-proprietors, this may unfortunately push them out of business.
The FSR states within their Annual Report that they are looking at ways to reduce the costs to sole-proprietors so there is hope that the costs can be reduced to allow them to remain competitive within the market.
The FSR Annual Report also recognizes that many organization will have failed to meet the requirements by the October 2017 deadline stating that “a substantial proportion of digital evidence produced after that date, disclosure of non-compliance will be required.”
~ Source: FSR – Annual Report
Once again, I am not saying that ‘ISO 17025’ is the best fit with Digital Forensics, but I do believe some sort of accreditation is required
We can’t ignore the fact that if accreditation is mandated in your country, there will be additional costs which could negatively impact some of the smaller digital forensic offices.
However, the goal is to have the overall consistency of digital forensic examinations increase to help ensure the evidence is presented fairly and accurately while reducing the chances of costly litigation due to incorrect or insufficient reports.
What are your thoughts regarding the information provided in this article?
My goal is to keep the discussion going on forensic standards and accreditation, whether it is ISO 27025 or otherwise.
I hope this article will help generate further debate amongst the digital forensic community as we all continue to look for ways to ensure excellence in our field.
Please post any comments at Computer Forensics World Forum at:
Join the Twitter Discussion #17025
This article has spawned several good discussion on Social Media including the following one on Twitter (#17025).
~ Brett Shavers (@Brett_Shavers)