EVIDENCE & FAILURE
One of the early stories tells of James Marsh who in 1832 was called as an expert witness in a murder trial to demonstrate that a sample of coffee had been contaminated with arsenic. Marsh identified arsenic was present, however, when the time came to present to the jury his sample had degraded giving rise to reasonable doubt and the accused walked free. McDermid writes:
“James Marsh was a proper scientist. He regarded this failure as a spur towards success. His response to the embarrassment of his court appearance was to devise a better test.”
This attitude towards failure as an opportunity to learn is also ingrained in many software development and operations frameworks. The story is also a reminder of the ephemeral nature of evidence. When an incident occurs, evidence that may help understand the root cause may not last long in memory and eventually may be lost. If root cause analysis is not treated with urgency after an incident is resolved, the evidence necessary to determine the root cause may vanish.
ASSUMPTIONS vs TRUTH
Another story tells the tale of Bernard Spilsbury who came to fame in 1910 for his expert testimony in the trial of Hawley Harvey Crippen accused of murder. McDermid describes Spilsbury as a having “a liberal sprinkling of charisma” and a “handsome, convincing orator”. During the trial “the judge referred to Spilsbury as ‘the greatest living pathologist.’” Spilsbury claimed a skin sample found in Crippen’s home had a scar just like a scar the female murder victim was known to have. However, the defense pointed out hair follicles in the sample indicate it could not have been scar tissue and thus did not point to the victim. Later DNA analysis cast doubt that the sample belonged to a female or was closely related to the victim.
This adolescent period where charisma trumped hard science, is a reminder of the responsibility of those of us who are looked to as experts. If you have unique expertise or knowledge in a group making a decision, you cannot rely on an adversarial system to help you arrive at the best decision. Instead, we must dedicate ourselves to the truth and state and challenge our own assumptions.
EVIDENCE COLLECTION KIT
Spilsbury is also known for establishing the “murder bag”, a collection of gloves, tweezers, evidence bags and other equipment to use in homicide investigations. When you approach an investigation of an incident, there is likely a standard set of tools and techniques you should use for investigation. You check event logs, application logs, CPU utilization, memory utilization, and database logs. Checking all these logs and performance counters manually can take a long time. Consider in your work what prepared kits or tools you could use to collect, process, and analyze logs and performance counters for abnormalities to save time in isolating an incident.
RECOMMENDATION
Forensics: What Bugs, Burns, Prints, DNA, and More Tell Us About Crime by Val McDermid is an engaging, captivating read. Learning how criminal investigators refined their tools and techniques over the history of forensic science can help spawn ideas about how to improve investigation and response to IT incidents.
