This is not controversial: reliable and timely insights into application and data dependencies improve overall operations and speed digital transformation and application modernization.
In February of 2020, Crosscode surveyed 10,000+ IT and development professionals to capture and measure the risks that stem from under-investing in this foundational IT capability. An overview and supplement of the survey results can be downloaded here.
Like any risk management topic, we need to identify, evaluate, categorize, and prioritize risks using some combination of (likelihood of occurrence) and (severity of occurrence) to order and prioritize all known risks.
"Not knowing what you don't know" will sink you every time
Herein lies the catch in what otherwise reads like a straightforward process; if you don’t know that a risk exists, you can’t begin the process of evaluations and prioritization – unknown risks are, by definition, uncategorized and unmanaged.
While there is often a numerical basis to calculating risk severity, those numbers are then grouped into bands (or quantum) whose definitions typically include recommended (or mandated) mitigation responses, e.g., any risk rated as “High” must have an approved mitigation plan within 30 days of discovery (or something like that).
While Risk Quantum may share universally recognized labels such as “High,” “Moderate,” and “Low,” their definitions are as varied as the Risk Domains that they cover, e.g., medical devices, automobiles, information systems, or manufacturing/project management.
There are (at least) two distinct Risk Domains where unknown code and data dependencies play a material role:
- Software development and deployment processes
- The management of production information systems and information that they process
The following table offers some representative definitions across these two Risk Domains:
Dark Matter: The things you don’t know that you don’t know
Both of the risk ratings above fail to include “UNKOWN” as a Risk Quantum – and, at some level, it’s easy to understand why. You can define a risk as “the discovery of currently unknown High-risk vulnerability” and rate the risk of that occurring as Moderate or Low risk or whatever.
…but, in fact, like “dark matter” whose existence is required in order to make sense of the rest of the universe, we feel the weight of “unknown risks” every day.
When Enterprises Confuse “No Risk” with “Unknown Risk”
Survey respondents were asked to rate the risks stemming from their organization’s policies and practices in support of the following eleven initiatives, often closely associated with the migration to modern DevOps CI/CD practices and/or digital transformation.
No-Risk and High Risk were closely correlated on both ends, e.g., the initiatives that were most likely to be rated as Unacceptably High would also be most likely to be rated as having No Risk whatsoever and, similarly, initiatives that were least likely to be rated as High Risk were also least likely to be rated as No Risk at all.
This suggests that a particular Initiative’s risk ratings do not move up and down smoothly over time.
Rather, survey respondents show that “No Risk” serves as a stand-in quantum for risks that we don’t know that we don’t know.
At some point in time – perhaps based upon a particular incident – the risk suddenly comes to light, and the organization scrambles to properly identify, evaluate, and categorize. It’s during this initial period, immediately “Upon Discover” where risks are rated as High. Once formal assessments are complete (and any needed controls are in place), the risk is reclassified as managed or acceptable.
Action: Know what you don’t know (or “No policy IS a policy”)
Mitigating newly discovered gaps and vulnerabilities is expensive and disruptive (and a risk in and of itself). Don’t wait until you are bitten - review risks that have been identified as “none” or “nominal” and ensure that these ratings are based upon a formal assessment (in accordance with whatever your risk management process dictates) versus simply not having taken the time to properly assess. Having “No Policy” on well-known scenarios is a policy.
How are your peers managing the risks stemming from unknown software and data dependencies in their modernization and digital transformation processes? Download our survey results here.