What is a Software Change Assessment?
Change is an inevitable part of the software lifecycle. As user needs evolve, companies grow, and technologies advance, software must change to meet the new needs and reflect the current state-of-the-art. Sometimes these changes are small, incremental improvements, other times they are major overhauls.
Regardless of the size of a change, in today’s complex technology environments managing change presents significant challenges. Effective change management requires you to fully understand the various elements of your software infrastructure and to be able to accurately predict both the intended and unintended consequences of the change. To be successful, IT teams must have solid processes in place and the support of best-in-class tools.
Change assessment is the systematic evaluation of a software change plan to determine exactly how the change will affect the organization’s software, hardware, people and processes. The goal of an impact assessment is to support change planning, accurately identify project risks, and estimate the level of effort that will be required to accomplish the planned changes.
In this eBook we present a five-step Change Assessment process. The approach here is based on accepted best practices in the field amplified by our own extensive experience managing change projects. It is intended to be high-level enough to be useful across organizations of different sizes and missions, but with specific concepts and guidance that you can put to use today supporting better change management.
The process guides your team methodically from an initial change request to a final prioritization and go/no-go decision on the change. As we’ll see, this is a process that champions teamwork and the input of subject matter experts.
It is also a process that champions software. With fully automated features for application discovery, code level dependency mapping and change assessment, our Crosscode Panaoptics solution supports a data-driven approach to change management that not only helps you conduct a more accurate analysis, but can save you and your team months of work gathering and analyzing key data about your infrastructure. Throughout this eBook we will explore how Panaoptics can help you as you progress through your change management journey.
Step 1: How to Define The Software Change
Define the Change is Step 1 of the five-step change assessment process. In this step, we consider all aspects of a proposed change to describe the systems and environments that will be affected and try to envision what will happen before, during and after the change.
Correctly identifying the scope of a change project will be critical to the future impact assessment steps. No matter how comprehensive and precise your assessment practices may be, an analysis of how a change will ripple through your architecture will be inherently flawed if you have failed to properly describe the change itself.
There is a great deal of variety in the way that organizations approach change definition. Some organizations rely on a rigorous process that mandates specific data be documented – frequently on a Request for Change (RFC) template or similar change management form. Other organizations use a more free-form approach where the change requester, project manager, or business analyst describes the change in a written narrative.
Regardless of how your organization structures its data collection for change definition, there are certain standard questions of who, what, when, where and how that a robust, usable definition will address.
Source of the Change
Many issues drive software change. Potential change sources include:
- Planned Development
- Issue Resolution
- Product Enhancement Requests
Aligning change requests to their source can be especially helpful down the line when the time comes to approve and prioritize requests.
Goal of the Software Change
Every change should be motivated by a clearly defined goal. A software change might improve workflow efficiency, address a flaw in the current solution, or introduce a new value-added capability or feature. The change definition should articulate what the organization hopes to accomplish and what business value is hoped to be realized by the change.
Magnitude of a software change
The magnitude of a software change is a function of both the complexity of the proposed change and the sheer volume of code and other changes. Larger projects not only demand a greater commitment of time and resources, they also are inherently more challenging to estimate.
Scope of the Change
Many software changes will reach far beyond the software itself. Some changes involve new or modified hardware, changes in policies and procedures, or a new workflow. It is easy to forget a routine follow-on activity such as updating documentation or providing user training, so it is important to spend time working through the scope to capture as many implications of the change as possible.
Risks of the Change
Identifying the potential negative outcomes of a proposed change is perhaps the single most important element of the change management process. Project risks come in a wide variety of forms, from cost overruns due to poorly understood requirements to system outages caused by breakdowns among unknown software dependencies. It is only by systematically evaluating all risks and putting appropriate mitigation plans in place that the organization can move forward with confidence.
Much of the information gathered during the Change Definition step will be expanded and refined over the course of the impact assessment process. In particular, both the scope of the change and the risks of the change will become the focus of the analysis performed in Step 2: Identify Key Differences and Step 3: Focus on Effects. By bringing in the appropriate expertise early and taking the time to develop a comprehensive definition up front, you can lay the foundation for a solid analysis down the line.
Crosscode Panoptics software gives your development team the insight and understanding needed to support in depth impact assessments. Panoptics’ change assessment features enable you to analyze the effect of any proposed system changes with a visually intuitive and easy-to-use tool that lets you see instantly how a proposed change will impact other applications and databases across the enterprise. Unique in the industry, Panoptics provides a mechanism to estimate the magnitude of a proposed change in terms of the number of methods that would be impacted by that change.
Panoptics’ application discovery and dependency mapping features are fully automatic, saving months of labor compiling the data for impact assessment.
Step 2: How To Identify the Key Software Differences?
Identify Key Differences is Step 2 of the impact assessment process. In this step we take a deep dive into the software, hardware, people, process and environment of our systems and anticipate what will be different after the change. The goal here is to compare before and after images of our infrastructure and operations to articulate what will be different and what those differences will be.
Software and Software Artifacts
Developing a complete description of key differences in your software environment will be the largest and most challenging component of this impact assessment step. Historically, change management practitioners have identified two broad classes of software change analysis – tractability analysis and dependency analysis. These approaches are not exclusive, and a good analysis protocol might blend them effectively.
In tractability analysis, the relationships between system requirements and software specifications are captured and documented. If a requirement or specification changes, the analyst can use the baseline tractability relationships to identify other requirements or specifications that may change or need to be changed as a result.
In dependency analysis, the architectural and code-level linkages between low-level software components are captured and documented. If a system element is changed, the analyst can use the baseline dependency linkages to identify other system elements that may change or need to be changed.
Not surprisingly, the quality of a software change analysis is highly reliant on the quality of the available data. Any gaps or errors in the baseline documentation will propagate through the analysis to produce errors in the assessment of key differences.
Good data for analysis can be hard to come by. Manual documentation for large complex software architectures is extremely difficult and expensive to maintain, and documentation is often not given a high priority in today’s rapid development, continuous deployment environments.
Fortunately, our enterprise solution, Crosscode Panoptics, fully automates the application discovery and code-level dependency mapping process. This automation enables your organization to complete analyses with a fraction of the time and expense seen in manual efforts, while at the same time driving up the quality and completeness of your baseline data.
Panoptics also provides powerful impact assessment functionality to complete this assessment step by letting you see and document how any proposed software change will ripple through your architecture and what other elements may need to be changed.
Along with the software itself, there are a number of standard software development artifacts that may be impacted by the change. Examples of such artifacts include user manuals, online help, and training materials, as well as the baseline architecture documentation itself describing requirements, specifications, and dependencies.
Software change projects frequently involve changes in the associated hardware requirements. Software changes can result in greater server storage needs, higher network throughput requirements, or the need for end users to have a more capable minimum configuration.
As with software changes, in the Identify Key Differences step we will seek to define the current hardware baseline as accurately as possible and then identify all the elements of that baseline that would be different after the change project.
Certain new hardware requirements may be difficult to accurately determine during an impact assessment. The need for some changes might not become apparent until upgraded software can be tested and bench marked. However, there are important steps you can take to anticipate potential new demands on hardware. A software change, for example, may provide new services to users, which in turn may imply greater network demands.
Significantly, many new hardware requirements can be traced to the third-party tools and components. Many times, organizations do not have complete, up-to-date information on the third-party and open source components in their systems. Again, Panoptics’ automated discovery and mapping capabilities can help here, providing an easy mechanism to scan your infrastructure and ensure you have a baseline understanding of all potential contributors to your baseline hardware requirements.
People and Process
Evaluating the follow-on differences to an organization’s staffing and policies for a proposed software change is generally a team activity. Subject matter experts from human resources and other business operations functions are in the best position to help establish a definition for the current state and make informed predictions as to what the key organizational differences will be.
As a preliminary step, all relevant documentation should be gathered and distributed to the team. Depending on the nature of the change project, this documentation could include organization charts, job descriptions, workflows, policies, and procedures. After the team has reviewed the documentation, a facilitated work session can provide a venue for the team to collaborate and determine all the likely organizational changes that would be needed.
Step 3: Focus on Effects Of Software Change
Focus on Effects is Step 3 of the impact assessment process. In this step we turn our attention to things that might go wrong with our change project. While no amount of planning can ensure that every step of a project goes off without a hitch, a proper risk analysis can give you a framework that helps to head off problems and minimize the impact of any problems that do arise. In this step we’ll systematically review each of the key differences, documenting risks associated with those differences and what the overall effect would be should those risks become realized.
The Risk Register
The risks that are documented by the team are maintained in a risk register. While risk registers come in various forms, from basic spreadsheets to full featured dedicated applications, they generally appear as a table where every risk is listed on a row, with columns to capture key pieces of relevant information such as the description of the risk, the type of risk, the magnitude of the risk, potential mitigation, and the likelihood of the risk will be realized. Pictured below is a simplified example of a risk register.
Although here we are implementing the tool during impact assessment, risk registers are a foundation project management tool used throughout the software development lifecycle to track risks and monitor the success of risk mitigation activities.
Sources of Risk Data
As you work to identify project risks, it is important that you incorporate as many information sources as possible. Risk identification is part art and part science, and having multiple perspectives will help to ensure that you don’t overlook critical risks in your assessment. Some of the possible sources of risk information include:
Project Documentation: The listing of key differences developed earlier in the assessment will be your starting point, but you will also want to review other major project documents such as the statement of work, work breakdown structure, and schedule.
The example checklist above shows client related risks, asking questions like “Do you have experience working with the client on previous projects?” Any “no” answers should be followed up with an analysis to determine what risks might be exposed based on the answer. Checklists can come from your own organization’s subject matter experts and also from external sources such as professional organizations.
Risk Assessment Checklists: Checklists are an excellent, and frequently overlooked, resource for risk analysis. Checklists generally present a series of subject matter checkpoint questions for the analyst to work through, with different checklists focusing on different areas of potential risk, such as business impact, process related, or cyber security.
Risk Repository: Your organization will hopefully maintain a shared record of prior impact assessment activities as well as lessons learned from past projects. Reviewing this historical information can provide insight into potential problems on your current project with the added value of reflecting your own team’s unique experiences.
Expert Review: Brainstorming work sessions with the project team and other experts from your organization can help reveal subtle project risks that might otherwise go undetected. Developers are often able to identify sticking points in the process such as the application of a new and untested technology or potential scope creep. To be effective, such work sessions must be conducted in a collaborative environment where team members are comfortable expressing personal concerns and challenging others’ assumptions.
The Panoptics Difference
Crosscode Panoptics’ industry leading discovery and code-level dependency mapping capabilities will significantly improve your ability to identify and mitigate risks when preparing for software change projects. Our unique algorithms dig deeper into your infrastructure than any other solution to reveal dependencies and applications that others will miss - including critical third party and open source components.
Panoptics’ gives you the ability to analyze and assess risks to development and change projects as a function of the number of dependencies, and therefore the applications and databases, that will be affected by any proposed change. You can see at a glance how these changes will propagate through your infrastructure and take steps to reduce the likelihood and potential impact of development defects and unintended consequences.
Step 4: Sort and Prioritize Software Risk
Making Sense of the Change and Risk Data
Sort and Prioritize is Step 4 of the impact assessment process. In this step we take the identified differences identified in Step 2 and the likely effects identified in Step 3 and rank the project risks. This ranking serves two purposes. First, it forms the basis for risk mitigation planning. Second, the risk assessment and ranking become valuable inputs to future decision making.
The most common and direct approach to sorting risks, and the approach we’ll take here, is called first-order prioritization. In first-order prioritization, the risks are sorted so that the risks with the highest potential impact and greatest likelihood of realization are at the top of the list, while the risks with the lowest potential impact and least likelihood of realization are at the bottom of the list.
Assigning a Risk Score
A risk score combines the impact and likelihood of a risk into a single number that can be used for sorting. While a risk score is a quantitative, numeric value, the
The magnitude (or severity) of a risk is usually described verbally using terms such as very small, small, medium, large and very large. Converting this description to a number is a simple matter of substitution. A very small could be given a value of 1, a small a value of 2 etc.assessment process typically begins with qualitative, verbal descriptions. In order to make a calculation, any verbal descriptions will need to be converted to a number.
When converting magnitude rankings from verbal to numeric values, you will generally want to apply a scale to the conversion. Rather than a simple conversion of 1, 2, 3, 4, 5, a scaled conversion might read 1, 2, 4, 8, 16. A scaled conversion provides a more dramatic distinction between low impact and high impact events. This greater distinction makes good sense when you consider that a low impact event might be negligible and of little consequence to the organization, while a high impact event might be truly catastrophic.
As with magnitude scores, probability or likelihood ratings might often be first considered verbally using terms such as very unlikely, unlikely, likely, and very likely. These verbal evaluations would then be converted to a number, frequently expressed as a percentage. There will typically be no call to scale probability numbers.
Once the two assessments are expressed numerically, the two values can be multiplied to produce a risk score. For example, a risk with a magnitude of 2 and a likelihood of 75 could be given an overall risk score of 150, while a risk with a magnitude of 16 and a likelihood of 30 could be given on overall risk score of 480.
Step 5: Decisions Based On Software Change Assessment
Developing a Recommendation Based on Full Impact Awareness
Making a Decision is the fifth and final step of the impact assessment process. In this step we consider all the data we have gathered and the analysis we’ve completed to make recommendations that take into account the costs, benefits, and steps needed to achieve the change with minimal risk to the organization.
For some projects, the necessary decision is whether or not to pursue the change project. This type of decision is frequently called the Go/No Go decision. If the results of the previous steps indicate that the project will require too much effort, the project risks are too great, or the cost to mitigate those risks are too high, then a decision not to proceed is warranted. A No Go decision is likely to be met with disappointment by the change requestor, so it is important that the impact assessment has been thorough and well documented.
For change projects that are accepted for implementation, the impact assessment analyst or team will make recommendations for minimizing the potential for negative outcomes. It is usually not practical or desirable to address all identified risks. Instead the team uses the prioritized listing developed in Step 4 to identify the highest risk elements and focus mitigation effort on those risks.
While formal requirements for documenting the analysis vary from organization to organization, most change management processes will call for each recommendation to be documented individually, describing the risk, identifying the proposed mitigation, and providing a justification and cost/benefit analysis of the proposed mitigation. If more than one potential mitigation exists for a given risk, the documentation should include a comparison of the two mitigations.
Panoptics’ Change Assessment Support
Detailed Accurate Assessments in a Fraction of the Time
Crosscode Panoptics’ change assessment features enable you to analyze the effect of any proposed system changes. Leveraging our unique dependency graph that maps your applications down to the code level, this visually intuitive and easy-to-use tool lets you see instantly how a proposed change will impact other applications and databases across the enterprise.
See instantly how a proposed change impacts other applications and databases.
Panoptics maintains the dependency maps automatically and in real time, saving you months of work manually preparing impact assessments. Panoptics’ mapping is also more thorough and comprehensive than any manual effort could hope to be. Your impact assessments will be more accurate and complete, revealing the full potential impact of proposed changes before a single line of code is written. You will also mitigate the risks of business disruption by capturing and analyzing dependencies that other solutions miss.
Crosscode’s impact assessments will empower you to:
- Make accurate time and budget estimates.
- Find potential problems earlier saving money and reducing risk.
- Modernize your software project estimation processes.
- Analyze projects at any scale and conduct analysis across your entire enterprise.
- View change projects through a single pane of glass across languages and databases.