Best Practices for SCA Tools
Building a successful application security program requires collaboration between product, engineering, and security teams to seamlessly integrate security into the development process from the start. A tiered approach provides a practical framework for achieving incremental, measurable progress — ensuring teams build momentum and continually improve their security posture.
With years of experience helping organizations manage their use of open source software and strengthen their software supply chain security, Sonatype has developed a proven methodology to successfully integrate their software composition analysis (SCA) platform. This approach is structured around four key pillars:
- Establish Clear Goals and Success Criteria
- Continuously Integrate Security into the DevOps Pipeline
- Implement Fast Feedback Loops for Vulnerability Management
- Automate Remediation to Improve Efficiency and Reduce Risk
Adopting this strategic framework and integrating SCA tools will enable organizations to increase the ease and speed of vulnerability remediation and save developers’ time and resources — ultimately, ensuring safe and secure software is shipped fast.
Establish Clear Goals and Success Criteria
Continuously Integrate Security into the DevOps Pipeline
Implement Fast Feedback Loops for Vulnerability Management
Automate Remediation to Improve Efficiency and Reduce Risk
Establish Goals and Success
Criteria for Your SCA Program
To achieve true best-in-class SCA practices, organizations must move beyond improving relative to their own past performance. While establishing a baseline, measuring progress, and iterating on improvements is crucial, we must also define best in class benchmarks for success to ensure your organization is running an objectively successful application security program that is scalable, effective, and truly competitive.
Setting measurable goals is critical for a successful SCA program. A leading indicator of success for software supply chain management is integration into the DevOps toolchain. True success, however, is measured through a reduction in risk. Sonatype suggests using the following metrics, available in Sonatype Lifecycle, as key performance indicators (KPIs) for the program.
DevOps Adoption
Available through the Developer Dashboard, the Adoption Profile graph shows progress in both supply chain management (SCM) and continuous integration / continuous delivery (CI/CD) integration.
Using techniques such as Automated Configuration (described below) make it easy to ensure that all applications are properly integrated. SCM integration should follow with all applications integrated into SCM within a month after CI/CD integration.
GOAL: APPLICATIONS INTEGRATED INTO CI/CD, SCM, AND SCA TOOLS
Average
0% - 20%
Good
21% - 90% adoption
Better
90%+ adoption
Best In Class: Organizations running excellent application security programs achieve 100% adoption in 6 months.
SONATYPE'S DEVELOPER DASHBOARD CAN SHOW PROGRESS ACROSS THE DEVOPS TOOLCHAIN
Risk Reduction
Risk is measured in policy violations through the Lifecycle product, so risk reduction is reflected in fewer policy violations over time. The Success Metrics dashboard provides valuable insights into how open source security risk is being eliminated.
Note that because new violations are being discovered daily, expect an overall downward trend but be prepared for some lumpiness in your trendline.
GOAL: REDUCTION IN CRITICAL VIOLATIONS
Average
0% - 10% reduction
Good
11% - 60% reduction
Better
61%+ reduction
Best In Class: Organizations running excellent application security programs are able to reduce their critical violations by 30% from initial onboarding after 6 months, and 61+% after 12 months.
Continual Dependency Management
Once organizations reach Pillar 4: Automation, many remediations are managed with minimal development effort through Golden Pull Requests. Intelligent remediation advice and prioritization helps developers focus on the most impactful changes. This allows for a significant reduction in Mean Time to Remediation (MTTR*) and an increase in Fix Rate. MTTR is the average time between a violation being discovered and remediated for all levels of severity.
GOAL: REDUCTION IN MEAN TIME TO REMEDIATE
Average
45+ Days
Good
21– < 30 Days
Better
< 20 Days
Best In Class: Organizations running excellent application security programs have an MTTR of 8 days for CVSS greater than 9.5 (CISA Recommends 15 Days).
Fix Rate is a percentage of attackable vulnerabilities that are fixed and an increase implies that vulnerabilities are being fixed faster than they are being discovered, reducing backlog and tech debt. Again, the Success Metrics dashboard allows organizations to measure and improve their MTTR for open source risk.
GOAL: INCREASE IN FIX RATE*
Average
30%
Good
50%
Better
65%
Best In Class: Organizations running excellent application security programs experience flattening Fix Rates after a “better” phase has been achieved.
* These are average fix rate values for applications that are actively managed and are an average over all SDLC stages. An actively managed application is an application that has been scanned at least monthly for at least three months and has had at least one vulnerability fixed in that timeframe.
Continuously Integrate Security into the DevOps Pipeline
Integrating security into the DevOps pipeline requires configuring automated tools to analyze components within the build process. This integration allows SCA tooling to participate in the build process through binary analysis and reachability analysis, enabling early detection of vulnerabilities.
Additionally, it creates the first feedback channel through the CI system and unlocks future capabilities around enforcement.
Automated Configuration
For organizations ready to jump right into Continuous Integration, Lifecycle SCM Onboarding can be skipped while still achieving easy onboarding. Enabling the following two capabilities allows CI integrations to provision a new Lifecycle Application pre-configured for feedback and automation within their source control workflows.
AUTOMATED CONFIGURATION CAPABILITIES
PIPELINE INTEGRATION CAPABILITIES
DevOps Pipeline Integration
Pipeline integration often requires deep collaboration with DevOps and development teams who understand the technical details of how their projects are built. Each programming language has its own approach to component management and application packaging. Sonatype provides comprehensive documentation to assist technical teams in configuring their CI scanners to ensure the most accurate evaluation.
While we recommend continuous integration for the best experience and results, organizations that are not yet ready to integrate into their DevOps pipelines can get a more limited integration through Hosted Repository Analysis or SCM Onboarding.
Continuous Monitoring
Continuous monitoring allows organizations to detect and address vulnerabilities in real-time, especially in applications that aren’t regularly scanned, which often pose the biggest risk. By continuously monitoring applications, security issues can be identified and mitigated promptly, reducing the risk of exploitation. Sonatype Lifecycle provides continuous monitoring across the software development life cycle (SDLC) to ensure ongoing security.
Implement Fast Feedback Loops for Vulnerability Management
Once integrated into the CI pipeline, applications benefit from real-time feedback of their open source composition. For vulnerabilities that are introduced during the normal process of software development, providing feedback within the SCA tooling prevents development against vulnerable open source that would need to be reworked when changing the vulnerable component. For vulnerabilities that are found in open source that have already been integrated, opening a Jira ticket allows product teams to prioritize remediation without major disruption to ongoing development.
Source Control Management
Modern developer collaboration centers around DevOps platforms such as GitHub, GitLab, Bitbucket, and Azure DevOps. Integrating within the Pull or Merge Request workflow allows organizations to “stop the bleeding” by catching new vulnerabilities before they are merged into the default branch.
By providing early feedback on feature branches, organizations prevent significant rework that would otherwise be required if security vulnerabilities are found later during the development process.
TASK MANAGEMENT CAPABILITIES
Task Management
When new vulnerabilities are discovered within applications, the next step is to prioritize their remediation. Product owners or technical leaders rely on task management systems to rank and address security issues within their development backlog. Creating a work item in these systems gives product owners the information they need to effectively prioritize the work and ensure it gets done.
FAST FEEDBACK PREVENTS RISK MANAGEMENT FROM BECOMING A BOTTLENECK TO SOFTWARE RELEASE CYCLES
Automate Remediation to Improve Efficiency and Reduce Risk
The ideal state for an SCA program is one where vulnerable components are replaced before they are introduced into the default branch of an application. When new vulnerabilities are discovered within existing components, they should be automatically updated to a non-vulnerable version.
By implementing fast feedback in previous pillars and leveraging Sonatype’s automation capabilities organizations greatly reduce the amount of development time required to remediate vulnerabilities. This state allows application security, DevOps, and development to easily manage their software supply chain.
Golden Pull Requests
When a vulnerable component is identified, a Golden Pull Request (PR) creates an automated pull request that updates the component to a secure version with minimal disruption to development.
Sprint velocity is highly guarded by product owners, so if a vulnerability requires additional research or if the upgrade requires code changes, it is likely best suited for a Jira ticket rather than a Pull Request. Golden Pull Requests, however, are analyzed to ensure that the newer version is API-compatible with the vulnerable version so remediation is as simple as clicking a Merge button.
Conclusion
Implementing a structured, incremental approach ensures that security is embedded throughout the development process, from integration to continuous monitoring and automation. So, why wait? Start applying these strategies to your SCA program today to strengthen your security posture and deliver secure, high-quality software faster.
