How much should the federal government worry about Log4j?

There is an old fable that talks about the circle of life in the plains of Africa where every morning a gazelle wakes up and knows that it must run faster than the lion or it will be eaten. The current Apache Log4j remote shell execution (RCE) exploit that is playing out during the writing of this blog post is a stark example of how that fable has some truth to it. I think a more realistic truth would change the gazelle's logic slightly to say that it doesn’t necessarily have to outrun the fastest lion, but rather the slowest gazelle.

Joking aside, speed is a big factor in your open source software (OSS) risk management and that is why achieving a high level of competency in DevSecOps. Maintaining a secure software supply chain that makes your risk visible in all stages of your software development life cycle (SDLC) is key. So the answer in this writer's opinion is that the Fed should be very, very worried about Log4j indeed.

Right now, there are companies literally scrambling with all IT hands on deck trying to figure out where the Apache log4j-core library might be lurking in their applications. Meanwhile, security teams are monitoring ingress and egress traffic to see signs that attacks have occurred. This is a nightmare scenario for many, and what is worse, is that to a certain extent it is an avoidable nightmare.

There were signs that hackers were taking advantage of the exploit early in December 2021, but it only took a few hours from the public announcement for the malicious activity to begin. Just ask the folks at Kronos how their day is going since their cloud service was taken down by a ransomware attack that some news outlets claim may have been due to the Log4j exploit. While the origin of the Kronos hacking is still being investigated, the Belgian government is the first to publicly announce they were infiltrated by unnamed attackers using the Log4Shell exploit.

The truth is zero days happen, and that isn't going to change. What should be changing is the ability to always know what is in your software and be able to quickly address vulnerabilities to stay ahead of the attackers. Some of the problems that are apparent in the current threat response are that many organizations have tools that only scan text files or in the worst cases no tools at all to make OSS risk visible. With scanning tools that just do name matching in build manifests or writing scripts to do keyword searches of your source control to find "log4j" in a file may not save you from being vulnerable.

Scans may miss Log4j!

The reason manifest scans are just not enough, as shown in the case of Apache log4j-core, is that it is often one of the transient dependencies that don't necessarily appear in a pom.xml or manifest file in source control. These kinds of very commonly used dependencies get pulled into the final application artifact at build time by a build agent like Maven that assembles all the needed components for you. That is part of the power of Maven and similar tools in other coding languages as they do the dependency work for you.

This is why we encourage scanning both manifests in source control and application binaries or full dependency manifests to ensure that you have an accurate software bill of materials (SBOM) that identifies all of the known risks in all of the components that find their way into your finished applications.

Who is affected?

Right now the full effect of the Log4Shell exploit is yet to be discovered but what we do know is that 100s of companies have identified that they are affected and some big names have suffered serious outages possibly to attacks based on the exploit. The urgently troubling part is that there is still active use of the vulnerable components as seen from their download statistics from public repo sources. Sonatype's Log4j dashboard harvests Maven Central data on how often the Log4j components are downloaded and shows us that 36% of the recent downloads of Log4j are still for the vulnerable versions. This is enlightening and very frightening at the same time since we now know how simple it is to perform the RCE exploit.

The government response

Historically government agencies have a reputation for being slow to react to vulnerabilities and often have to catch up to the commercial side of the industry. This time the federal government is pulling no punches or resting on its haunches for this mitigation. In a move that sent shockwaves through the federal space, the Cybersecurity and Infrastructure Security Agency (CISA) announced very specific directives that all affected applications must be patched by December 24th. That's a two-week deadline; pause for a minute and let that sink in a bit. When does the government get anything done so quickly?

The speed at which CISA and other federal cyber security organizations are pulling together and proactively addressing the log4shell exploit is a testament to the perceived danger to critical systems that the vulnerability poses. CISA provides a threat response page and also created a repository on GitHub that lists an unofficial listing of different vendor products and their status regarding the Log4j vulnerability.

The CISA Directives

The Emergency Directive (ED) 22-02 requires that federal agencies not under the Department of Defense (DoD) and Intelligence Community (IC) comply with several actions including taking inventory of all applications to identify what apps take input from the Internet, check for vulnerable products and libraries in their apps, and mitigate or remove any applications affected by CVE-2021-44228 (Log4Shell). The directive further requires federal agencies to submit a report on their affected applications and the actions taken by 5:00 PM EST on December 28, 2021.

An important factor in allowing these application owners under this directive to achieve the desired outcomes and identify or remediate their applications quickly will depend on if they already have a high level of DevSecOps competency and are using quality tools that can provide them true SBOM for their applications.

Can I get an SBOM?

We at Sonatype have been championing SBOMs for years and it is critically imperative that application owners have them readily available in the face of zero day attacks like the Log4shell exploit. We also champion the use of a policy engine that allows you to apply sets of rules relating to OSS risk and be able to scan and react to risks in all stages of the SDLC. If you don't use our full Sonatype product stack already, you might wonder how you can quickly get an SBOM. Great news, we provide a free scanner that will allow you to upload an application artifact or download the scanner and run it locally to discover any known OSS risks like Log4j.

What's next?

The timing of this zero day exploit is quite unfortunate with the ongoing world issues and right at the beginning of the US winter holiday season. Many of our customers were very well equipped in this battle with Log4j and others are still hard at work but it is safe to say that having a high level of competency in managing OSS risk in your software supply chain is the best defense against the next zero day or responding to the still-unfolding saga surrounding the Log4Shell exploit. 

The story continues for Log4j with the 2.16 update changing the default behavior for the Java Naming and Directory Interface (JNDI) API but then being subject to a denial of service (DOS) attack identified in CVE-2021-45105. As of the writing of this article the current suggested version for log4j-core is version 2.17. We at Sonatype will continue to focus on helping our customers navigate the shifting landscape with the power of our 65 person research team augmented by our AI/ML solutions to find ways to deliver actionable OSS risk intelligence even faster for Log4j and the yet unknown threats to come.

Many questions about possible similar attacks involving lookups and deserialization in similar APIs like JNDI have been floating around and people are wondering if more zero day exploits are lurking in critical applications. One article from darkreading.com points out that new vectors including websocket connections have increased the log4j attack surface more than previously known.

This is scary stuff and everyone in the Fed needs to be onboard with patching and mitigating this monumental exploit we are facing right now. If you are a federal organization that is looking for the right tools to help you secure your software supply chain now and in the future please do not hesitate to reach out to one of our sales reps or IM me on LinkedIn and I will be happy to point you in the right direction.