Equifax officials said today that its massive security breach was possible via unpatched web application server vulnerability Apache Struts CVE-2017-5638, confirming what some in the security community expected to be the case last week when the news first broke.

In an update to its FAQ page on EquifaxSecurity2017.com, officials said it has been working with an independent cybersecurity firm to determine what information was accessed and which customers have been impacted.

Equifax announced last Thursday that personal information belonging to 143 million customers was accessed by hackers, in addition to credit card numbers for about 209,000 consumers. Beyond facing its customers wrath in the days that have followed, Equifax is now also subject to an FTC investigation. 

“We know that criminals exploited a U.S. website application vulnerability. The vulnerability was Apache Struts CVE-2017-5638. We continue to work with law enforcement as part of our criminal investigation, and have shared indicators of compromise with law enforcement,” Equifax said.

Apache Struts CVE-2017-5638 was made public on March 7, 2017, and a patch was made available that day. In a statement today, Apache said “the Equifax data compromise was due to their failure to install the security updates provided in a timely manner.”

Equifax discovered the breach on July 29 and didn’t disclose when it sought to patch the flaw, Bloomberg says.

In a blog post by Contrast Security CTO Jeff Williams last week, he said that while ensuring you don’t use libraries with known vulnerabilities is a good practice, it is not easy since changes come out frequently.

“Often these changes require rewriting, retesting, and redeploying the application, which can take months. I have recently talked with several large organizations that took over four months to deal with CVE-2017-5638. Even in the best run organizations, there is often a gap of many months between vulnerability disclosure and updates being made to applications,” Williams wrote.

A statement from Apache Struts VP René Gielen on Saturday, before Equifax confirmed what caused the security breach, said:

“We as the Apache Struts PMC want to make clear that the development team puts enormous efforts in securing and hardening the software we produce, and fixing problems whenever they come to our attention. In alignment with the Apache security policies, once we get notified of a possible security issue, we privately work with the reporting entity to reproduce and fix the problem and roll out a new release hardened against the found vulnerability. We then publicly announce the problem description and how to fix it. Even if exploit code is known to us, we try to hold back this information for several weeks to give Struts Framework users as much time as possible to patch their software products before exploits will pop up in the wild. However, since vulnerability detection and exploitation has become a professional business, it is and always will be likely that attacks will occur even before we fully disclose the attack vectors, by reverse engineering the code that fixes the vulnerability in question or by scanning for yet unknown vulnerabilities.”

In the post, Gielen outlines five best practices for using open or closed source supporting library in software products and services:

1. Understand which supporting frameworks and libraries are used in your software products and in which versions. Keep track of security announcements affecting this products and versions.

2. Establish a process to quickly roll out a security fix release of your software product once supporting frameworks or libraries needs to be updated for security reasons. Best is to think in terms of hours or a few days, not weeks or months. Most breaches we become aware of are caused by failure to update software components that are known to be vulnerable for months or even years.

3. Any complex software contains flaws. Don't build your security policy on the assumption that supporting software products are flawless, especially in terms of security vulnerabilities.

4. Establish security layers. It is good software engineering practice to have individually secured layers behind a public-facing presentation layer such as the Apache Struts framework. A breach into the presentation layer should never empower access to significant or even all back-end information resources.

5. Establish monitoring for unusual access patterns to your public Web resources. Nowadays there are a lot of open source and commercial products available to detect such patterns and give alerts. We recommend such monitoring as good operations practice for business critical Web-based services.