Part 1 - Is DevOps the Wild West? - The Culture of Security in Development
By: Troy Hagerty
What do we mean exactly by "Wild West"? If you do a quick Google search about source code and such keywords like "breach", you can see for yourself what major organizations/projects were victimized in the past couple of years:
We are calling it out. DevOps and Security teams don’t seem to have the same priorities.
We’ve seen it with numerous clients, and we think we know the root cause of the problem:
There appears to be a cultural divide between DevOps and SecOps.
Out of curiosity, is Security front of mind in your Development team?
Many DevOps Environments have Huge Security Risks Out in the Open
Breach after breach, we see the world of Development (often known as “DevOps” or software coding), clearly being exposed to the elements (hackers, ransomware, etc.). We also witness the minimization of the severity of these breaches. Do they really matter? Are they really resolved once companies say they are?
The attackers get in, access source code, but don’t change anything, thus not affecting the functionality of the processes in place. Despite that, what some may neglect to notice is that the attackers still were able to copy all the code, thus exposing intellectual property to other outside parties and nefarious actors.
With knowledge of how the code functions in hand, they’d be able to use it as a baseline for developing future attacks, which would be much more devastating to the organization if they manage to figure out how each logic flow works and how each element calls to one another. If an attacker does manage to figure out that logic flow and how the internal systems talk to one another, they have the keys to the kingdom.
According ZDNET, software development hasn’t kept up with the same pace as network and application security has. “The accelerated pace of work that we live in contradicts most security teams’ best practices.” It appears that the incentive to release updates, which meet business needs, outweighs secure practices.
What makes DevOps good at Development?
Brad Palchesko, Senior Developer at a major streaming platform, had a chat with us about the culture of development. We wanted to get an idea of what drives Dev teams. He had the following insights:
"Developers want to be free. They don’t want any delays or roadblocks in their creative process. The culture of DevOps runs deep, with its roots tied into open-source communities, artistic expression, sharing of information, and organic innovation and manifestation."
Freedom vs Security
In our experience with observing DevOps environments and discussions with developers around the industry, we see a clear divide in the following manner.
Freedom. Developers want the freedom to collaborate with others to improve the quality of their code, as well as aid in their creative capabilities. Software developers have long optimized the benefits of open-source tools, working together on projects, seamlessly integrating applications as the key to growth of the internet and improvement of our daily lives. Both developers and reviewers with access to repositories in the same environment can both pull and push in these environments. Most developers have access to every code repository. This allows for improved efficiencies and quality, but it can sacrifice security.
Security. With a newly released standard that replaces the one approved in 2013, ISO/IEC 27001:2022 requires clear separation of development, test and production environments (8.31). It also requires security code reviews for all changes (8.32). Technical testing (vulnerability scanning and penetration tests) shall be done periodically within the development environments (8.8). Secure coding principles shall be written and enforced (8.28). Test data and operational data shall be segregated to protect the environments from using live data (8.31). Static and dynamic code scans should take place (8.22). Access to program source code shall be restricted to users on a need-to-know basis (8.4).
In the DevOps environments we have assessed and researched, security is highly lacking to the degree which critical system breakdowns are evident in almost every environment we have observed.
We have also frequently noted that some organizations have developers using private and uncontrolled Github accounts to perform their job functions (Oh no!). Furthermore, outsourced providers (freelancers) get unfettered access to code without restriction, while vendor contracts and policies are not enforced, nor are these Dev services logged and monitored for nefarious user activity.
The high-level Dev/Sec Issues are broken down as follows:
As noted above, Freedom vs Security provides a conflict between DevOps and Security.
Lack of skilled resources exist in the marketplace. Also, without efficient processes and clearly understood security policies and procedures, insecure practices may result in Dev environments.
Companies are slow to change existing practices, but quick to meet business needs. Changing organizational culture is not always at the top of the to do list. Furthermore, orgs spend more time doing and less time preparing for failures, such as contingencies and business continuity. What happens when 5 people leave? What happens when a critical SaaS provider goes down for a month?
Many organizations don’t know what they don’t know. They may also not have the financial means to implement technologies to support the DevSecOps control environment and infrastructure. Vulnerability management, penetration testing and other security tests can be cost prohibitive.
Can DevOps Environments be Secure?
We believe so, and this can be done via mostly non-technical means.
Development and Security organizations require a coordinated response to information security threats, incidents, as well as secure system architecture and design. Merging these environments into DevSecOps requires intuitive cultural shifts customized specifically to the organization, its business objectives, limitations and strengths.
ARORA strives to bridge this gap, while offering DevOps enough freedom to create and offering Security enough visibility to anticipate and respond.
Where can ARORA bridge the "gap"?
Cultural shift towards increasing Security-mindedness within functional teams
Training on Security as well as cross-training of functionalities
Resourcing – People, layers of people needed to perform basic quality functions
Tech leadership focused on Security and Quality Assurance
Built-in Security Controls and Quality Control Processes
Technical Security support systems (DevSecOps)
Check out Part 2 for some of the biggest problems in the DevOps and their relationship with security