DevSecOps Panel – Best DevOps Security Practices & Best Tools

Welcome to part 2 of our DevSecOps interview panel series. In case you missed it, check out part 1 of our DevSecOps panel to find out what DevSecOps is and what the challenges are when dealing with security in a DevOps setup. And now, let’s get stuck into the next questions.

JAXenter: If you look at the whole construct of an application life cycle from the outside, where is it most vulnerable, security-wise?

Bart Copeland, ActiveState CEO: The application life cycle is most vulnerable when builds go live into production. When it comes to production, how do you trust what you have running, in particular on a long-term basis? Once an application has undergone its verification procedure, usually in a quality assurance process – or at the very least an automated test suite, you are now limited with these results as being the final view into your application’s components. However, this last view lacks oversight on the challenges that occur in operations when configurations are different than other environments used in the software development process. In production, security, uptime, and stability are of paramount importance.

This is why baking in security at the time the code is written — and of particular note, when architectural decisions are being made — has a huge impact on security. At this time developers must have an understanding of their company’s security policies and standards so that security components can be chosen wisely at development time. All third-party open source components should be scanned for license compliance and vulnerabilities and these should be known during development, not after.

Robert Ross, Curtail CTO: Applications are most vulnerable in production, especially with software development life cycles becoming increasingly shorter. As companies switch to DevOps and CI/CD models to move faster and provide application updates sooner, software developers continually release new features and often push code updates as fast as they’re written. The classic six-month development timelines of dev, quality assurance (QA) and beta testing have been compressed to days and sometimes hours.

With current quality testing tools, developers often don’t know how a new software version will perform in production or if it even will work in production. The Cloudbleed bug is an example of this problem. In February 2017, a simple coding error in a software upgrade from security vendor Cloudflare led to a serious vulnerability discovered by a Google researcher several months later. Although Cloudflare still worked, the bug meant that it was leaking sensitive data.

Along with these short-term consequences, flaws can create long-term security issues. Heartbleed, a vulnerability that arose in 2014 stemming from a programming mistake in the OpenSSL library, left large numbers of private keys and sensitive information exposed to the internet. This enabled theft that would otherwise have been protected by SSL/TLS encryption.

Kaushik Mysur, Instaclustr Director of Product Management: Applications are most vulnerable wherever quality control processes are lacking, or where developers are able to point to unreasonable friction or “following the process” to excuse poor results. To eliminate these scenarios, it’s critical to successfully balance quality assurance with flexibility. Developers ought to feel empowered to own the quality of their releases, and responsible for issues under their purview. DevSecOps emphasizes this, as part of its manifesto, to ensure security is owned by all engineering teams from design to deployment and operations. It also emphasizes building features securely over dedicated security features (an important distinction). If a loosely coupled microservices application is built with a centralized security control, it is bound to invite security breaches.

Gary Duan, NeuVector CTO: Applications are most vulnerable in production, where they undergo higher profile exposure and attackers are more likely to target and test environments for vulnerabilities. That is where live data can be accessible and is valuable to hackers. That said, practicing proper DevSecOps to secure applications throughout development pays dividends in bolstering security during the production phase.

Wei Lien Dang, StackRox co-founder and VP of product: At runtime is arguably when applications are most vulnerable. While vulnerabilities in applications may be introduced or overlooked as part of the development process, it is only when they are running that actual exploits can materialize. Organizations have many different tools and best practices they can leverage to secure applications earlier in their lifecycle, and the trend in containerization towards more minimal components and fewer dependencies only further reduces the attack surface. At runtime, vulnerabilities coupled with possible misconfigurations leaves organizations susceptible to a much broader range of potentially unknown security issues, which is why continuous monitoring for all applications is a critical aspect of any security program.

The cloud-native stack will be no different from any other infrastructure – the majority of security compromises will continue to result from misconfigurations. It’s always been this way, and it doesn’t look like it’s going to change any time soon. Crypto mining attacks, for example, often succeed because a hacker gains entry to the control plane of the infrastructure, via an exposed Kubernetes dashboard, for example. The very attributes that can make cloud-native infrastructure so much more secure – all the controls native to the infrastructure – also present the greatest risk when they’re either not enabled or configured incorrectly.

SEE ALSO: Securing containers throughout the entire build-ship-run lifecycle means shifting left and right

JAXenter: In your opinion, are there best practices when it comes to security in the context of DevOps? What should developers, operators, and security experts always consider, even if DevSecOps has not yet become the paradigm of choice?

Bart Copeland, ActiveState CEO: With the rise of DevOps and increasingly shorter software cycles, the only way for organizations to maintain open source security is to automate. My team and I have defined four phases to Open Source Language Automation. These are the tenets by which organizations and their developers can fully benefit from the full potential of open source languages while minimizing security risks. The four phases and their resulting outcomes are:

• Define Open Source Language Policies: Programmatically enforce open source language policies to ensure compliance against the entire organization.
• Centralize Open Source Language Dependencies: Track languages and packages across DevOps to assess open source usage.
• Automate Open Source Language Builds: Reduce vulnerabilities and increase application quality.
• Deploy and Manage Open Source Language Artifacts: Automatically update all of your test, stage and production servers with the appropriate and latest open source language builds.

 Even if DevSecOps has not been implemented, applications should still be architected and designed with security in mind. This means that all possible threat scenarios are considered up front. And mitigating controls and compliance requirements are built-in to counter threats before they occur. Shifting your security left gives you a forward view of security. This means that more secure applications are produced, costly inefficiencies are avoided and organizations receive a better return on their investment.

Robert Ross, Curtail CTO: It’s just too risky these days to push software and services live using old-school QA testing, especially for CI/CD rollouts. Software teams need a way to identify potential bugs and security concerns prior to release, with speed and precision and without the need to roll back or stage. A much better method is to test in production and evaluate release versions side by side. By simultaneously running live user traffic against the current software version and the proposed upgrade, users would see only the results generated by the current production software unaffected by any flaws in the proposed upgrade. Meanwhile, administrators would be able to see how the old and new configurations respond to actual usage. This would allow teams to keep costs down, while also ensuring both quality and security, and the ability to meet delivery deadlines – which ultimately helps boost return-on-investment.

From a security perspective it’s important to manage the risks associated with third party and open source components. Upgrading these components requires time and effort to make the update and verify that it’s stable, but not upgrading these components risks allowing flawed software to continue to run. Keeping software updates current or having alternative software stacks configured and prepared provides the best defense when zero-day attacks become known.

Kaushik Mysur, Instaclustr Director of Product Management: One key DevSecOps consideration that teams should explore is the opportunity presented by automation. It’s now possible to greatly automate security and avoid limiting the pace of development. Security management tools and others can be leveraged to implement CI/CD pipelines that are automated, secure, and result in secure applications with minimal exposed threat vectors and vulnerabilities. Also, there’s tremendous value in utilizing security-hardened managed services and open source tools, which can reduce the burden on internal teams to expedite development without compromising security. Where possible, leverage them rather than building everything ground up securely.

Gary Duan, NeuVector CTO: As a basic principle, security should be a constant consideration at every stage of the application lifecycle. For a specific example of best practices, the initial development phase should include implementations of technologies providing build-time and registry image scanning, digital signing, each of which can act as preventative medicine removing issues that would be much more difficult to address later. The test phase should include efforts to reduce attack surfaces by removing vulnerabilities and hardening service and workload configurations, such that software has an established “known good state” before entering production. Teams must then be equipped to defend applications in live production environments, where DevSecOps faces its greatest challenge.

Wei Lien Dang, StackRox co-founder and VP of product: These different teams need to agree on shared responsibilities and how relevant information will be exchanged to more easily operationalize security across applications and infrastructure. Finding ways early on to automate basic security practices and utilize programmatic controls is critical to establishing the basis for more complex operations.

SEE ALSO: 5 DevOps trends everyone should get ready for in 2020

JAXenter: Which existing security tools do you recommend? 

Bart Copeland, ActiveState CEO: Obviously, I’d recommend the ActiveState Platform. As the industry’s first SaaS solution for open source language automation, it delivers open source code security and build engineering that empower developers to catch security vulnerabilities throughout the entire software development lifecycle (SDLC). We’ve designed the Platform so that with minimal knowledge, a developer can automatically build from any open source language runtime, resolve all dependencies, and then certify it against compliance and security criteria in just a few minutes.

Robert Ross, Curtail CTO: Our product, ReGrade, enables DevSecOps teams to simultaneously run live user traffic against the current software version and the proposed upgrade, to quickly uncover security issues, differences, bugs and other defects in quality. This sophisticated network traffic analysis elicits vulnerabilities and bugs, allowing organizations to make corrections before final deployment and reduce risk. By accelerating development while stopping vulnerabilities before they occur, ReGrade keeps systems protected and running in the face of attacks, unplanned downtime and equipment failure.

Additionally, two-factor authentication providers such as YubiKey are a powerful way to secure user identity and prevent credential theft attacks.

Kaushik Mysur, Instaclustr Director of Product Management: Importantly, there is no one security tool that fits all. Depending on the application you are building, your deployment environment and network topology, a range of tools are available (and smart) to use. For instance, if you are looking at securing your containerized app on Kubernetes, here are some tools to check out: CIS Benchmarks, Network Policy, Istio, Grafeas, and Clair.

Gary Duan, NeuVector CTO: For one, our own. We provide a full lifecycle container security platform which automates many of the security checkpoints to protect containerized environments throughout the application lifecycle. Security must be built in end-to-end from vulnerability management to run-time protectioin. We are in a unique position to protect the run-time environment with a Layer 7 container firewall that provides deep packet inspection, threat detection, connection visualization and monitoring. Security is automated by baselining the network connections, processes and file system activity in all containers, enabling NeuVector to flag any abnormal behavior, thereby detecting attacks at the earliest stages of the kill chain. We also help DevSecOps teams in aligning their environments with security standards, including Docker Bench and Kubernetes CIS Benchmarks.

For enterprises that want to shift security left beyond their container pipeline, we partner with companies such as jFrog, Synopsys and Sonatype to provide deeper code analysis and pipeline management and security.

Wei Lien Dang, StackRox co-founder and VP of product: We recommend tools, either open source or commercial, that can be easily integrated into DevOps toolchains and CI/CD pipelines to reduce vulnerabilities and enforce policies earlier in the application lifecycle. For containerized applications, this includes image scanning solutions with the ability to fail builds or prevent deployments based on pre-configured or customizable policies. Modern infrastructure platforms such as Kubernetes offer native security capabilities which always provide a good starting point to enable security best practices and ensure scalability, consistency, and portability.

Thanks very much!

The post DevSecOps Panel – Best DevOps Security Practices & Best Tools appeared first on JAXenter.