Integrating Cyber Security into the CI/CD Pipeline

One of the most difficult challenges companies face today is how to be more flexible and adaptive in a dynamic business environment. ‘Speed to market’ demands identifying and capitalizing on opportunities as soon as they appear and has become a major driver for the adoption of DevOps in the financial market industry. These changes have paved the way for new design approaches such as Continuous Integration and Continuous Delivery (CI/CD) and Infrastructure as Code (IaC) based on automated configuration and provisioning.

GreySpark believes that this evolution in well-established practices raises serious security concerns about data privacy, data residency and data protection, compliance with numerous regulations and standards and restrictions impacting the financial services industry.

Security controls should be integral to the CI/CD pipeline and must address these intrinsic risks. The transition to the CI/CD pipeline therefore requires teams to automate the processes behind the building, testing and deployment of applications without compromising security. The emergence of CD orchestration tools and automated security scanning has helped to minimise OS code vulnerabilities and errors caused by manual packaging without blocking the CI/CD pipeline.

Configuration management tools such as Chef are currently available for deploying and scaling web applications and services. Various testing tools with out-of-the-box cloud services are used to build and run automated website tests such as Selenium, Ghost Inspector, Runscope and Apica, among others.  Those tools can be run continuously from the cloud, monitoring the websites for issues. The CI/CD pipeline can also accommodate a manual approval control that will pause the automated flow and will request an approver with appropriate identity and access management permissions for authorisation to proceed.

CD requires a test environment that is configured to match production as closely as possible and includes automated acceptance testing, security checks, stress and performance testing. It should be a fully auditable framework for the deployment of software upgrades, patches, configuration changes and new features. Undoubtedly, the test environment should be based on cooperation between cross-functional teams aiming to achieve an in-depth understanding of dependencies in their environment and standardising configurations for CI/CD pipelines and building their infrastructure as code (IaC) with security measures baked in.

Financial services firms will have also to ensure that the security and integrity of their DevOps environment complies with their regulatory requirements and adheres to the following principles:

• The removal of the siloed approach and the integration of the efforts of development, cybersecurity and operational teams to identify and resolve security vulnerabilities;
• Shifting security checks to early stages in the CI/CD pipeline;
• The automation of security testing and the utilisation of IaC;
• The standardisation of the runtime environment; and
• The implementation of security monitoring and feedback flows.

GreySpark recommends a secure DevOps approach (DevSecOps) that includes security controls that can be enforced by a wide range of currently available security monitoring tools including:

• Secure access to the version control system, continuous integration server, code repository, binaries and configuration files;
• Separation between the development and production repositories;
• A secure key and credential management system;
• The authentication and authorisation checks for all users accessing the pipeline based on segregation of duties principle;
• An audit trail of all events in the CI/CD pipeline for regulatory purposes; and
• The archiving of logs for records retention.

For successful implementation of the above controls, GreySpark recommends the following:

1. Development teams have to be trained in security best practices and threat modelling to identify and mitigate security risks. They should be able to prevent common security vulnerabilities during early design or testing stages. Any potential high-risk change should initiate automated tests as part of CI/CD pipeline and may trigger manual review and approval action by authorised parties in the Cyber Security team.
2. To transfer responsibility for application security to the development team, many organisations have wired automated security scanning tools such as BDD-Security, Gauntlt and Mittn into the CI/CD pipeline which can run on demand to identify security vulnerabilities.
3. Test teams must also be able to create automated scenarios for unit and integration testing based on tools as Chef Test Kitchen, Serverspec or rspecpuppet. Manual testing can also be used for adhoc “war game” scenarios focusing on bug hunting and user experience checks.

Security testing is one of the critical elements of software development process. Static Analysis Security Testing (SAST) tools such as plug-ins from Coverity, Klocwork, Fortify or Black Duck are gaining ground over the Dynamic Analysis Security Testing (DAST) tools which are usually run by intrusion testers and vulnerability analysts and are not tailored for CD environments. Best performing SAST software is able to work iteratively and incrementally, providing fast and concise pass/fail feedback. All gathered security testing data should be used for post-mortem analytics and the improvement of existing security policies and controls including those for the coding practices. It is also applicable to the open source (OS) code that may be introduced into any application via developer downloads, commercial apps or code reuse, outsourced development or third-party libraries.

It is critical to deploy security tools that can identify OS vulnerabilities in the CI/CD pipeline and fail the build workflow in case of a serious issue. This is particularly important due to the fact that OS software is readily available and widely accessible and OS code vulnerabilities can be easily exploited. Therefore, it is critical to deploy automated security scanning tools capable of capturing the OS weaknesses in real time thus reducing the gap between vulnerabilities being identified and being remediated.

Making standardised configuration changes to live infrastructure using Chef, Puppet and other tools in accordance with the pre-set templates will help lower the security and operational risks. Automated continuous checks can be set up as an online service for the detection of common security and configuration weaknesses or for the identification of non-compliance with pre-defined standards or configuration templates.

GreySpark believes that DevOps practices, if used efficiently and effectively through collaboration between development, operations and cybersecurity teams, can contribute to the timely closure of vulnerability windows as vulnerability prevention measures are no longer seen as blockers but rather as integral part of the CI/CD pipeline.