Organizations adoption of Cloud Infrastructure as a Service (IaaS) is ubiquitous.
This 4 part blog series aims to outline the key security considerations for organizations looking to adopt devOps and or are already utilizing Infrastructure services from Cloud providers viz. Amazon Web Services (AWS), Azure or Google Cloud Platform.
Depending on the crawl/walk/phase of the adoption different considerations outlined below would apply.
The blog series has been divided into following 4 parts with this being the 2nd part.
Part 1 – Security Consideration # 1 – Visibility, Continuous Controls Monitoring and Compliance
Part 2 – Securing CI/CD processes for a secure devOps ecosystem
Part 3 – Security Consideration #3 – Infrastructure Identity Governance and Administration
Part 4 – Security Consideration #4 – Privilege Access Management and Keys management
Security Consideration# 2 – Securing CI/CD processes for a secure devOps ecosystem
Infrastructure as code, Immutable Infrastructure, Phoenix Servers or Continuous Drift Management concepts are on the rise and being widely adopted by organizations moving over to cloud. CI/CD, devOps process and tools are helping organizations to realize these concepts. With infrastructure being represented as “code templates” and “not as physical entities” it becomes imperative to integrate with the CI/CD tools/processes to have a secure devOps ecosystem .
2A. Infrastructure as Code Security
With the advent of Infrastructure as Code, Infrastructure code configurations could reside in multiple repositories and in various forms. Securing Code templates across these sources is paramount. Code templates could be in the form of Cloud formation templates, Azure Resource templates, Terraform configurations, Chef cookbooks or recipes.
Securing these sources is as critical as securing the target infrastructure itself and involves discovering and continuous monitoring of-
- Who has access to these repositories/systems and code templates stored on them ?
- Context with which users have access to these resources (via Groups or explicitly assigned)
- What do the users do with this access (Collecting usage information to define baseline patterns)?
- Do these templates have security misconfigurations and how can those be discovered and remediated ?
Access Visibility on the code templates is paramount to secure IAAS services.
This could be considered complete only when the visibility cuts across tools viz. Chef/Puppet and source code repositories like GitHub and IaaS providers.
Further augmenting the security for cloud ecosystem requires scanning of these code templates on various sources.
For ex. An EC2 instance on AWS with open security group to internet is as vulnerable as a CF template on Github which has the same set of configurations for an EC2 instance and thereby both of them need to be discovered, continuously monitored and remediated.
This further needs to be followed by access governance with the necessary checks determining the overall access exposure. Attestation, Intelligent Access Request system, Preventive Policy and Segregation of Duty management are essential to ensure the security of Infrastructure as Code objects. These items would be discussed in part 3 of the blog series.
2B. Golden Image Baking
Baking images for automated deployments is a very important process to ensure the security of workloads and it also lays the foundation for immutable infrastructure.
At Saviynt, we have broadly categorized the image baking process into 4 steps –
- Selection of bare metal OS based image and make it ready for the pre-bake phase. Usually involves mounting of volumes and file system creation
- In the pre-bake phase application binaries, runtime libraries, any host level agents (For ex. OSSEC, EC2 Systems Manager agents etc.) for configuration management and specific agents to collect metrics and host-level fine integrity checks are packed (Packer is a great tool to achieve this) and pushed to the pre-baked image
- Third step comprises of hardening the OS per industry benchmarks viz. CIS (Center for Internet Security)
- Last phase usually involves submitting the image for scanning either to home-grown tools or 3rd party image scanners
Following a step based image baking process is important to ensure any and all workloads in the cloud ecosystem are built using the same code templates, configurations and adhere to same security standards.
Above all whenever needed, servers/VMs could be launched from scratch anytime replacing the old servers thereby laying the foundation of Immutable Infrastructure.
2C. Snowflaking:
Snowflakes are servers which keep evolving and changing from their first deployed state due to code changes, patching or configuration changes. Read Martin Fowler’s article on the concept of snowflakes.
To scale in the cloud, it is important to implement Immutable Infrastructure by adopting the Phoenix Server pattern.
Phoenix servers can be created from scratch by rebuilding them anytime. Any changes in the code, no matter how small they are, should require to deploy the complete application stack. This deployment model not only helps in cutting down the application troubleshooting efforts due to missed configurations on culprit servers, but also prevent insecure configurations or vulnerable variables getting injected on the servers. The idea is to build the entire application tier from scratch whenever required just like a “Phoenix Bird” rises from nothing.
2D. SSH-Less environment:
SSH-Less environment is a fundamental shift in the way applications have been designed, managed and deployed on traditional datacenter.
SSH access to servers are usually needed to view logs, stopping/starting services or deploying the application binaries/patches etc.
In the cloud scale this leads to several issues-
- Managing drift (app server configuration changes, application configuration, integration wirings like API URLs, serviceAccount IDs) on servers become extremely difficult
- Snowflaking of servers leading to inconsistent experience. This is due to an inconsistent environment with different configurations within an application cluster
- Possibility of injecting insecure configurations on servers
- Managing a secure process to distribute SSH keys/passwords to developers is tedious
- Rotation of Passwords/SSH keys would require the re-distribution of new SSH keys/passwords to developers
Creating a pure SSH – less environment requires a mindset change which could be further followed by re-designing and architecting the way applications are managed/deployed.
Centralized Logging, Pipeline based deployments using tools like Jenkins to achieve Immutable Infrastructure, creation of external interfaces/tools for devOps teams to run patches, start/stop services etc. are key elements to design and implement for moving towards SSH-Less environment thereby further strengthening the security posture of Infra services.
2E. Auto Scaling:
Concept of “static infrastructure” or to be precise “a static application tier” no more exists. With the concept of Auto Scaling application tier ceases to be static and thereby maintaining excel sheets based inventory is neither required nor should be exercised. Application Tier has to be auto scaled to reap the true benefits of Cloud’s Elasticity not only for scalability, costs and reliability purposes but it also serves as an important ingredient towards security of the infrastructure.
With Auto Scaling, organizations can enforce same security baseline standards across their workloads and the possibility of an insecure configured workloads being added to their environment by manual intervention becomes minimal.
However, making the application tier to use auto scaling requires significant changes in the application design which needs to be factored in.
Hope you all found this blog useful. Feedback and suggestions are always welcome. The next part will be focused on Infrastructure Identity Governance and Administration.
