GitOps Challenges: When to Opt and When Not To

Post on July 19, 2021

Back to all posts

Like we have discussed in our other article series on GitOps, it is the process of implementing continuous deployment for cloud-based applications, especially Kubernetes applications. GitOps emphases on a developer-centric experience when operating the environment by employing simple tools used by developers, which includes Git and continuous deployment tools.

Some of the notable GitOps tools mentioned earlier are Flux, Git, Kubernetes, ArgoCD, Jenkins X, WKSctl, Gitkube, Helm Operator, Quay, and so on.

GitOps is a distributed version control system, which maintains source code and contains text, certificate, or configuration files. You can use Git to maintain all these files and collaborate with different team members.

The main purpose of GitOps is to have,

  • A Git repository comprising declarative descriptions of the infrastructure required in the production environment.
  • An automated process to match the production environment with the defined state in the repository.

When you deploy a new or update an existing application, you should only update the repository, and other processes will be taken care of by the automated process.

The main principles of GitOps are

  • GitOps places a Git repository at the center of the infrastructure declarations and uses it to manage the current and past state of the application infrastructure.
  • Git repository represents the current state of a production environment.
  • Git stores the state of the application.
  • Git helps you know what happens in the application.
  • Git operations, like creating a commit or opening a pull request, help you to change the state of the application.
  • Git flow controls the deployment, test, and rollback.
  • The cluster state should match the described state in the Git repository, once you change the Git state.
  • GitOps does not allow hand-rolled deployments, ad-hoc cluster changes, and live configuration changes. If you want any change, it must be committed to Git first.

GitOps applies the DevOps practices like version control and continuous integration /continuous delivery (CI/CD) to automating infrastructure. You can use CI/CD tools to replicate and roll back environments to previous versions from the repository. GitOps is a simple and effective infrastructure for smaller projects and teams.

Challenges of GitOps

  • When you use GitOps, everything in an application must be coded, which means that an application is going to a Kubernetes endpoint. The networking complexities, like exposing a new Kubernetes service change, should be included as a configuration along with the application code. Therefore, any deviation from the desired state that cannot be coded can be an issue for GitOps practices.
  • The infrastructure should be declarative and expressive of the final state to use GitOps. Currently, all the GitOps tooling is focused on the Kubernetes ecosystem. Therefore, it is expected that there is a capacity availability on a Kubernetes cluster or easily add the additional capacity. If you want to scale the infrastructure for the Kubernetes cluster and support resources, this can come outside of the declared state. For example, in Kubernetes, you can set the number of replicas, and when a scaling event occurs based on CPU and memory exceeds the replica, there will be a deviation.  
  • In Gitops, the deployment should be immediate, which may cause issues during risky releases where additional safety will be required in the form of incremental release, although, defining the multiple incremental states is against the scope of GitOps. 
  • The code update is part of a continuous delivery workflow and is automated to handle the changes. For large teams, there can be multiple automated processes writing to the same files simultaneously, which can cause conflicts that require human interference.

When to opt for GitOps

In this current digitally transforming world, software applications work at a large web scale and deal with cloud-native approaches such as microservices, containerization, and service mesh, etc. When you use containers and Kubernetes and the load is high, the applications should scale up the infrastructure running. When the load is low, the application should scale down, and dynamically in a complex manner across different applications and microservices. GitOps is used to overcome these operational challenges.

Following are the reasons to opt GitOps

  • Faster deployment: The Continuous Deployment technology helps for faster deployment with a feedback control loop and allows you to deploy at any time. Using GitOps, you need not have to switch tools for deploying your application, and it is controlled by the version control system used for developing the application.
  • Better developer experience:  Git helps developers to manager Kubernetes easily and icnreases the cproductivity of newly onboarded developers.
  • Reliability: Git helps you to easily return to a stable release in case of any meltdown, which reduces the recovery time.
  • Consistent: GitOps is very consistent as infrastructure, that is, a single model provides application, Kubernetes management, and so on.
  • Self-documenting deployment/environment: You can check the master branch to get a complete history of every change on the system and all the details of the deployment. It helps in easy collaboration with other teams or shares knowledge with a new member. GitOps provides an audit trail of any changes in the system.
  • Security governance and Compliance: GitOps helps large infrastructure environments to be secure and compliant. You can lock the permissions of the people having the approval to merge to a branch.
  • Easier Credential Management : Gitops permits you to manage deployments in the environment,which needs access to the repository and image registry.Developers need to have direct access to the environment. 
  • Easier and faster error recovery: GitOps provide a complete history of the environment changed over time, which helps for easier and faster error recovery.
  • Lightweight and vendor-neutral: GitOps is open source, and not resource-intensive, and can be used on an extensive range of platforms and setups.
  • Easier to change history: Git accounts for all the changes made, which helps you to use any standard Git interface to view an audit trail and revert the changes easily. GitOps allows you to inspect different sets of configurations on branches, and share them with other team members.
  • Immutable and reproducible deployments: Git tracks changes to configurations, which helps you to reproduce and immutable any build or deployment, and thus helps with Kubernetesto create the environments and build assets.

When not to opt for GitOps

Some enterprises are still not comfortable opting the GitOps way in production because of the immature approaches and thinking towards it. Let us see what are some thighs that bother companies when it comes to GitOps.

  • When you need a matured approach to software development: GitOps just covers the partial subset processes involved in the software development life cycle - Deployments, rollbacks, and configuration. What about the other processes like compilation, storage of artifacts, storage of images, static code analysis, etc? GitOps doesn’t answer many questions related to software development and release.
  • When you need more visibility: In an enterprise environment, the number of GitOps repositories and configuration files are higher. Thus, combing the text files is not a suitable option to address certain questions. For example, the question of how many times the applications are deployed cannot be answered in GitOps as Git repositories changes are difficult to map to application deployments. Certain changes can result in the deployment of multiple applications or a change in a minor configuration value.
  • Not suitable for programmatic updates: In a CD process, any application should be updated regularly, and this process should be automated. When the CI process ends and requests a release to the test environment, it has to create the pull request. But, GitOps is designed for manual editing and conflict resolution. The multiple CI processes can lead to writing to the same GitOps repository causing conflicts. 
  • GitOps can be difficult to scale: When you are a big company with having a large number of environments, repositories, and applications. Maintaining the commits and tracking everything can be difficult and can lead to configuration duplication.
  • When you want to authenticate input validation: If a Git repository is the interface between the Kubernetes cluster and the CI/CD process, you cannot verify whether the files are committed. For example, if you make an API call instead of a Git PR, the validity of the request is checked, but in GitOps user can decide to ensure that the manifest or Helm files are correct.
  • Difficult to Audit: The GitOps repositories store the complete history of all changes, which can be used for audit processes. However, a GitOps repository is a versioned store of text files. To audit any issue, it requires the complete Git history and complete search in text files, which is problematic to implement and leads to error.
  • When you would like to have more control over security: The complex enterprise infrastructure needs a solution to secure information outside of the normal CI/CD process. The information, like private keys or passwords, to access databases should be audited and kept in centralized secure storage. The Git repositories cannot store the information, as it has to be encrypted and decrypted. Also, the information Secrets is stored in the Git history. Lack of support for centralized secret management makes it not a good choice to opt for.

Conclusion

GitOps deployments have more advantages compared to other deployment approaches. GitOps is intended for declarative infrastructures and is becoming highly popular among cloud-native applications. The Git repository signifies the state of the system, and Git history is the same as the deployment history. The rollbacks in GitOps are very easy to execute by using only a previous Git hash. Although GitOps is not specific to Kubernetes, current GitOps tools work better with Kubernetes in the form of cluster controllers. The GitOps controller monitors the state of the Git repository, and when you commit, the cluster will match the same state. All things at GitOps make it an amazing option for Kubernetes-related application deployments.

 

Reach us at info@zelarsoft.com to book your Free Consultation today & Follow us on LinkedIn

Lets Talk and Meet If you have questions, suggestions, ideas, or just need further information about Zelar and our services, please do not hesitate to contact us in the way most convenient for you.

Enquire Now