The cultural movement that is DevOps — which, in short, encourages close collaboration among developers, IT operations, and system admins — also encompasses a set of tools, techniques, and practices. As part of DevOps, the CI/CD process incorporates automation into the SDLC, allowing teams to integrate and deliver incremental changes iteratively and at a quicker pace. Together, these human- and technology-oriented elements enable smooth, fast, and quality software releases. This Zone is your go-to source on all things DevOps and CI/CD (end to end!).
"DevOps is dead." Well, not exactly. But the DevOps methodology of "you build it, you run it" has been failing development teams for years. On this week's episode of Dev Interrupted, we sit down with Kaspar von Grünberg, founder and CEO of Humanitec. Listen as Kaspar explains the significant cognitive load placed on developers as a result of DevOps practices, how that has caused software engineering to be the only industry since Medieval times not to drive towards specialization, and why platform engineers provide a solution to the outdated DevOps model. Episode Highlights: (2:24) What is platform engineering? (7:05) Should VPEs have a platform team right now? (11:29) Difference between SREs and platform teams (17:14) DevOps is dead (19:11) How scale affects team size (26:12) Standardization of the space (28:08) Kaspar's work at Humanitec (32:30) The future of platform engineering Episode Excerpt: Dan: If I'm a VP of engineering right now, listening to this pod, should I have a platform engineering team? Kaspar: Yes. I mean, no, you can always argue a customer has an interest of you having a plethora of engineering team, but I am looking at the return on investment of these teams. And there, it's so large, you can gain so much from this. There's so much inefficiency in these workflows that, yes, you definitely should have one. And having a platform engineering team doesn't like sounds, you know, more costly if you want that. It is- take a product manager, halftime if you want. But structure this correctly, structure this as a product, find a couple of people that are responsible for this, take them from different groups, you don't need me to rehire, apply these principles, you know, take this on structure. And you'll see a very, very fast return with fairly low costs. And so definitely, definitely yes. And I want to get back to one of the absolutely correct things you said. We have these fundamentalists shouting at us. You know, everybody has to do everything in context. Otherwise, you're abstracting people away. And those are these. It's this type of thing you can always say. Everybody could always say that they say never restrict developers, never take away from this. Of course not. But that's not the idea. Like platform engineering is not about taking context away, the contrary holds true. It's about providing context. If you're looking at 700 different script formats, that's not context. That's cognitive overload. You don't win anything. And so that is really like, our industry is the only industry that is not actually driving towards specialization. Since the medieval ages to now. Every industry has always specialized. We're the only fucking industry in the world that is actually working against specialization. I already have a problem with these fundamentalist approaches or viewpoints that many of them just have never really worked at scale. And scale for me means, like, production-grade two fifty-three, four or five hundred engineers over a longer period of time. And to believe that in these situations, you can just shift everything to everybody is so insanely naive. And then the next argument I always hear is like, Hey, you build it, you run it, Werner Vogels said so, I mean, let's pause and look at the situation where Werner Vogels said that he said in a blog post, 2006. That guy was in the CTO position for a UX director of research for like 12 months before he was a researcher. That guy had never worked at scale. And Amazon's teams, at that point, were a couple of dozen developers. The sentence that this guy said in 2006 says nothing about the reality of a bank in the US East with two and a half thousand developers that are drowning in policy. It's just naive to say that. It doesn't make sense.________ Join Kaspar and I at PlatformCon 2023 on June 8th and 9th - it's virtual and free!
Argo CD is a continuous delivery (CD) tool that has gained popularity in DevOps for performing application delivery onto Kubernetes. It relies on a deployment method known as GitOps. GitOps is a mechanism that can pull the latest code and application configuration from the last known Git commit version and deploy it directly into Kubernetes resources. The wide adoption and popularity are because Argo CD helps developers manage infrastructure and applications lifecycle in one platform. To understand Argo CD in-depth, we must first understand how continuous delivery works for most application teams and how it differs from GitOps. What Is Continuous Delivery? Continuous delivery (CD) is a software delivery process/framework that enables developers to continuously push their code changes into the production servers without accessing the infrastructure through a push-based pipeline. This process reduces the time a code takes to reach its end users and improves release velocity. It is a repeatable process that enables scaling your application to address the growing demands of end users. How Does Continuous Delivery Work? Continuous delivery is a part of the application delivery lifecycle that deploys the application post-build into a resource. In this context, our infrastructure will be Kubernetes without the use of Argo CD. Alternatively, if we use Jenkins, the process will be as follows: To release a feature upgrade or a bug fix to the end users, a pre-configured continuous integration (CI) pipeline will build a Docker image as per the Docker file.Ops Then, a script will push it to the configured Docker repository. To move this image into Kubernetes, the deployment.YAML file will be updated with the new image tag and name for fetching the latest image from the Docker registry. Challenges With CD Into Kubernetes Before the rise of Kubernetes, CD most applications teams enabled CD with tools like Jenkins and Spinnaker. The architecture of Kubernetes is complex, and these tools could not efficiently deploy into Kubernetes and deploy without errors due to the complexity of Kubernetes architecture. These issues made using tools like Jenkins challenging to work with Kubernetes. Challenges with traditional CD while deploying into Kubernetes: Tools Installation and Management One needs to install tools like Kubectl and Helm. These add to the operational activities. Accessing Kubernetes Clusters One must configure access management in the CD tool to enable authorization to Kubernetes clusters and execute changes. If Kubernetes clusters run on a cloud provider, those credentials must be configured and shared outside the cluster. Security and Operational Challenge The configurations will rise in proportion with the increase in clusters which increases operational overhead. While increased operational overheads may not be challenging, it risks the system's security when cluster credentials get shared with external services and tools. Issues While Scaling Infrastructure Each team needs its own set of Kubernetes cluster credentials so that the users can access that specific application resources in a cluster. While operating Kubernetes at scale with a CD tool like Jenkins deploying into multiple clusters needs reconfiguration again for each new cluster. Lack of Visibility When a CD tool deploys an application into Kubernetes without GitOps or a Kubernetes-native CI/CD pipeline, the tool loses visibility into the deployment post applying the configuration to the deployment.YAML files. Once the kubectl command has been executed team must wait until someone reports an incident. Also, the status of execution remains unclear. The continuous delivery into Kubernetes can be made efficient with Argo CD, which works on the principle of GitOps. Before understanding GitOps, let us understand push vs. pull-based CI/CD in the upcoming section on GitOps. What Is GitOps and How Is It Different From Traditional CD? Traditional CD and GitOps differ on the core principles of push and pull-based deployments. Most CI/CD processes work on a push mechanism, which means things move to their respective destination at the trigger of an event. For example, when a developer has finished writing his code, he must execute a set of commands to move his code into the server for deployment. In a Kubernetes environment, the developer has to configure the clusters using tools like Kubectl and Helm in the pipeline to apply changes. Argo is a CD tool that uses a pull-based mechanism. A pull-based CD mechanism means the destination triggers an event to pull the data from the source (Git) to deploy at the destination. Argo CD, which resided inside the cluster for reasons explained later on the blog, pulls the most recent verified version of code into the cluster for deployment. There are a lot of benefits to this model, like improved security and ease of use. This pull-based mechanism is called GitOps, where the source code management systems like Git are treated as the only source of truth for application and configuration data. How Does Argo CD Work? Argo CD works in a reversed flow mechanism as compared to push-style deployments. The new mechanism enables Argo CD to run from inside a Kubernetes cluster. Kubernetes faces challenges with the traditional CD mechanism because CI/CD tools, like Jenkins, sit outside the cluster, whereas Argo CD sits inside the cluster. While inside the cluster, Argo CD pulls changes from Git and applies them to the residing cluster. Instead of pushing changes like older generation tools by being inside the cluster, Argo CD prevents sensitive information from being exposed to other tools outside the Kubernetes cluster and environment. Argo CD can be set up in two simple steps: Deploy the Argo CD agent to the cluster. Configure Argo CD to track a Git repository for changes. When Argo CD monitors change, it automatically applies them to the Kubernetes cluster. When developers commit the new code updates to the Git repository, automated CI pipelines will auto-start the build process and build the container image. Then as per the configurations, the CI pipeline will push and update the Kubernetes manifest files. The pipelines will update the new image version name and details on the deployment.yaml file. Argo CD can track this new update, pull the image, and deploy it onto the target cluster. When the Kubernetes cluster is ready, Argo CD sends a report about the status of the application and that the synchronization is complete and correct. Argo CD also works in the other direction, monitoring changes in the Kubernetes cluster and discarding them if they don’t match the current configuration in Git. Best Practices To Follow While Using Argo CD Separate Git repositories for application source code and app configuration Separate Git repo for system configurations Why Separate Repositories? The main reason for having separate repositories for application source code and app configurations is because app config code is not only present in the deployment file but also in the configmaps, secrets, storage, svc, etc. Kubernetes uses. These files change independently from the source code. When a Developer or DevOps wants to change a service.yaml file, which is an application configuration and not a part of the software code, he has to run the whole CI pipeline to sync those changes into production. The CI pipeline will run only when a code change is updated. Clubbing the app configurations and software code together makes the setup complex and inefficient. So as soon as DevOps changes the config on the git repository, Argo CD will become aware of the changes and update the destination cluster as it constantly monitors the repo as soon as config files change in the Git repository. Benefits of Using Argo CD K8s configurations can be defined as code in the Git repository. Config files are not applied for individual laptops of developers/DevOps. Updates are traceable as tags, branches, or pinned specific manifest versions at Git commits. Same and only interface for updating the cluster Git as the single source of truth Avoid untraceable kubectl command applications Version-controlled changes with audit trail Single sign-on (SSO) with providers such as GitLab, GitHub, Microsoft, OAuth2, OIDC, LinkedIn, LDAP, and SAML 2.0 Support for webhooks triggering actions in GitLab, GitHub, and BitBucket Easy Rollbacks If a new code commit is pushed into the Git repository and the changes are applied to the cluster with Argo CD auto sync, the cluster fails. DevOps can revert to the previous working state from the list of the last known best repository versions, just like a Windows restore point. Also, one need not process the laborious process of manually riveting every cluster and doing a clean-up, as Argo CD will do that all by itself. Avoid Snowflake Clusters With Argo CD Argo CD is watching the Git repository and changes in the cluster. Anytime a change happens in the Git repository or the cluster, Argo CD will compare the two states to check for any misconfigurations or differences. If they don't match the desired state defined in the Git repo with the actual state of the cluster, Argo CD will become active and quickly sync the cluster as described in the cluster. So, in that case, if someone goes and makes manual changes in the cluster, Argo CD will detect it and sync to the desired state. It overrides the manual changes. The automatic override helps the system stay stable and guarantees that the Git repository is the only source of truth at any point in time. It also provides full transparency of the cluster and lets it become a Snowflake cluster. Recreate Clusters From Scratch When a cluster completely crashes, and one has to build it from scratch, Argo CD can be pointed to the Git repository, where the complete cluster configuration is defined. It will recreate the same state as the previous one. This is a fully autonomous process where developers and DevOps do not have to worry about disaster recovery post-clean-up processes. This is possible because Argo CD accepts the cluster configuration as code in a declarative way. What Does Argo CD Provide Over Standard GitOps Benefits? Better Team Collaboration With Easy Access Control Production clusters must have limited access, and only some of your team members should be allowed access. To configure different access rules to these clusters, Argo CD can enable approvals for pull requests for authorized developers and engineers. This helps in managing the cluster permissions. No need to create a cluster role and user account on Kubernetes. Non-human users like CI/CD tools or other peripheral tools in the DevOps ecosystem outside the cluster can be configured in Argo CD, which resides inside the cluster. This architecture of Argo CD ensures that those credentials remain inside the cluster making the system robust and secure. Argo CD Architecture Overview Argo CD is a Kubernetes-native CD tool that supports and reads various Kubernetes manifest files such as YAML, Ksonnet, Jsonnet, Helm charts, and Kustomize. It can follow updates to branches, tags, or pinned to a specific version of manifests at a Git commit. Argo CD control plane consists of three main components: Application Controller API Server Repository Service Application Controller Argo CD can detect changes in Git and sync with the repo due to the Application Controller component. This feature enables syncing out-of-date or modified destination configurations to the last approved Git version. The application controller syncs between the local cache created by the repo service and the Git repository because it is a less resource-intensive process. The application controller can also be configured to accept direct changes in code and configuration at the destination without reverting back to the last known configuration on Git. This authority must be granted only to a selected team resource. When this direct change is made, it notifies the DevOps and developers about the difference in the configuration to update the Git repositories. API Server The API server, like a Kubernetes API server, is a service to expose the components of Kubernetes and Argo CD to interfaces like a CLI or web GUI or other third-party tools. The APIs are primarily used to carry out functionalities like application deployment and management, executing rollback of any user-defined actions, managing cluster credentials stored in K8s secrets, and enforcing RBAC Git webhooks. Repository Service Accessing the Git repository is always time-consuming for Argo CD; accessing the Git repo every time will constitute a pull request. Hence, this internal Repository Service makes a local cache of the application manifest and Git repositories on the Kubernetes cluster a replica of the Git repository. It is responsible for generating and returning Kubernetes manifests on input data like repository URL, Git revisions (i.e., branch, tags), application path, and template-specific settings; the server generates Kubernetes manifests.
DevSecOps, in layman's language, is a combined form of software development, security, and software operations. According to Gartner's research, "It is estimated that at least 95% of cloud security failures through 2022 will be the fault of the enterprise". Therefore, while developing any application, the developer must not have loose ends that may make an enterprise vulnerable to such attacks. Similarly, DevSecOps is understanding the software and learning to code while learning to operate and maintain that code at the same time. It is essential to keep in mind that a single security breach can lead customers to lose confidence in any business. Therefore, it is vital to prioritize the maintenance of rigorous security measures. DevSecOps involves integrating security into both application development and operations, as well as promoting collaboration between teams and leveraging automation and tooling to construct robust and secure applications. In the DevSecOps approach, security is addressed proactively during the development process rather than as an afterthought. Security testing and bug fixing are integrated into the development cycle to detect security vulnerabilities early in the software development life cycle. This approach facilitates innovation, boosts developer velocity, and allows for speedy release cycles while maintaining a focus on security. DevSecOps has proven beneficial for achieving faster development, more rapid feature releases, and the implementation of agile practices. By integrating security into the development process from the start, DevSecOps helps to reduce the risk of security breaches and other cybersecurity threats that can be costly to organizations in terms of reputation, legal liabilities, and financial losses. DevSecOps also helps to promote a culture of collaboration and communication between teams, which can lead to better alignment of security and development objectives. It can also help organizations to meet compliance requirements, as security is integrated into the development process from the beginning. Overall, DevSecOps is essential for any organization that wants to build secure, reliable, and high-quality software products that meet the needs of their customers while minimizing security risks. Despite the progress that enterprises have made in adopting modern business practices, such as transitioning to cloud providers and utilizing agile frameworks, DevSecOps is frequently overlooked by stakeholders in terms of organizational priority. There is often a lack of a clear framework for DevSecOps initiatives that executives can readily support. Gartner forecasts that up until 2022, 75% of DevOps initiatives will fail to meet expectations due to difficulties in organizational learning and implementing changes. Adopting DevSecOps The process of enterprises implementing DevSecOps is a long-term undertaking that spans multiple years, and initiating it early can be advantageous for the organization in the long run. While there is a wealth of resources available to promote awareness of DevOps and its benefits, there is comparatively less information available on DevSecOps and a comprehensive framework that organizations can use to smoothly integrate DevSecOps into their operations. Below are some of the key steps to keep in mind as you begin your DevSecOps journey. 1. Do We Need DevSecOps? The initial step in embarking on a DevSecOps journey is to gain a complete understanding of what DevSecOps entails and why it is necessary. Once this comprehension is established, the focus should shift to assessing who will benefit from the adoption of DevSecOps and how. This will necessitate a thorough evaluation of the business use case, available resources, and the organization's current pain points. During this phase, it is essential to be transparent about any existing technical debt, defects, and bugs, as this will aid in identifying areas for improvement and opportunities to pinpoint the root cause of defects. This process will enable the identification of gaps in current applications and processes and provide insights to evaluate available opportunities. 2. How Do We Promote/Champion It? The subsequent step involves identifying a group of ambassadors or champions who are aligned with and committed to the DevSecOps mission within the organization. This presents an opportunity to recruit enthusiastic individuals who can bring fresh perspectives to the table. Multiple channels should be utilized to promote the opportunity throughout the organization to attract a diverse group of individuals, including engineers, operations personnel, security specialists, testers, and managers. Ideal candidates should be motivated, eager to learn, adaptable to ambiguity, and adept team players. This cohort will assist in bringing the DevSecOps mission to life and advocating for the cause, facilitating more widespread DevSecOps adoption throughout the enterprise. 3. DevSecOps Strategy To implement an organization-wide change with DevSecOps, creating a DevSecOps strategy is crucial. This involves instilling a "security-first" mindset in all individuals involved and incorporating security best practices from the beginning. When developing the strategy, it is important to consider priorities, the cost of time and resources, and ensure that efforts are time-bound for successful implementation. The strategy serves to establish alignment on what needs to be achieved as part of the DevSecOps adoption. 4. Leadership Buy-In Leadership and executives hold a significant responsibility in promoting and embracing DevSecOps, and it is crucial to obtain their buy-in to ensure no other business objective or key result hinders the adoption of DevSecOps in the organization. This is where you present the DevSecOps strategy to the leadership and inform them of the initial setup costs in terms of time, money, and resources. At the same time, it is an opportunity to educate them about the long-term benefits and their impact on the organization. 5. Implementation Phase The real work begins in the execution phase, where time is of the essence. It is crucial to start small, gather feedback, and iterate. During this phase, the available tools should be evaluated to help expedite the adoption process. 6. Success Criteria and Measurement Feedback is a crucial aspect of growth in life, starting from childhood, where we receive continuous feedback from our parents that helps us learn and improve our skills. Similarly, in a DevSecOps environment, it is essential to have a system in place for constant feedback to facilitate continuous improvement. Tools such as alarms, dashboards, and monitoring through alerts can help audit applications and detect issues proactively. Additionally, establish an ongoing feedback mechanism, such as quarterly Agile retrospectives or surveys for employees to provide feedback. It is essential to have governance and guardrails in place to measure the success of DevSecOps adoption. The steps above serve as a roadmap to enable organizations to successfully implement DevSecOps and create secure software right from the outset. Short-term investments made in DevSecOps can yield long-term benefits such as the ability to release better, faster, and more secure products to customers. Continuous feedback mechanisms can facilitate ongoing improvement and iteration. By utilizing DecSecOps, organizations can avoid common pitfalls associated with it and ensure that the integration of DevSecOps represents a cultural shift in their development processes rather than a one-time effort.
Key Takeaways Accelerating enterprise software delivery through automated release processes in a SAFe environment can help organizations to improve their software product quality, agility, flexibility, visibility, security, and governance and stay competitive in a rapidly changing marketplace, maximizing business value, trust, and time to market. By automating and incorporating quality gates and security scans in the CI/CD pipelines, businesses may improve the quality of their software products, identify critical vulnerabilities, bugs, code smells, and security hotspots, and as a result, future tech debt will be decreased. Effective release management requires a focus on teamwork and communication and the use of automation and continuous feedback loops to identify and address issues as soon as they appear. Introduction In recent years, the software development tech sector has experienced tremendous changes, with classic Waterfall methodologies giving way to more agile methodologies like SAFe (Scaled Agile Framework). SAFe has caused a shift from waterfall to agile software development processes due to the need for faster, more adaptable, and more efficient methods. Increasing adoption of Agile methodology throughout organizations has made Scaled Agile Framework (SAFe) an ideal solution for scaling Agile practices. While SAFe is an excellent framework for large-scale Agile development, many businesses have difficulty handling the complexity of release processes within SAFe. Therefore, it explored how automated release methods can actually accelerate software delivery in SAFe in order to enable companies to release high-quality software more quickly and frequently. Is Safe an Effective Release Management Framework? SAFe (Scaled Agile Framework) is a popular framework for implementing agile practices at scale. It is designed to help organizations deliver software more efficiently and effectively by providing a structured approach to agile development across large, complex organizations. In summary, SAFe is well-suited for release management because it provides a structured approach to agile development at scale. SAFe is specifically designed to work at scale, making it an ideal framework for large organizations with multiple teams working on complex software products. The framework includes a set of best practices, roles, and tools that can help organizations deliver software more efficiently and effectively while also promoting collaboration, transparency, and continuous improvement. The Significance of Time to Market and Recommended Practices for DevOps The goal of DevOps release management is to ensure that software releases are delivered quickly, reliably, and with minimal risk. This includes continuous integration and continuous deployment (CI/CD) pipelines, version control systems, automated testing, monitoring and feedback mechanisms, and collaboration between development and operations teams. Time to market is a critical factor in the success of software products, as it directly impacts the ability of businesses to stay competitive, capture market share, and meet the needs of customers. Adopt software development best practices that enable faster development and release cycles. This includes implementing agile development methodologies, automating and using continuous integration and delivery (CI/CD) pipelines, automating testing and deployment processes, and ensuring that development teams are aligned with business goals and objectives. Release Management in SAFe: A Challenge Managing Dependencies Agile approaches are becoming increasingly popular among product and engineering teams within organizations, which leads to challenges in managing release processes. In a SAFe system, managing release dependencies requires coordinating several Agile teams across various product lines and ensuring that all teams are in agreement with the release timeline. Manual Methods Software releases to production environments can be delayed by manual release management methods because they take a long time, are prone to error, and are time-consuming. Lack of Consistency The lack of consistency in release processes creates problems for many teams across the organization. In a SAFe environment with numerous teams and products, it can be challenging to establish a common process that can be automated for automated release processes. Accelerating Enterprise Software Delivery Through Automated Release Processes Automating the release process can help in a SAFe system to streamline and speed up software delivery, reduce the probability of errors, reduce future tech debt, and improve overall product quality. Things To Consider Prior To Automating Release Processes Establish a clear release management process: A clear release management process should be established that outlines the steps required to move software from development to production. This process should include guidelines for the environment and release planning and readiness, code reviews, security reviews, implementation/deployment plan reviews, QA/ integration/ UAT strategy and criteria, go/no-go criteria, release DoD (Definition of Done), and well-defined rollback/backout plans. Define release types and approval process: Normal or standard release, off-cycle release, emergency release, SOPs, and the criteria for approvals Change management: Following strict change management guidelines in creating Request for Change (RFCs) using change management tools to document change descriptions, deployment plans, risk assessment, infrastructure information, approval documentation, etc. Define approval criteria: Depending on the organization’s release governance guidelines, appropriate approval criteria should be defined for each stage of the release process. Some examples: approval before deploying code from dev to the integration/UAT environment. Approval to merge final code to master branch. Automating Releases Using Automation Tools: Automation tools should be used to automate the release process and approvals. These tools can include automated testing tools, deployment tools, and release orchestration tools. Version control, build automation, deployment orchestration, and deployment version tracking are a few key things to consider in the initial planning of the project. Implementing enterprise CI/CD pipelines: Continuous integration and continuous deployment should be implemented to ensure that code changes are quickly and efficiently integrated into the release process. A good branching strategy with automated testing and deployment scripts will help meet business demands Implementing Quality Gates into the CI/CD Pipelines: A quality gate is a requirement the software must satisfy in order to move on to the next stage that is enforced and built into your pipeline. With this strategy, the code will be held to a set of standards and best practices that it must follow to stay in good shape. As a result, there will be less need for manual regression testing during the development cycle, which will speed up project execution overall. The pipeline can self-monitor the quality of the delivered code thanks to these quality gates, which are usually automated. Security Scans: Incorporate security scans on artifacts into your pipeline checks, preferably at the start of the project. Before the code is released further, this quality gate mandates that security scans check for out-of-date packages, identify code vulnerabilities, code signing, etc., and the results be evaluated and passed against a specific threshold. Before moving the code to the following step, a gate should start the scan and verify its success and completion. As a result, future tech debt will be decreased, and the code will be maintained to the most recent releases. Automating Approval Workflows: Approval workflows should be established to ensure that code changes are thoroughly reviewed and approved before they are deployed to production. Integrating release approvals in the CI/CD pipeline across all product teams can provide several benefits for organizations, including improved release quality, increased efficiency, greater collaboration, increased visibility, consistent standards, and improved compliance. These benefits can help organizations deliver higher-quality software faster and with greater confidence. An override mechanism should be in a place where manual testing is involved to override known errors or to override approvals that are not necessary. Conclusion By standardizing release management processes, addressing resistance to adopting the latest SAFe Agile practices and CI/CD implementation, integrating legacy systems, and addressing compliance and security concerns, organizations can successfully implement automated release processes and accelerate their software delivery. Accelerating enterprise software delivery through automated release processes in a SAFe environment can help organizations to improve their software product quality, agility, flexibility, visibility, security, and governance and stay competitive in a rapidly changing marketplace, maximizing business value and time to market. However, organizations must address several challenges to ensure successful implementation.
I first ran into the concept of Continuous Integration (CI) when the Mozilla project launched. It included a rudimentary build server as part of the process and this was revolutionary at the time. I was maintaining a C++ project that took two hours to build and link. We rarely went through a clean build, which created compounding problems as bad code was committed into the project. A lot has changed since those old days. CI products are all over the place and, as Java developers, we enjoy a richness of capabilities like never before. But I’m getting ahead of myself. Let’s start with the basics. Continuous Integration is a software development practice in which code changes are automatically built and tested in a frequent and consistent manner. The goal of CI is to catch and resolve integration issues as soon as possible, reducing the risk of bugs and other problems slipping into production. CI often goes hand in hand with Continuous Delivery (CD), which aims to automate the entire software delivery process, from code integration to deployment in production. The goal of CD is to reduce the time and effort required to deploy new releases and hotfixes, enabling teams to deliver value to customers faster and more frequently. With CD, every code change that passes the CI tests is considered ready for deployment, allowing teams to deploy new releases at any time with confidence. I won’t discuss continuous delivery in this article, but I will go back to it as there’s a lot to discuss. I’m a big fan of the concept but there are some things we need to monitor. Continuous Integration Tools There are many powerful continuous integration tools. Here are some commonly used tools: Jenkins: Jenkins is one of the most popular CI tools, offering a wide range of plugins and integrations to support various programming languages and build tools. It is open-source and offers a user-friendly interface for setting up and managing build pipelines. It’s written in Java and was often my “go to tool.” However, it’s a pain to manage and set-up. There are some “Jenkins as a service” solutions that also clean up its user experience, which is somewhat lacking. Travis CI: Travis CI is a cloud-based CI tool that integrates well with GitHub, making it an excellent choice for GitHub-based projects. Since it predated GitHub Actions, it became the default for many open-source projects on GitHub. CircleCI: CircleCI is a cloud-based CI tool that supports a wide range of programming languages and build tools. It offers a user-friendly interface but it’s big selling point is the speed of the builds and delivery. GitLab CI/CD: GitLab is a popular source code management tool that includes built-in CI/CD capabilities. The GitLab solution is flexible yet simple, it has gained some industry traction even outside of the GitLab sphere. Bitbucket pipelines: Bitbucket pipelines is a cloud-based CI tool from Atlassian that integrates seamlessly with Bitbucket, their source code management tool. Since it’s an Atlassian product, it provides seamless JIRA integration and very fluid, enterprise oriented functionality. Notice I didn’t mention GitHub Actions, which we will get to shortly. There are several factors to consider when comparing CI tools: Ease of use: Some CI tools have a simple setup process and user-friendly interface, making it easier for developers to get started and manage their build pipelines. Integration with Source Code Management (SCM): Tools such as GitHub, GitLab, and Bitbucket. This makes it easier for teams to automate their build, test, and deployment processes. Support for different programming languages and build tools: Different CI tools support different programming languages and build tools, so it’s important to choose a tool that is compatible with your development stack. Scalability: Some CI tools are better suited to larger organizations with complex build pipelines, while others are better suited to smaller teams with simpler needs. Cost: CI tools range in cost from free and open-source to commercial tools that can be expensive, so it’s important to choose a tool that fits your budget. Features: Different CI tools offer distinct features, such as real-time build and test results, support for parallel builds, and built-in deployment capabilities. In general, Jenkins is known for its versatility and extensive plugin library, making it a popular choice for teams with complex build pipelines. Travis CI and CircleCI are known for their ease of use and integration with popular SCM tools, making them a good choice for small to medium-sized teams. GitLab CI/CD is a popular choice for teams using GitLab for their source code management, as it offers integrated CI/CD capabilities. Bitbucket Pipelines is a good choice for teams using Bitbucket for their source code management, as it integrates seamlessly with the platform. Cloud vs. On-Premise The hosting of agents is an important factor to consider when choosing a CI solution. There are two main options for agent hosting: cloud-based and on-premise: Cloud-based: Cloud-based CI solutions, such as Travis CI, CircleCI, GitHub Actions, and Bitbucket Pipelines, host the agents on their own servers in the cloud. This means you don’t have to worry about managing the underlying infrastructure, and you can take advantage of the scalability and reliability of the cloud. On-premise: On-premise CI solutions, such as Jenkins, allow you to host the agents on your own servers. This gives you more control over the underlying infrastructure, but also requires more effort to manage and maintain the servers. When choosing a CI solution, it’s important to consider your team’s specific needs and requirements. For example, if you have a large and complex build pipeline, an on-premise solution, such as Jenkins, may be a better choice, as it gives you more control over the underlying infrastructure. On the other hand, if you have a small team with simple needs, a cloud-based solution, such as Travis CI, may be a better choice, as it is easy to set up and manage. Agent Statefulness Statefulness determines whether the agents retain their data and configurations between builds: Stateful agents: Some CI solutions, such as Jenkins, allow for stateful agents, which means the agents retain their data and configurations between builds. This is useful for situations where you need to persist data between builds, such as when you’re using a database or running long-running tests. Stateless agents: Other CI solutions, such as Travis CI, use stateless agents, which means the agents are recreated from scratch for each build. This provides a clean slate for each build, but it also means you need to manage any persisted data and configurations externally, such as in a database or cloud storage. There’s a lively debate among CI proponents regarding the best approach. Stateless agents provide a clean and easy to reproduce environment. I choose them for most cases and think they are the better approach. Stateless agents can be more expensive too as they are slower to set up. Since we pay for cloud resources, the cost can add up. But the main reason some developers prefer the stateful agents is the ability to investigate. With a stateless agent, when a CI process fails, you are usually left with no means of investigation other than the logs. With a stateful agent, we can log into the machine and try to run the process manually on the given machine. We might reproduce an issue that failed and gain insight thanks to that. A company I worked with chose Azure over GitHub Actions because Azure allowed for stateful agents. This was important to them when debugging a failed CI process. I disagree with that but that’s a personal opinion. I feel I spent more time troubleshooting bad agent cleanup than I benefited from investigating a bug. But that’s a personal experience and some smart friends of mine disagree. Repeatable Builds Repeatable builds refer to the ability to produce the same exact software artifacts every time a build is performed, regardless of the environment or the time the build is performed. From a DevOps perspective, having repeatable builds is essential to ensuring software deployments are consistent and reliable. Intermittent failures are the bane of DevOps everywhere and they are painful to track. Unfortunately, there’s no easy fix. As much as we’d like it, some flakiness finds its way into projects with reasonable complexity. It is our job to minimize this as much as possible. There are two blockers to repeatable builds: Dependencies: If we don’t use specific versions for dependencies even a small change can break our build. Flaky tests: Tests that fail occasionally for no obvious reasons are the absolute worst. When defining dependencies, we need to focus on specific versions. There are many versioning schemes but, over the past decade, the standard three-number semantic versioning took over the industry. This scheme is immensely important for CI as its usage can significantly impact the repeatability of a build, e.g., with Maven, we can do: <dependency> <groupId>group</groupId> <artifactId>artifact</artifactId> <version>2.3.1</version> </dependency> This is very specific and great for repeatability. However, this might become out of date quickly. We can replace the version number with LATEST or RELEASE, which will automatically get the current version. This is bad as the builds will no longer be repeatable. However, the hard-coded three-number approach is also problematic. It’s often the case that a patch version represents a security fix for a bug. In that case, we would want to update all the way to the latest minor update but not newer versions. For that previous case, I would want to use version 2.3.2 implicitly and not 2.4.1. This trades off some repeatability for minor security updates and bugs. A better way would be to use the Maven Versions Plugin and periodically invoke the mvn versions:use-latest-releases command. This updates the versions to the latest to keep our project up to date. This is the straightforward part of repeatable builds. The difficulty is in the flaky tests. This is such a common pain that some projects define a “reasonable amount” of failed tests and some projects re-run the build multiple times before acknowledging failure. A major cause of test flakiness is state leakage. Tests might fail because of subtle side effects left over from a previous test. Ideally, a test should clean up after itself, so each test will run in isolation. In a perfect world, we would run every test in a completely isolated fresh environment, but this isn’t practical. It would mean tests would take too long to run and we would need to wait a great deal of time for the CI process. We can write tests with various isolation levels, sometimes we need complete isolation and might need to spin up a container for a test. But most times we don’t, and the difference in speed is significant. Cleaning up after tests is very challenging. Sometimes state leaks from external tools, such as the database, can cause a flaky test failure. To ensure repeatability of failure, it is a common practice to sort the test cases consistently, this ensures future runs of the build will execute in the same order. This is a hotly debated topic. Some engineers believe this encourages buggy tests and hides problems we can only discover with a random order of tests. From my experience, this did find bugs in the tests, but not in the code. My goal isn’t to build perfect tests, so I prefer running the tests in a consistent order such as alphabetic ordering. It is important to keep statistics of test failures and never simply press retry. By tracking the problematic tests and the order of execution for a failure, we can often find the source of the problem. Most times the root cause of the failure happens because of faulty cleanup in a prior test, which is why the order matters and its consistency is also important. Developer Experience and CI Performance We’re here to develop a software product, not a CI tool. The CI tool is here to make the process better. Unfortunately, many times the experience with the CI tool is so frustrating that we end up spending more time on logistics than actually writing code. Often, I spent days trying to pass a CI check so I could merge my changes. Every time I get close, another developer would merge their change first and would break my build. This contributes to a less than stellar developer experience, especially as a team scales and we spend more time in the CI queue than merging our changes. There are many things we can do to alleviate these problems: Reduce duplication in tests: Over testing is a common symptom that we can detect with coverage tools. Flaky test elimination: I know deleting or disabling tests is problematic. Don’t do that lightly. But if you spend more time debugging the test than debugging your code, it’s value is debatable. Allocate additional or faster machines for the CI process. Parallelize the CI process: We can parallelize some build types and some tests. Split the project into smaller projects: Notice that this doesn’t necessarily mean microservices. Ultimately, this connects directly to the productivity of the developers. But we don’t have profilers for these sorts of optimizations. We have to measure each time, this can be painstaking. GitHub Actions GitHub Actions is a Continuous Integration/Continuous Delivery (CI/CD) platform built into GitHub. It is stateless although it allows the self-hosting of agents to some degree. I’m focusing on it since it’s free for open-source projects and has a decent free quota for closed-source projects. This product is a relatively new contender in the field, it is not as flexible as most other CI tools mentioned before. However, it is very convenient for developers thanks to its deep integration with GitHub and stateless agents. To test GitHub Actions, we need a new project, which, in this case, I generated using JHipster with the configuration seen here: I created a separate project that demonstrates the use of GitHub Actions here. Notice you can follow this with any project, although we include Maven instructions in this case, the concept is very simple. Once the project is created, we can open the project page on GitHub and move to the “Actions” tab. We will see something like this: In the bottom right corner, we can see the “Java with Maven” project type. Once we pick this type, we move to the creation of a maven.yml file as shown here: Unfortunately, the default maven.yml suggested by GitHub includes a problem. This is the code we see in this image: name: Java CI with Maven on: push: branches: [ "master" ] pull_request: branches: [ "master" ] jobs: build: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - name: Set up JDK 11 uses: actions/setup-java@v3 with: java-version: '11' distribution: 'temurin' cache: maven - name: Build with Maven run: mvn -B package --file pom.xml # Optional: Uploads the full dependency graph to GitHub to improve the quality of Dependabot alerts this repository can receive - name: Update dependency graph uses: advanced-security/maven-dependency-submission-action@571e99aab1055c2e71a1e2309b9691de18d6b7d6 The last three lines update the dependency graph. But this feature fails, or at least it failed for me. Removing them solved the problem. The rest of the code is a standard YAML configuration. The pull_request and push lines near the top of the code, declare that builds will run on a pull request and a push to the master. This means we can run our tests on a pull request before committing. If the test fails, we will not commit. We can disallow committing with failed tests in the project settings. Once we commit the YAML file, we can create a pull request and the system will run the build process for us. This includes running the tests, since the “package” target in Maven runs tests by default. The code that invokes the tests is in the line starting with “run” near the end. This is effectively a standard unix command line. Sometimes it makes sense to create a shell script and run it from the CI process. It’s sometimes easier to write a good shell script than deal with all the YAML files and configuration settings of various CI stacks. It’s also more portable if we choose to switch the CI tool in the future. Here, we don’t need it since Maven is enough for our current needs. We can see the successful pull request here: To test this out, we can add a bug to the code by changing the “/api” endpoint to “/myapi”. This produces the failure shown below. It also triggers an error email sent to the author of the commit: When such a failure occurs, we can click the “Details” link on the right side. This takes us directly to the error message you see here: Unfortunately, this is typically a useless message that does not provide help in the issue resolution. However, scrolling up will show the actual failure, which is usually conveniently highlighted for us as seen here: Note: there are often multiple failures, so it would be prudent to scroll up further. In this error, we can see the failure was an assertion in line 394 of the AccountResourceIT, which you can see here. The line numbers do not match. In this case, line 394 is the last line of the method: @Test @Transactional void testActivateAccount() throws Exception { final String activationKey = "some activation key"; User user = new User(); user.setLogin("activate-account"); user.setEmail("activate-account@example.com"); user.setPassword(RandomStringUtils.randomAlphanumeric(60)); user.setActivated(false); user.setActivationKey(activationKey); userRepository.saveAndFlush(user); restAccountMockMvc.perform(get("/api/activate?key={activationKey}", activationKey)).andExpect(status().isOk()); user = userRepository.findOneByLogin(user.getLogin()).orElse(null); assertThat(user.isActivated()).isTrue(); } This means the assert call failed. isActivated() returned false and failed the test. This should help a developer narrow down the issue and understand the root cause. Going Beyond As we mentioned before, CI is about developer productivity. We can go much further than merely compiling and testing. We can enforce coding standards, lint the code, detect security vulnerabilities, and much more. In this example, let’s integrate Sonar Cloud, which is a powerful code analysis tool (linter). It finds potential bugs in your project and helps you improve code quality. SonarCloud is a cloud-based version of SonarQube that allows developers to continuously inspect and analyze their code to find and fix issues related to code quality, security, and maintainability. It supports various programming languages such as Java, C#, JavaScript, Python, and more. SonarCloud integrates with popular development tools such as GitHub, GitLab, Bitbucket, Azure DevOps, and more. Developers can use SonarCloud to get real-time feedback on the quality of their code and improve the overall code quality. On the other hand, SonarQube is an open-source platform that provides static code analysis tools for software developers. It provides a dashboard that shows a summary of the code quality and helps developers identify and fix issues related to code quality, security, and maintainability. SonarCloud and SonarQube provide similar functionalities, but SonarCloud is a cloud-based service and requires a subscription, while SonarQube is an open-source platform that can be installed on-premise or on a cloud server. For simplicity’s sake, we will use SonarCloud, but SonarQube should work just fine. To get started, we go to sonarcloud.io, and sign up. Ideally with our GitHub account. We are then presented with an option to add a repository for monitoring by Sonar Cloud as shown here: When we select the “Analyze new page” option, we need to authorize access to our GitHub repository. The next step is selecting the projects we wish to add to Sonar Cloud as shown here: Once we select and proceed to the setup process, we need to pick the analysis method. Since we use GitHub Actions, we need to pick that option in the following stage as seen here: Once this is set, we enter the final stage within the Sonar Cloud wizard as seen in the following image. We receive a token that we can copy (entry 2 that is blurred in the image), we will use that shortly. Notice there are also default instructions to use with Maven that appear once you click the button labeled “Maven.” Going back to the project in GitHub, we can move to the “Project Settings” tab (not to be confused with the account settings in the top menu). Here we select “Secrets and variables” as shown here: In this section, we can add a new repository secret, specifically the SONAR_TOKEN key and value we copied from the SonarCloud as you can see here: GitHub Repository Secrets are a feature that allows developers to securely store sensitive information associated with a GitHub repository, such as API keys, tokens, and passwords, which are required to authenticate and authorize access to various third-party services or platforms used by the repository. The concept behind GitHub Repository Secrets is to provide a secure and convenient way to manage and share confidential information, without having to expose the information publicly in code or configuration files. By using secrets, developers can keep sensitive information separate from the codebase and protect it from being exposed or compromised in case of a security breach or unauthorized access. GitHub Repository Secrets are stored securely and can only be accessed by authorized users who have been granted access to the repository. Secrets can be used in workflows, actions, and other scripts associated with the repository. They can be passed as environment variables to the code, so it can access and use the secrets in a secure, reliable way. Overall, GitHub Repository Secrets provide a simple and effective way for developers to manage and protect confidential information associated with a repository, helping ensure the security and integrity of the project and the data it processes. We now need to integrate this into the project. First, we need to add these two lines to the pom.xml file. Notice that you need to update the organization name to match your own. These should go into the section in the XML: <sonar.organization>shai-almog</sonar.organization> <sonar.host.url>https://sonarcloud.io</sonar.host.url> Notice that the JHipster project we created already has SonarQube support, which should be removed from the pom file before this code will work. After this, we can replace the “Build with Maven” portion of the maven.yml file with the following version: - name: Build with Maven env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN } # Needed to get PR information, if any SONAR_TOKEN: ${{ secrets.SONAR_TOKEN } run: mvn -B verify org.sonarsource.scanner.maven:sonar-maven-plugin:sonar -Dsonar.projectKey=shai-almog_HelloJHipster package Once we do that, SonarCloud will provide reports for every pull request merged into the system as shown here: We can see a report that includes the list of bugs, vulnerabilities, smells, and security issues. Clicking every one of those issues leads us to something like this: Notice that we have tabs that explain exactly why the issue is a problem, how to fix it, and more. This is a remarkably powerful tool that serves as one of the most valuable code reviewers in the team. Two additional interesting elements we saw before are the coverage and duplication reports. SonarCloud expects that tests will have 80% code coverage (trigger 80% of the code in a pull request), this is high and can be configured in the settings. It also points out duplicate code, which might indicate a violation of the Don’t Repeat Yourself (DRY) principle. Finally CI is a huge subject with many opportunities to improve the flow of your project. We can automate the detection of bugs, streamline artifact generation, automated delivery, and so much more. In my humble opinion, the core principle behind CI is developer experience. It’s here to make our lives easier. When it is done badly, the CI process can turn this amazing tool into a nightmare. Passing the tests becomes an exercise in futility. We retry again and again until we can finally merge. We wait for hours to merge because of slow, crowded queues. This tool that was supposed to help becomes our nemesis. This shouldn’t be the case. CI should make our lives easier, not the other way around.
As software development continues to evolve, there are two approaches that have gained a lot of attention in recent years - Agile and DevOps. Agile has been around since the early 2000s and focuses on delivering software frequently through iterative and incremental development. DevOps, on the other hand, is a newer approach that focuses on speeding up the software delivery process through collaboration, automation, and continuous delivery. While both Agile and DevOps aim to improve efficiency and collaboration within the development team, there are some key differences between the two approaches. Agile is focused on the software development process, while DevOps is focused on deployment, integration, and delivery. Agile uses a methodology of sprints, daily stand-ups, and retrospectives to deliver working software frequently. DevOps, on the other hand, uses continuous integration and continuous deployment to speed up the delivery process. Agile Agile is a software development methodology that focuses on delivering value to customers through iterative and incremental development. It values collaboration, flexibility, and customer satisfaction. Agile teams work in short sprints, usually lasting 1-4 weeks, and aim to deliver working software at the end of each sprint. Agile development involves continuous feedback from the customer and the team, and the ability to adapt to changing requirements. Agile practices include daily stand-up meetings, sprint planning, backlog grooming, and retrospective meetings. The Agile Manifesto defines four core values: individuals and interactions over processes and tools, working software over comprehensive documentation, customer collaboration over contract negotiation, and responding to change over following a plan. DevOps DevOps is a culture that emphasizes collaboration, communication, and integration between development and operations teams. The goal of DevOps is to improve the quality and speed of software delivery, by automating processes and reducing the time it takes to go from development to production. DevOps involves a combination of practices, such as continuous integration, continuous delivery, and continuous deployment. DevOps teams work in a continuous cycle of development, testing, deployment, and monitoring. This allows for rapid feedback and the ability to quickly fix issues. DevOps teams also value the use of automation tools, such as configuration management, orchestration, and monitoring tools. DevOps vs Agile Agile and DevOps are complementary approaches that share many of the same values and principles. Both aim to deliver high-quality software that meets the needs of the customer. However, there are some key differences between the two approaches. Agile is focused on the software development process, while DevOps is focused on the entire software lifecycle. Agile teams work in short sprints, while DevOps teams work in a continuous cycle. Agile teams rely on manual testing and deployment, while DevOps teams automate these processes. Agile teams prioritize flexibility and customer collaboration, while DevOps teams prioritize speed and efficiency. Agile DevOps Focus Software development process Deployment, integration and delivery process Goals Delivering working software frequently Speeding up software delivery and feedback Methodology Iterative and incremental Continuous delivery and deployment Process Sprint planning, daily stand-ups, retrospectives Continuous Integration and Continuous Deployment Team Self-organizing cross-functional teams Collaborative and integrated teams Communication Face-to-face communication, regular meetings Strong collaboration, communication and feedback loop Feedback Regular customer feedback and iterations Continuous feedback through automated testing and monitoring Culture Empowered and autonomous teams Collaborative, feedback-oriented culture Tools Agile project management tools, issue trackers Automated testing, monitoring, and deployment tools Which Approach Will Win the Battle for Efficiency? The answer to this question depends on the specific needs and goals of your organization. If your goal is to improve the speed and efficiency of your software delivery, then DevOps may be the better approach. DevOps allows for rapid feedback, quick issue resolution, and automation of manual processes. However, if your goal is to prioritize flexibility, collaboration, and customer satisfaction, then Agile may be the better approach. Agile allows for continuous feedback from the customer and the team, and the ability to adapt to changing requirements. Ultimately, the choice between DevOps and Agile depends on the specific needs and goals of your organization. It is possible to combine elements of both approaches to create a customized approach that works best for your organization. Conclusion DevOps and Agile are two popular approaches in software development that aim to improve efficiency and productivity. Both approaches have their own principles, practices, and benefits. The choice between DevOps and Agile depends on the specific needs and goals of your organization. It is possible to combine elements of both approaches to create a customized approach that works best for your organization.
CI/CD (Continuous Integration and Continuous Delivery) is an essential part of modern software development. CI/CD tools help developers automate the process of building, testing, and deploying software, which saves time and improves code quality. GitLab and Jenkins are two popular CI/CD tools that have gained widespread adoption in the software development industry. In this article, we will compare GitLab and Jenkins and help you decide which one is the best CI/CD tool for your organization. GitLab vs Jenkins 1. Ease of Use GitLab is an all-in-one platform that provides a comprehensive solution for CI/CD, version control, project management, and collaboration. It has a simple and intuitive user interface that makes it easy for developers to set up and configure their pipelines. On the other hand, Jenkins is a highly customizable tool that requires some technical expertise to set up and configure. It has a steep learning curve, and new users may find it challenging to get started. 2. Integration GitLab and Jenkins support integration with a wide range of tools and services. However, GitLab offers more native integrations with third-party services, including cloud providers, deployment platforms, and monitoring tools. This makes it easier for developers to set up their pipelines and automate their workflows. Jenkins also has a vast library of plugins that support integration with various tools and services. 3. Performance GitLab is known for its fast and reliable performance. It has built-in caching and parallel processing capabilities that allow developers to run their pipelines quickly and efficiently. Jenkins, on the other hand, can suffer from performance issues when running large and complex pipelines. It requires manual optimization to ensure it can handle the load. 4. Security GitLab has built-in security features that ensure code is secure at every pipeline stage. It provides features, like code scanning, vulnerability management, and container scanning, that help developers identify and fix security issues before they make it into production. Jenkins relies heavily on plugins for security features. This can make it challenging to ensure your pipeline is secure, especially if you are using third-party plugins. 5. Cost GitLab offers free and paid plans. The free plan includes most features a small team would need for CI/CD. The paid plans include additional features like deployment monitoring, auditing, and compliance. Jenkins is an open-source tool that is free to use. However, it requires significant resources to set up and maintain, which can add to the overall cost of using the tool. The Technical Difference Between GitLab and Jenkins Feature GitLab Jenkins Version Control Git N/A (requires integration with a separate VCS tool). Continuous Integration Yes, built-in. Yes, built-in. Continuous Delivery Yes, built-in. Requires plugins or scripting. Security Built-in security features. Requires plugins or scripting. Code Review Built-in code review features. Requires plugins or scripting. Performance Generally faster due to built-in Git repository May require additional resources for performance Scalability Scales well for small to medium-sized teams. Scales well for large teams. Cost Free for self-hosted and cloud-hosted versions. Free for self-hosted and has a cost for cloud-hosted. Community Active open-source community and enterprise support. Active open-source community and enterprise support. GitLab vs Jenkins: Which One Is Best? GitLab and Jenkins are two popular tools used in the software development process. However, it’s difficult to say which one is better as it depends on the specific needs of your project and organization. GitLab is a complete DevOps platform that includes source code management, continuous integration/continuous delivery (CI/CD), and more. It offers features such as Git repository management, issue tracking, code review, and continuous integration/continuous delivery (CI/CD) pipelines. GitLab also has a built-in container registry and Kubernetes integration, making it easy to deploy applications to container environments. On the other hand, Jenkins is a popular open-source automation server widely used for continuous integration and continuous delivery (CI/CD) pipelines. It offers several plugins for various functionalities, such as code analysis, testing, deployment, and monitoring. Jenkins can be easily integrated with other tools in the software development process, such as Git, GitHub, and Bitbucket. Ultimately, the choice between GitLab and Jenkins will depend on your specific needs and preferences. GitLab is an all-in-one solution, while Jenkins is more flexible and can be customized with plugins. GitLab is a better choice if you want an integrated solution with an intuitive interface and built-in features. Jenkins is better if you want a customizable and extensible automation server that can be easily integrated with other tools in your workflow. Conclusion GitLab and Jenkins are excellent CI/CD tools that offer a range of features and integrations. However, GitLab has the edge when it comes to ease of use, integration, performance, security, and cost. GitLab’s all-in-one platform makes it easy for developers to set up and configure their pipelines, while its native integrations and built-in features make it more efficient and secure than Jenkins. Therefore, if you are looking for a CI/CD tool that is easy to use, cost-effective, and reliable, GitLab is the best option for your organization.
Tracking Mean Time To Restore (MTTR) is standard industry practice for incident response and analysis, but should it be? Courtney Nash, an Internet Incident Librarian, argues that MTTR is not a reliable metric — and we think she's got a point. We caught up with Courtney at the DevOps Enterprise Summit in Las Vegas, where she was making her case against MTTR in favor of alternative metrics (SLOs and cost of coordination data), practices (Near Miss analysis), and mindsets (humans are the solution, not the problem) to help organization better learn from their incidents. Episode Highlights (1:54) The end of MTTR? (4:50) Library of incidents (13:20) What is an incident? (19:41) Cost of coordination (22:13) Near misses (24:21) Mental models (28:16) Role of language in shaping public discourse (29:33) Learnings from The Void Episode Excerpt Dan: Hey, everyone; welcome to Dev Interrupted. My name is Dan lines, and I'm here with Courtney Nash, who has one of the coolest possibly made-up titles, but possibly real: Internet Incident Librarian. Courtney: Yep, that's right, yeah, you got it. Dan: Welcome to the show. Courtney: Thank you for having me on. Dan: I love that title Courtney: Still possibly made up, possibly, possibly... Dan: Still possibly made up. Courtney: We'll just leave that one out there for the listeners to decide. Dan: Let everyone decide what that could possibly mean. We have a, I think, maybe a spicy show, a spicy topic. Courtney: It's a hot topic show. Dan: Hot topic, especially since we're at DevOps Enterprise Summit, where we hear a lot about the DORA metrics, one of them being MTTR. Courtney: Yes. Dan: And you might have a hot take on that. The end of MTTR? Or how would you describe it? Courtney: Yeah, I feel a little like the fox in the henhouse here, but Gene accepted the talk. So you know, there's that. Dan: So it's on him. Courtney: [laughing] It's all Gene's fault! So I have been interested in complex systems for a long time; I used to study the brain. And I got sucked down an internet rabbit hole quite a lot quite a while ago. And I've had beliefs for a long time that I haven't had data to back up necessarily. And we see these sort of perverted behaviors, not that kind of perverted, but where we take metrics in the industry, and then with Goddard's Law, pick whatever you pick up, people incentivize them, and then weird things happen. But I think we spend too little time looking at the humans in the system and a lot of time focusing on the technical aspects and the data that come out of the technical side of systems. So, I started a project about a year ago called The Void. It's the Verica Open Incident Database, actually a real, not made-up name. And it's the largest collection of public incident reports. So, if you all have an outage, and you hopefully go and figure out and talk about what happened, and then you write that up, but that's out in the world, so I'm not writing these, I'm curating them and collecting. I'm a librarian. So, I have about 10,000 of them now. And a bunch of metadata associated with all these incident reports. Engineering Insights before anyone else... The Weekly Interruption is a newsletter designed for engineering leaders by engineering leaders. We get it. You're busy. So are we. That's why our newsletter is light, informative, and oftentimes irreverent. No BS or fluff. Each week we deliver actionable advice to help make you - whether you're a CTO, VP of Engineering, team lead, or IC — a better leader. It's also the best way to stay up-to-date on all things Dev Interrupted — from our podcast to trending articles, Interact, and our community Discord. Get interrupted.
CI/CD has been gaining a lot of attraction and is probably one of the most talked topics for the novices in DevOps. With the availability of CI/CD tools available in the market, configuring and operating a CI/CD pipeline has become a lot easier than what it was 5-6 years ago. Back then, there were no containers and the only CI/CD tool that dominated the sphere was Jenkins. Jenkins provided you with a task runner, so you could define your jobs to run either sequentially or in parallel. Today, the scenario is different. We have numerous CI/CD tools available in the market, which provides us with added features and functionality in comparison to Jenkins. One such renowned CI/CD tool is GitLab CI and that is precisely what we will be covering in this article. In this article, we will configure a CI/CD pipeline with GitLab CI/CD and execute Selenium testing over it through LambdaTest. Basics of CI/CD CI/CD is a collection of best practices followed to ensure you are delivering product updates to your web application on a consistent and reliable basis. Your web application is bound to grow with every sprint that is taken into a new release cycle. Initially, you may have a small team responsible for code changes in your web application. In such cases, you wouldn’t mind doing everything directly, you build the code, you test it yourself, and deploy it to the production. However, as your team grows, there will be a lot of interaction points and the probability of error increases as you try to migrate all of the code changes from one staging environment to another. This is where the CI/CD pipeline plays a pivotal role. Any successful business running online is highly-dependent on how their CI/CD pipelines are configured. According to High Scalability: Uber is now in 400 cities and 70 countries. They have over 6000 employees, 2000 of whom are engineers. Only a year and a half ago there were just 200 engineers. Those engineers have produced over 1000 microservices that are stored in over 8000 git repositories. If you observe how fast a business can grow, then you can imagine the challenges Uber might have to come across to coordinate with 10x engineers down the road, in just a year and a half, had they not incorporated a CI/CD pipeline. In today’s world, it would be extremely hard to imagine a web application that is scalable in terms of speed and consistency without following CI/CD best practices. Now, what are CI and CD? CI refers to Continuous Integration and CD implies Continuous Delivery. Combining both can achieve continuous deployment. Let us look at what they mean. What Is Continuous Integration? In traditional SDLC models, developers would migrate new features into an environment one-by-one in isolation. This created issues when you have multiple developers working over multiple features. Continuous Integration is a practice that ensures developers are able to commit numerous changes to the main branch of your web application through a shared repository, in a systematic manner. By leveraging the practice of Continuous Integration, your developers can integrate code around hotfixes, product enhancement, etc., into a shared repository, multiple times a day. That way, your overall go-to-market launch can accelerate, allowing you to be agile. If you have given edit access to GitHub repository to developers in your team, you only need to ensure the developers are following best practices, code styling, and, most importantly, the test cases are not failing. As long as these requirements are fulfilled, you shouldn’t disallow anybody to check in your code. This will help your company scale continuously. What Is Continuous Delivery? Continuous Delivery only happens after CI is performed. As the name suggests, the practice Continuous Delivery ensures you have an automatic pipeline configured to deploy code changes from one staging environment to another. Continuous Delivery includes all the steps necessary to make your software deployable. This includes running comprehensive tests, quality assurance using testing tools, execution of builds, code signing, documentation, and deployment to pre-prod or user acceptance environments. Don’t Confuse Continuous Delivery With Continuous Deployment Think of Continuous Delivery as everything except the deployment. You prepare deployment but you don’t actually deploy it to the production servers. You leave it to human intervention steps that will ensure when and where to deploy. Continuous Delivery is suitable for teams where Continuous Deployment is not required. However, Continuous Deployment is a practice that can only be implemented if you have a well-defined migration system set up, which makes it infeasible for organizations with less employees on-board. Which brings us to our next question. What Is Continuous Deployment? Continuous Deployment actually follows up with Continuous Delivery. It is an extension of Continuous Delivery. It takes Continuous Delivery a step further to a stage where the deployment for the new release of the version on the production is conducted automatically. The only requirement for Continuously Deployment is that the process, the checks and tests set up, guarantee a crash-free experience. Now, since it’s a completely automated system, it’s imperative that you spend more time developing very strict test cases because, here, you don’t have any chance for manually reviewing your the migration. Once it’s gone; it’s gone. Which is why Continuous deployment isn’t feasible for all companies. Continuous Deployment should have the strictest rules possible before deploying the code because of the process being a fully automated system without any human intervention. Being the last step in the chain of the automated pipeline production, it’s imperative the checks and tests, at this level, are the strictest and anything less than a 100% should be rejected without any leeway. In spite of all the benefits that comes with Continuous Deployment, a team should validate the requirements and only adopt Continuous Deployment if the development environment, production sensitivity, and the test system allow seamless adoption. Keep in mind, if the systems in the place are not mature enough, then the deployment might prove to be a catastrophic one for any team. Which is why most teams go with Continuous Delivery only and there’s no harm in that. It totally depends on what are you building and how critical it is. There’s no hard and fast rule that you should use Continuous Deployment. What Is GitLab CI/CD? GitLab has an excellent CI/CD offering for projects hosted on GitLab and other git providers. Using GitLab CI/CD, you can incorporate all three stages we discussed: Continuous Integration Continuous Delivery Continuous Deployment What makes GitLab CI/CD powerful is that it allows you to host your Git repository to any of the other Git providers, such as GitHub, and you can still harness it’s CI/CD system. You don’t even have to change your Git provider to use GitLab CI/CD. The only requirement to run CI/CD is the presence of a special GitLab CI YAML configuration file. GitLab CI YAML file contains all the instructions and data required to run different CI/CD pipelines. There are plenty of customization options available to shape the pipelines according to custom needs. Note: .gitlab-ci.yml is version-controlled and placed in the repository. This allows the old versions of your repository to build successfully, making it easier for your team to adopt the CI practice. Reason being, if the GitLab CI YAML is placed in the repository itself, it means you have now put the logic of CI/CD into your repository. Making you free from the worries that your CI/CD system might fail and you might lose your data. Now, wherever a code lives, your CI/CD is present there, making it simpler to shift from one hosting environment to another, as long as it uses the same pipeline. That way, your team can easily make use of CI branches as special different pipelines and jobs and you have a single source of truth for all CI/CD pipelines. What Are GitLab CI/CD Environment Variables? Env variables are dynamic-named values that can be used to make CI/CD pipelines completely dynamic and parameterized. In general, it’s always the best practice to keep removing hard coded values and use environment variables to make the jobs portable and provider agnostic. Specifically, GitLab has a huge list of predefined variables that can aid to build robust and flexible CI/CD pipelines. The most commonly used and important variables comprise of: CI_COMMIT_REF_NAME CI_COMMIT_BRANCH CI_COMMIT_TAG CI_EXTERNAL_PULL_REQUEST_IID These variables allow pipeline shaping according to different git branches and IMO. This provides great flexibility in differentiating jobs based on the environments. It is always better to use as many environment variables as possible to make your jobs customizable and flexible. What Are GitLab Cached Dependencies? Every CI/CD job requires some kind of building phase where the est target is built using 3rd party dependencies. Depending on the stack, these dependencies are fetched using plugin managers, module importers, etc. The common pain point in building with 3rd party modules across all languages is that it takes a lot of time to fetch dependencies from 3rd party sources and compile them. Imagine doing this process over a hundred times a day for multiple projects and calculate the time and resource wastage it incurs. Not a pleasant picture, right? If there was a way to cache these built dependencies and use these cached dependencies for multiple pipelines, it would make the CI build much faster and reduce bandwidth wastage and will unclog the CI pipelines so the same Infra can be used for much more builds. GitLab’s cached dependencies allow you to exactly do this straight out of the .gitlab-ci.yaml file. It’s as simple as setting a cache dictionary in a YAML file and key attribute. Just ensure you use the same key in all the jobs where cached directory is required. Common practice to ensure cache between branches is to use git bases environment variables as cache key. For example, CI_COMMIT_BRANCH can help you utilize cache whenever a job is run for a branch. GitLab CI/CD provides powerful primitives to invalidate cache. This can be done via UI or by clearing the cache key. An extension: You can optionally fetch dependencies and build them only to package manifest file changes. This is superior to using cache always. For example, only fetching Node.js dependencies whenever the package.json changes. How To Trigger a CI/CD Pipeline GitLab CI/CD allows you to trigger your pipeline using the following ways: Git-Based Triggers Webhooks/Crons Manual Intervention Git-Based Triggers The easiest way to trigger CI/CD is to perform any git based operation, such as pushing in a branch, merging pull request, or creating a tag for which handers are mentioned in the gitlab.yaml file. This is the most frequently used and most convenient method to trigger CI/CD. Webhooks Webhooks provide a convenient method to trigger CI/CD on demand by making an HTTP post call to specialized URLs. This is very useful for event-based triggering where the webhook can be called whenever a required event occurs: You can setup a cron to run nightly builds on GitLab by hitting a curl request on the webhook URL at the desired interval. Any other event can be used as long as the webhook can be hit in response to the event. Manual Intervention GitLab has a provision where manual intervention by authorized users can be requested to continue the next steps of the job. In the gitlab.yaml, you can mention a part of the pipeline to run only after somebody with access in the team can resume the job from UI: This feature enables constructing Continuous Delivery pipelines that we have discussed already. Everything, except deployment, can be automated and, only after the manual intervention, the deployment can take place. Exclusive Parameters for GitLab CI/CD: Only and Except Only and Except are two parameters that set a job policy to limit when jobs are created. These constructs are the nut and screw of the GitLab CI/CD pipeline that allow customization and conditional execution of jobs to shape it according to your own needs: Only specifies the names of branches and tags for which the job will trigger. Except specifies the names of branches and tags for which the job will not trigger. Only and Except are inclusive in nature and allow the use of regular expressions. This is a really interesting feature and makes any kind of customization possible by playing over strings. Only and Except allow us to specify a repository path to filter jobs for forks. Some of the interesting values that Only and Except take are: branches tags merge_requests This follows the precondition that GitLab CI/CD is supported for a git-based application. If you use multiple keys, under Only or Except, the keys will be evaluated as a single conjoined expression. That is: Only: means “include this job if all of the conditions match.” Except: means “exclude this job if any of the conditions match.” With Only, individual keys are logically joined by an AND. With Except, is implemented as a negation of this complete expression. This means the keys are treated as if joined by an OR. This relationship could be described as: There is a huge list of attributes that can be used by Only and Except conditions. I really recommend you check out those values. Executing Selenium Testing Scripts for GitLab CI/CD Let’s apply what we learned here. The project we’ll be using today for this GitLab CI/CD tutorial is the HourGlass 2018, which is a MERN (Mongo Express React and Node.js) stack application. It’s a simple time management application using the best practices available at that time. Unfortunately, in the JS world, those best practices change every month. Some of these might have been updated, but most of these are still relevant and this is a full production scale and production style development repository, with all the best practices available. Cloning the GitHub Repository Make sure you clone the HourGlass GitHub repository to your GitLab CI instance. After cloning, route to the master branch and check the GitLab YAML file: image: node:10.19.0 stages: - install_dependencies - build - test - deploy install_dependencies: stage: install_dependencies cache: key: $CI_COMMIT_REF_SLUG-$CI_PROJECT_DIR paths: - node_modules/ script: - yarn install only: changes: - yarn.lock #continuous integration unit-test: stage: test cache: key: $CI_COMMIT_REF_SLUG-$CI_PROJECT_DIR paths: - node_modules/ policy: pull script: yarn test # Only runs in case of continuous delivery integration-test: stage: test cache: key: $CI_COMMIT_REF_SLUG-$CI_PROJECT_DIR paths: - node_modules/ policy: pull services: - mongo script: - echo $MONGO_URI_TESTS - yarn test:integration only: - merge_requests - prod lint: stage: test cache: key: $CI_COMMIT_REF_SLUG-$CI_PROJECT_DIR paths: - node_modules/ policy: pull script: yarn lint e2e-test: stage: test services: - mongo cache: key: $CI_COMMIT_REF_SLUG-$CI_PROJECT_DIR paths: - build/ policy: pull script: - node node_modules/node-static/bin/cli.js build --port 5000 --spa & - yarn start-prod-server - node node_modules/pm2/bin/pm2 logs & - sleep 3 - yarn run test:e2e dependencies: - Build-client only: refs: - tags except: - /^((?!release).)*$/ Build-client: stage: build cache: key: $CI_COMMIT_REF_SLUG-$CI_PROJECT_DIR paths: - node_modules/ - build/ policy: pull-push script: yarn build-client artifacts: paths: - build Build-docs: stage: build script: yarn docs cache: key: $CI_COMMIT_REF_SLUG-$CI_PROJECT_DIR paths: - node_modules/ policy: pull only: - merge_requests - prod Build-storybook: stage: build script: yarn build-storybook cache: key: $CI_COMMIT_REF_SLUG-$CI_PROJECT_DIR paths: - node_modules/ policy: pull only: - merge_requests - prod deploy-backend: stage: deploy image: ruby:latest cache: key: $CI_COMMIT_REF_SLUG-$CI_PROJECT_DIR paths: - node_modules/ policy: pull script: - apt-get update -qy - apt-get install -y ruby-dev - gem install dpl - dpl --provider=heroku --app=$HEROKU_APP_NAME --api-key=$HEROKU_API_KEY dependencies: - e2e-test when: manual allow_failure: false only: refs: - tags except: - /^((?!release).)*$/ deploy-frontend: stage: deploy cache: key: $CI_COMMIT_REF_SLUG-$CI_PROJECT_DIR paths: - node_modules/ policy: pull variables: REACT_APP_API_HOST: $REACT_APP_API_HOST_PROD script: - yarn build-client - node ./node_modules/.bin/surge -p build/ --domain $SURGE_DOMAIN when: manual allow_failure: false dependencies: - e2e-test only: refs: - tags except: - /^((?!release).)*$/ Configuring the CI/CD Pipeline in GitLab CI To trigger our CI/CD pipeline, we will need to edit the README.md file. We can make these changes directly through Web IDE. We can go ahead and add a sample comment and hit the commit. Make sure to perform the change in the master branch. Once you commit the code, you can jump into the CI/CD section over GitLab and notice the job executed successfully. You can find the following in running state: Build-client Linting Unit test Now, raise a pull/merge request to the production branch. For the demo, we’ve kept the production branch as the main branch of the Git repository and not the master. Now, we need to merge from master to prod. Note: By default, before you submit the merge request, you will find the checkbox ticked to “delete source branch when merge request is accepted.” Deselect that checkbox. GitLab CI/CD won’t perform a merge unless the pipeline succeeds. Which means it won’t pass the changes until your test scripts are done executing themselves. That is a great feature to help you pass a stable release. Also, you don’t have to wait for the test scripts to be completed so you can perform a merge. All you need to do is click on the button to “Merge when pipeline succeeds,” and the GitLab CI will take care of the merging post your test-script execution. All the jobs, by default, will be executed in parallel, unless you specify them otherwise. This greatly reduces the overall time consumed by the CI process. You can see which builds are being passed and what jobs are running after you hit the merge request. Now, you may notice that an integration test would be running in the detached pipeline, along with the previously discussed jobs in the latest pipeline, i.e., linting and unit testing. Integration testing will ensure your backend and frontend are in good synch and your APIs are responding well. Creating Tag for a Release To create a release, we will need to generate a tag through GitLab CI/CD. Git tags are extremely useful if you want to bookmark some important changes. Now, create a new release from the master branch. You can add any message over there to help you remember what this main release contains or maybe you can add some build release checkpoints as well. Then you can create the tag. As soon as the tag is created, a process will begin to execute. Now, you can see the CI/CD and notice the new pipeline on the top is created in response to the tag creation event. In this pipeline, you will notice that there are four stages. The first one is making sure all the dependencies are installed. This is crucial as you are now creating the final build and you don’t want any confusions, bugs, warnings, to be ignored in that. The next stage is where the build-client is being established. Third, you will notice three kinds of tests that will be running, i.e., unit test cases, linting, and end-to-end tests. End-to-end testing is where you will be incorporating your Selenium testing scripts to perform cross browser testing. You will then have those test scripts executed over an online Selenium Grid offered by LambdaTest. At last, when Selenium testing scripts would be passed and the pipeline will move onto the next stage, where the deployment over the backend and frontend stage will take place. Note: These stages are manually triggered. You will find a play button once the pipeline goes to that stage. That’s it! Once you hit the play button, you can deploy the changes to its respective environments. You can go ahead and validate your Selenium testing scripts over the LambdaTest Automation Dashboard. As you can see in the recorded video of your Selenium testing, this is the same HourGlass application we have been trying to deploy. Note: The web application will be accessible over the localhost using 127.0.0.1:5000. This is the step where you are running a static server to host your frontend file and separate backend server. Later, you can run an end-to-end test using LambdaTest Tunnel. Continuous Testing Using LambdaTest Selenium Grid As you noticed, we ran our script over an online Selenium Grid of LambdaTest. What was the need? Well, doing so can help you quickly and automatically validate your code changes into the staging environment where they are being migrated through your CI/CD pipeline. That way, you are continuously integrating the code for new features, continuously deploying them from one staging environment to another, and now, you are able to continuously test those code changes too. Every time a code is committed to a branch, for which you have your Selenium testing scripts ready, that piece of code will be validated for browser compatibility testing. Allowing you to accelerate the test cycles with the help of continuous testing. Now, let’s ponder a little about how we ran Selenium testing over a locally hosted web application through LambdaTest. Here is the Selenium testing script used for the Jest framework to perform automated browser testing: const webdriver = require('selenium-webdriver'); const { until } = require('selenium-webdriver'); const { By } = require('selenium-webdriver'); const lambdaTunnel = require('@lambdatest/node-tunnel'); const username = process.env.LT_USERNAME || 'Your_LambdaTest_Username'; const accessKey = process.env.LT_ACCESS_KEY || 'Your_LambdaTest_Access_Key'; const capabilities = { build: 'jest-LambdaTest-Single', browserName: 'chrome', version: '72.0', platform: 'WIN10', video: true, network: true, console: true, visual: true, tunnel: true, }; const tunnelInstance = new lambdaTunnel(); const tunnelArguments = { user: process.env.LT_USERNAME || Your_LambdaTest_Username', key: process.env.LT_ACCESS_KEY || 'Your_LambdaTest_Access_Key', }; const getElementById = async (driver, id, timeout = 2000) => { const el = await driver.wait(until.elementLocated(By.id(id)), timeout); return await driver.wait(until.elementIsVisible(el), timeout); }; const getElementByClassName = async (driver, className, timeout = 2000) => { const el = await driver.wait( until.elementLocated(By.className(className)), timeout ); return await driver.wait(until.elementIsVisible(el), timeout); }; const getElementByName = async (driver, name, timeout = 2000) => { const el = await driver.wait(until.elementLocated(By.name(name)), timeout); return await driver.wait(until.elementIsVisible(el), timeout); }; const getElementByXpath = async (driver, xpath, timeout = 2000) => { const el = await driver.wait(until.elementLocated(By.xpath(xpath)), timeout); return await driver.wait(until.elementIsVisible(el), timeout); }; function timeout(ms) { return new Promise(resolve => setTimeout(resolve, ms)); } describe('webdriver', () => { let driver; beforeAll(async () => { const istunnelStarted = await tunnelInstance.start(tunnelArguments); driver = new webdriver.Builder() .usingServer( 'https://' + username + ':' + accessKey + '@hub.lambdatest.com/wd/hub' ) .withCapabilities(capabilities) .build(); // eslint-disable-next-line no-undef await driver.get(`http://127.0.0.1:5000/signup`); // https://hourglass.surge.sh/signup }, 20000); afterAll(async () => { await driver.quit(); await tunnelInstance.stop(); }, 15000); test( 'Signup test', async () => { const nameInput = await getElementById(driver, 'name'); await nameInput.clear(); await nameInput.sendKeys('Mayank'); const emailInput = await getElementById(driver, 'email'); await emailInput.clear(); await emailInput.sendKeys('mybach8@gmail.com'); const passwordInput = await getElementById(driver, 'password'); await passwordInput.clear(); await passwordInput.sendKeys('password'); const cnfPassInput = await getElementById(driver, 'confirmPassword'); await cnfPassInput.clear(); await cnfPassInput.sendKeys('password'); const prefWorkingHours = await getElementById( driver, 'preferredWorkingHours' ); await prefWorkingHours.clear(); await prefWorkingHours.sendKeys('10.0'); const btn = await getElementByClassName( driver, 'LoaderButton btn btn-lg btn-default btn-block' ); await btn.click(); await timeout(2000); const successText = await getElementByClassName( driver, 'registerSuccess' ); const successTextValue = await successText.getText(); console.log(successTextValue); return expect(successTextValue).toContain('Congratulations'); }, 20000 ); test( 'Login test', async () => { await driver.get(`http://127.0.0.1:5000/login`); // https://hourglass.surge.sh/signup // const lnk = await getElementByName(driver, 'li1'); // await lnk.click(); // const lnk1 = await getElementByName(driver, 'li2'); // await lnk1.click(); const emailInput = await getElementById(driver, 'email'); await emailInput.clear(); await emailInput.sendKeys('mybach8@gmail.com'); const passwordInput = await getElementById(driver, 'password'); await passwordInput.clear(); await passwordInput.sendKeys('password'); const btn = await getElementByClassName( driver, 'btn btn-lg btn-default btn-block' ); await btn.click(); await timeout(2000); const successText = await getElementByClassName( driver, 'btn btn-primary' ); const successTextValue = await successText.getText(); console.log(successTextValue); expect(successTextValue).toContain('Manage Time tracks'); }, 20000 ); }); Code Walkthrough There are some tunnel arguments we are using here to work up the LambdaTest Tunnel. There is an npm package that is released by LambdaTest to setup tunnel automatically: const lambdaTunnel = require('@lambdatest/node-tunnel'); What we have done here is, before every test, we are setting up a new WebDriver and this driver is aimed at the public URL of the LambdaTest Selenium Grid Hub. We are using the username and access key provided by the LambdaTest account: const tunnelArguments = { user: process.env.LT_USERNAME || Your_LambdaTest_Username', key: process.env.LT_ACCESS_KEY || 'Your_LambdaTest_Access_Key', }; Then, you provide all the capabilities, such as you want to use the LambdaTest Tunnel over a specific browser, browser version, operating system, with video recording etc: const capabilities = { build: 'jest-LambdaTest-Single', browserName: 'chrome', version: '72.0', platform: 'WIN10', video: true, network: true, console: true, visual: true, tunnel: true, }; We are constructing the URL using the environment variables, i.e., your LambdaTest username and access key: .usingServer( 'https://' + username + ':' + accessKey + '@hub.lambdatest.com/wd/hub' ) Once the remote WebDriver for online Selenium Grid is setup, we are awaiting for the signup page to load: await driver.get(`http://127.0.0.1:5000/signup`); // https://hourglass.surge.sh/signup }, 20000); The below piece of code will ensure tunnel instances will start first and only after that will your Selenium testing script will be executed: const istunnelStarted = await tunnelInstance.start(tunnelArguments); Best Practice After you are done with all the test cases, make sure you delete the tunnel instance. This will help save your concurrency limit available over the LambdaTest Selenium Grid. There is a limited amount of tunnel you can run per count, depending upon the LambdaTest pricing you opt for and if you leave a tunnel in running state, even after your tests are executed. It won’t allow you to use the tunnel in other test cases, so it’s a best practice to close a tunnel after your testing has been completed: afterAll(async () => { await driver.quit(); await tunnelInstance.stop(); }, 15000); Monitoring Logs LambdaTest provides you with an intuitive interface for analyzing the results of Selenium testing scripts. You can get a variety of logs such as network logs, command logs, raw Selenium logs, and metadata. You can also record a video of the entire script execution, along with command-by-command screenshots. You may notice the test has been successfully triggered over the online Selenium Grid of LambdaTest. You can visit the different tabs in the automation dashboard to figure out what went wrong while debugging your scripts. Here is how the logs are provided: Selenium Logs Command Logs Console Logs Eventually, GitLab CI will deploy the backend on Heroku and the frontend on Surge. After opening the URL, you can see frontend is deployed on Serge and my backend is deployed on Heroku. This is done automatically by the GitLab CI/CD pipeline. Now, let’s quickly note down some of the best practices you need to keep in mind for CI/CD before we wrap this GitLab CI/CD tutorial. Best Practices for CI/CD Now that you’ve had a fair-share of knowledge around leveraging GitLab CI pipelines for Selenium testing, I suggest you make notes of these best practices for CI/CD to build better web applications, faster. Build Fast and Test Faster The success of CI/CD systems depends on the execution speed. If CI cycles take a huge amount of time for each commit, developers will find alternate and faster bypasses to get their code integrated quickly. This often involves pathways, which skip tests in favor of optimistic updates. This can cause havoc on production. I think I don’t even need to mention the consequences of integrating untested code. CI/CD Environments Should Be Secured Often ignored, but very crucial to protect your CI/CD environment. It is one the most sensitive pieces of infrastructure to protect as it contains access to your codebase, highly sensitive data and various environments. Furthermore, it is one of the most used systems in a large and high frequency development team. Any outages on CI/CD can cause tremendous loss of productivity and financial losses in the worst cases. CI/CD Should Be the Only Way to Deploy to Production CI/CD pipelines are as successful as the last person using them. All of the effort in developing CI/CD fails if it is not adopted by the team. CI/CD should be strictly the only way to deploy to the prod. In fact, rollbacks should be deployed via the same pipeline. Always Keep Rollback Options in CI/CD Pipelines Ability to rollback a change shouldn’t involve complex procedures. It should be as simple as possible to rollback a change. It’s always better to rollback changes at 3 AM rather than debugging them on production. Fail Early You should run your fastest test early in the pipeline. The idea is to reject the build if it fails any test. Rejecting early saves a lot of time make the turnaround time really small. Run Tests Locally Before Committing to the CI/CD Pipeline CI starts on your local development system. All basic CI tests should first be run on your local system as it is fast, saves time, and conserves CI/CD infra on platform for more critical and later stage pipelines. Tests Should Run in Ephemeral Environment To provide consistent results for CI/CD pipelines it is important that tests run in a fresh state every time. Ephemeral environments are a necessity for making testing idempotent. Containers are a suitable environment as they make it easy to provide a fresh environment. Decouple Deployment and Release As mentioned in the Continuous Delivery introduction, decoupling deployment from the release process makes the release process purely marketing and strategy team decision. This has huge benefits in terms of flexibility and speed. Wrapping Up Kudos!! You have now successfully executed Selenium testing to perform all kinds of checks before the release got deployed. GitLab CI prepared the release process for you and took away all the hassle from doing those last minute checks, by allowing you to integrate with an online Selenium Grid of 3000+ real browsers, ensuring a seamless UI for your web-application. This is how we can actually build a robust CI/CD pipeline for your company or team there. If you still have any questions, feel free to post them in the comment section below. Happy testing!
The pull request (PR) review process, if not set up well in your team, can create a lot of bottlenecks in getting your code merged into the main branch and into production. By adding more context and information automatically to your PRs, you save yourself and your teamwork. Take the scenario of fixing a typo in documentation. If there’s a backlog of PRs that need attention, such a PR may take two days — or longer — just to be approved. Here, learn about continuous merge (CM) with gitStream. gitStream is a tool that allows you to add context and automation to your PRs, classifying PRs based on their complexity. This ensures that a review won't stay in the queue for long as it can be quickly assigned to the right person, immediately approved, or have the appropriate action identified easily. This hands-on article demonstrates how to add gitStream CM to your repository. In this article, you’ll learn: How to configure your repository How to create pull requests (PRs) How to add the CM feature to your PRs Quick gitStream Setup Guide If you’re keen to get all the benefits of gitStream and continuous merge right away, all you need to do is follow these simple steps. If you want to understand how gitStream works, how you can customize it, and more options, it will follow right after. Choose Install for free on gitStream's GitHub marketplace page. Add 2 files to your repo: a) .cm/gitstream.cm b) .github/workflows/gitstream.yml 3. Open a pull request. 4. Set gitStream as a required check. A Comprehensive Guide to gitStream and Continuous Merge Filter functions and context variables are used to effect automated actions, such as adding labels (add-label@v1), assigning reviewers (add-reviewers@v1), and approving requests (approve@v1), among others. Everything is included in a .cm configuration file named gitstream.cm. All instructions to gitStream CM are detailed in the docs found at docs.gitstream.cm. gitStream also uses GitHub Actions to do its work, so you’ll need to add the gitstream.yml file to your GitHub Actions directory at .github/workflows/. The main components to fulfill gitStream’s CM are: The configuration files: gitstream.cm and gitstream.yml The filter functions: Code that tries to check and/or select certain data types from the input for checks during a PR creation The context variables: The inputs fed to the filter functions The automation actions Note: Some steps use Python only for demonstration purposes. It’s not required knowledge. Prerequisites To follow this tutorial, ensure you have the following: Hands-on knowledge of Git and GitHub workings: You must know activities such as creating a repository, PRs, commits, and pushes. A GitHub account Git installed in your working environment You can find and review the final project code on gitStream's GitHub marketplace page linked earlier in this article. Step 1: Set Up gitStream on Your Repo Create an empty repo and give it a name, then install gitStream to it from the marketplace. After installation, you can either: Clone the repository to your environment, or, Create a folder and point it to the repository. This tutorial uses the second option. Create a folder called gitStreamDemo. In this folder, create two directories: .github/workflows and .cm, using the commands in a terminal window as below: mkdir -p .github/workflows mkdir .cm In the .github/workflows folder, create a file called gitstream.yml, and add the following YAML script: name: gitStream workflow automation on: workflow_dispatch: inputs: client_payload: description: The Client payload required: true full_repository: description: the repository name include the owner in `owner/repo_name` format required: true head_ref: description: the head sha required: true base_ref: description: the base ref required: true installation_id: description: the installation id required: false resolver_url: description: the resolver url to pass results to required: true resolver_token: description: Optional resolver token for resolver service required: false default: '' jobs: gitStream: timeout-minutes: 5 # uncomment this condition, if you dont want any automation on dependabot PRs # if: github.actor != 'dependabot[bot]' runs-on: ubuntu-latest name: gitStream workflow automation steps: - name: Evaluate Rules uses: linear-b/gitstream-github-action@v1 id: rules-engine with: full_repository: ${{ github.event.inputs.full_repository } head_ref: ${{ github.event.inputs.head_ref } base_ref: ${{ github.event.inputs.base_ref } client_payload: ${{ github.event.inputs.client_payload } installation_id: ${{ github.event.inputs.installation_id } resolver_url: ${{ github.event.inputs.resolver_url } resolver_token: ${{ github.event.inputs.resolver_token } Next, create a file called gitstream.cm in the .cm folder and add the following code: manifest: version: 1.0 automations: show_estimated_time_to_review: if: - true run: - action : add-label@v1 args: label: "{{ calc.etr } min review" color: {{ '00ff00' if (calc.etr >= 20) else ('7B3F00' if (calc.etr >= 5) else '0044ff') } safe_changes: if: - {{ is.doc_formatting or is.doc_update } run: - action: add-label@v1 args: label: 'documentation changes: PR approved' color: {{'71797e'} - action: approve@v1 domain_review: if: - {{ is.domain_change } run: - action: add-reviewers@v1 args: reviewers: [<listofreviewers>] - action: add-label@v1 args: label: 'domain reviewer assigned' color: {{'71797e'} set_default_comment: if: - true run: - action: add-comment@v1 args: comment: "Hello there. Thank you for creating a pull request with us. A reviewer will soon get in touch." calc: etr: {{ branch | estimatedReviewTime } is: domain_change: {{ files | match(regex=r/domain\//) | some } doc_formatting: {{ source.diff.files | isFormattingChange } doc_update: {{ files | allDocs } In the file, you’ll see the following four automation actions: show_estimated_time_to_review: This automation calculates the estimated time a review to a PR may take. safe_changes: This shows if changes to non-critical components done in a PR are safe, such as document changes. The PR is automatically approved. domain_review: This automation runs to show if a change was made to the domain layer. set_default_comment: This is fired every time a PR is opened and raises an acknowledgment comment to the user that a PR has been created. At the end of the document, there’s a section containing filter functions for the automation actions. The actions are run after certain conditions specified in the filter functions or keys are met. Step 2: Calculating the Time To Review In the first automation, check the value of the etr variable and decide which label to assign to the PR. For more information on how ETR is calculated, check out this blog. Create a file called main.py in the root of your folder. Then, create three folders using the command below: mkdir views domain data Add the following to the main.py file: def show_message(name1, name2): print(f'Hello, {name}. Welcome to the gitStream world') if __name__ == '__main__': print_hi('Mike') Copy the main.py file as is and paste it to the other three folders. Rename them to match the folders’ names (domain.py) for the domain folder. For the dummy documentation file, create a README.md file in the root of your folder and add the following markdown script. # gitStreamDemo Demo Showing How To Set Up gitStream on Your First Repo Now, run these commands to initialize the repository, stage the files for committing, and make a commit, in that order: git init git add . git commit -am “initialization” Next, point the folder to your repository using the command below: git remote add origin https://github.com/<your-username>/<your-repo-name> Finally, push it: git push -u origin main Step 3: Creating the Repository As you may have noticed, there’s a sample bug in the code. In any programming language, you must call the function using its exact name. But in this case, print_hi was called instead of show_message. As a team member or an open-source contributor, you can fix this by opening a PR. First, create a branch called fix-function-call and checkout into the branch using the commands below: git branch fix-function-call git checkout fix-function-call Next, replace the name print_hi with show_message in all the .py files, then commit and push the changes. git commit -am “changed function name” git push --set-upstream origin fix-function-call Now, open your repository in GitHub. You’ll see the following card: Click on Compare & pull request. On the next page, click the Create pull request button. Once the gitStream automation has finished running, you’ll see the domain reviewer assigned tag. Additionally, a comment has been created. Add this Dijkstra’s Shortest Path Algorithm script just below the show_message function in each of the .py files again. These scripts calculate the shortest path for a node in a graph. Commit the changes and then push the code. git commit -am “updates” git push Creating a Safe Change For the final automation, you’ll add text to the README.md file created earlier. Create a new branch and checkout to it. You do so because you’ll need a new PR to demonstrate this automation. git checkout main git branch update_docs git checkout update_docs Then, add this sentence to the README.md file: Continuous Merging is very beneficial to the Open-Source Community. Commit and push. git commit -am “updated the docs” git push --set-upstream origin update_docs When the checks are done, you’ll see a different label with the PR already approved. Help Developers Make the Most of Their Time Reviewing and merging PRs are crucial in contributing to software development and enhancing team productivity. However, being unable to classify PRs by complexity can lead to long wait times or much back-and-forth in the review process. CM remedies this issue by classifying PRs based on the complexity, automating some actions including tagging the appropriate reviewers, assigning them PRs, and approving PRs among others to reduce the backlog. Use gitStream to add CM to your existing repos.
Boris Zaikin
Senior Software Cloud Architect,
Nordcloud GmBH
Pavan Belagatti
Developer Advocate,
Harness
Nicolas Giron
Site Reliability Engineer (SRE),
KumoMind
Alireza Chegini
DevOps Architect / Azure Specialist,
Smartwyre