GitHub Enterprise Integration

🚧

Early Adopter Release

This feature is currently in the early adopter release. You can join the early adopter program from the Optional Features and Early Access section in the admin area. Please note that during this phase, we may publish new agent versions more frequently and without prior notice. This might require you to update the agent more often than expected after the early adopter phase. Learn more about updating the agent.

How to provide feedback?

• Book a meeting with us using the booking tool.
• Leave your feedback as a note on the ProductBoard card.
• If you find a bug, submit a ticket to support.

Overview

The GitHub Enterprise integration helps you automatically discover and manage self-built software in your IT landscape using GitOps. This integration captures metadata and simplifies the discovery of self-built software by using manifest files in GitHub. These files describe the properties of self-built software artifacts. This allows developers to operate within the familiar GitHub environment while ensuring that project details are accurately represented and maintained within the architectural landscape, promoting improved oversight and alignment.

If your organization uses a different Source Code Management (SCM) platform, such as Azure, GitLab, or Bitbucket, you can set up self-built discovery by placing a manifest in the root directory of your Git repository. Currently, we do not provide out-of-the-box integrations for other SCM platforms. To learn how to set up self-built discovery using a manifest file, visit Microservice Discovery in Your CI/CD Pipeline.

📘

Note

The SAP LeanIX Technology Risk and Compliance product is required for the GitHub Enterprise integration.

Benefits

• Automated discovery: The integration automatically identifies and integrates your self-built software projects from GitHub Enterprise into your SAP LeanIX inventory. This automation saves you time and effort by eliminating the need for manual import of microservices.
• Seamless collaboration: The integration connects to your version control system and uses the workflows your developers already know. This makes it easier for developers to contribute to your architectural practice.
• Enhanced architectural visibility: The integration improves the visibility of your self-built software within your IT landscape. It provides a comprehensive view of your entire architectural landscape, aiding in decision-making.
• Open-source GitHub agent: The SAP LeanIX GitHub agent is an open-source project, which enhances the transparency of the integration. The agent discovers self-built software in self-hosted GitHub Enterprise setups and communicates this information to SAP LeanIX workspaces.

Scope and Limitations

The GitHub Enterprise integration helps you discover microservices by placing a YAML manifest file in the root directory of your repository. This integration does not include:

• Generating and uploading Software Bill of Materials (SBOM). For more information on generating SBOMs, refer to Generating SBOMs.
• Automated discovery of microservices without placing a YAML manifest file in your repository.

Prerequisites

• In SAP LeanIX:
  • Get admin access to your workspace.
  • Ensure that the workspace is provisioned for self-built software discovery. For details, see Workspace Setup for Self-Built Software Discovery.
• Create a technical user with admin permissions. For instructions, see Create a Technical User.
• Get admin permissions for your organization on GitHub. This is necessary to install and manage the GitHub App.
• Coordinate with the relevant stakeholders in your organization to deploy the on-premise GitHub agent on your infrastructure. They will also need to provide the URL where the agent is accessible within your network for the GitHub App creation.

Configuring the Integration

To set up the GitHub Enterprise integration, deploy an agent-based setup within your infrastructure. This agent, provided as a Docker container, can be deployed on-premises, such as on a Kubernetes cluster. The agent continuously syncs changes from the discovered manifest files (leanix.yaml) in GitHub to your SAP LeanIX workspace. This ensures your workspace is always up-to-date with the latest changes. To authenticate and facilitate this process, the agent requires a GitHub App. To learn more about GitHub Apps, refer to the GitHub documentation.

The following sections guide you through the steps to configure this integration effectively.

📘

Note

You can configure the integration with a single GitHub instance. This instance can be associated with multiple organizations.

Step 1: Create a GitHub App

Creating a GitHub App requires a valid URL for sending webhook events. This URL must correspond to the network address of the SAP LeanIX agent, which will look like https://host:port after deployment. The agent relies on receiving updates from GitHub to function correctly. These updates are sent to the provided URL.

If you're unsure how to obtain a stable URL for the deployed agent, consult with your platform engineering or DevOps team. This step is crucial to ensure the smooth operation of the integration.

To create a GitHub App, follow these steps:

1. In the administration area of SAP LeanIX, navigate to the Integrations section, then click Add Integration.

2. On the page listing available integrations, locate the GitHub Enterprise integration, then click Configure.

   

3. On the integration configuration page, enter the base URL of your GitHub instance (for example, https://leanix-github.com) and the network address where you plan to deploy the agent (for example, https://host:port). Click Create GitHub App.

   

4. In GitHub, generate a private key and save it. This key, along with the GitHub App ID, will be required when deploying the agent.

   Configuration details are prefilled based on the information provided in the previous steps. For details on which events the agent needs permission to listen to, refer to GitHub App Permissions.

   

5. In GitHub, get the GitHub App ID in Settings > Developer settings > GitHub Apps > {Your GitHub App} > General.

   

6. Ensure that the GitHub App can be installed on any account in your GitHub instance.

   

   Alternatively, you can adjust this setting after creating the app in Settings > Developer settings > GitHub Apps > {Your GitHub App} > Advanced > Make public.

   

GitHub App Permissions

To ensure the GitHub App functions correctly, you need to grant the following minimum permissions.

Repository permissions:​

• Commit statuses: Read-only
• Administration: Read-only
• Contents: Read and write
• Metadata: Read-only

Organization permissions:​

• Members: Read-only

GitHub webhook events subscribed to:

• Meta: Triggered when this app is deleted and the associated hook is removed.
• Label: Triggered when a label is created, edited, or deleted.
• Organization: Triggered when an organization is created, deleted, renamed, a member is invited, added, or removed.
• Public: Triggered when a repository changes from private to public.
• Push: Triggered when a Git push is made to a repository.
• Repository: Triggered when a repository is created, deleted, archived, unarchived, publicized, privatized, edited, renamed, or transferred.

Step 2: Deploy the Agent

After you've set up a GitHub App, the next step is to deploy the agent. To do this, copy the docker-compose code from the configuration page in SAP LeanIX, then click Done.

The following snippet shows an example of docker-compose code. Ensure to copy the code from your workspace.

version: '3.8'
 
 services:
  github-agent:
    image: ghcr.io/leanix/leanix-github-agent:v0.1.0
    platform: linux/amd64
    ports:
      - "8080:8080"
    environment:
      - GITHUB_ENTERPRISE_BASE_URL=https://your-url
      - GITHUB_APP_ID={{App id which can be found in the GitHub App page}} 
      - PEM_FILE=/privateKey.pem
      - LEANIX_DOMAIN=your-subdomain.leanix.net
      - LEANIX_TECHNICAL_USER_TOKEN={{Administrator token generated by EAM}}
    volumes:
      - {{Location of the pem key downloaded from the GitHub app page}}:/privateKey.pem

The following table lists environment variables.

For a reliable and consistent operation, we suggest deploying the container using a container orchestration platform such as Kubernetes. This is crucial as GitHub expects the webhook receiver (the agent) to respond within 10 seconds of receiving an event, and will terminate the connection if it doesn't. For details, refer to the GitHub documentation.

For local testing, you can use docker-compose and a tunneling tool like ngrok to redirect traffic to your local setup.

👍

Recommendation

To monitor the agent's status and ensure it's running properly, use the health route host:port/actuator/health.

Step 3: Install the GitHub App

After creating a GitHub App and deploying the agent, the next step is to install the GitHub App on the relevant GitHub organizations. This grants the SAP LeanIX agent access to these organizations.

You can install the app in one of the following ways:

• From the integration page in SAP LeanIX: Here, you can find a KPI indicating how many organizations are currently covered by the agent. To install the app on more organizations, click on the relevant link.

  

  You're redirected to the installation page in GitHub.

  

• In the GitHub App settings: You can directly navigate to this page by going to https://{host url of your GitHub instance}/settings/apps/{name of the created GitHub app}. This will take you to the settings page of the GitHub App you created, where you can manage its installation across your organizations.

Step 4: Add Manifest Files to Projects

Manifest files, specifically leanix.yaml files, are essential tools in the config-as-code practice. They enable you to describe and discover microservices from GitHub in SAP LeanIX. Place leanix.yaml files in the projects that correspond to the microservice or software artifact you want to describe. One manifest file is used to describe one microservice.

To view the structure and schema of the leanix.yaml file, see Manifest File and Schema.

When configuring the manifest file, you can start with basic elements first (such as name and description) and then gradually evolve your governance to include more aspects (like linking to teams). Any changes made to the YAML file will be reflected in SAP LeanIX in near real-time.

For monorepos, where multiple deployed microservices are contained in a single repository, you can place the leanix.yaml file in the directories where a deployed service resides. The integration will find and process them accordingly. Here’s an example configuration:

backend
  Dockerfile
  leanix.yaml
  src
    main.py
    requirements.txt
frontend
  Dockerfile
  leanix.yaml
  package.json
  src
    index.js
    ...

Step 5: Monitor and Manage the Integration

After setting up the integration, you can check the integration status at the top of the page. Here, you can find the deployed version of the GitHub agent. If the version appears in red, this indicates that a more recent version is available for update.

The following metrics provide an integration summary:

• Microservices Discovered: This shows the number of valid manifest files in connected repositories. You can open a boilerplate manifest file, modify it as needed, and add it to your repository to discover more microservices.
• GitHub Organizations: This indicates the number of organizations where the GitHub App is installed, along with the total number of organizations. You can navigate to GitHub from here to install the GitHub App on other organizations.
• Sync Errors: Displays the number of errors that occurred in the last seven days. From here, you can navigate to synchronization logs for the integration.

Viewing Discovered Microservices

On the integration page, you can view discovered microservices and their respective repositories. You can navigate to microservice fact sheets from here. If a repository is linked to a discovered microservice, clicking the repository name takes you to the manifest file. If no microservices are discovered for a repository, clicking the repository name directs you to the repository's root.

Updating the Agent Version

We periodically release new versions of the SAP LeanIX GitHub agent to ensure optimal performance. The currently deployed version is displayed at the top of the GitHub integration page. If a new version becomes available, you'll see a notification.

To update the GitHub agent to the latest version, follow these steps:

1. Pull the latest version of the agent ghcr.io/leanix/leanix-github-agent:x.x.x.
2. Redeploy the updated agent.

📘

Note

During the early adopter release phase, we may release new versions of the GitHub agent more frequently to ensure thorough testing. As a result, you may need to update the agent version more often.

Deleting the Integration

If needed, you can delete the GitHub Enterprise integration. To do this, in the upper-right corner of the integration page, click the three-dot icon > Delete Integration. This action will remove the link to GitHub from all microservice fact sheets that were imported through the integration. However, this will not delete or archive any fact sheets, as you may have added some details to them manually. Therefore, you will need to delete these fact sheets yourself if needed.

To identify fact sheets that are no longer connected to GitHub following the deletion of the integration, you can apply a filter to show only those with a GitHub Enterprise external ID set to null. This will help you find and manage all fact sheets that have lost their connection to GitHub due to the deletion of the integration.

To complete the deletion, do the following:

1. Remove the GitHub App from your organization’s settings. For instructions, refer to the GitHub documentation.
2. Delete the deployments from your agent.

Troubleshooting

Here are some common problems and their solutions:

• Authentication errors: While the GitHub App is designed to minimize such issues, you may still encounter authentication errors if a required permission was not selected. You can find these errors in the synchronization log in SAP LeanIX. If this happens, review the permissions you've selected during the setup process to ensure all necessary permissions are granted.
• Errors parsing the manifest file: If there are invalid entries in the manifest file, they appear in the synchronization log in SAP LeanIX. To fix this, check the manifest file to ensure it has the correct structure and syntax.

FAQs

What happens if I delete a leanix.yaml file?

Deleting a leanix.yaml file will unlink the corresponding fact sheet in SAP LeanIX. The external ID for GitHub will be removed, and the fact sheet will no longer be linked to GitHub. However, the fact sheet itself will not be automatically deleted, as it may contain other relevant information. If the fact sheet is no longer needed, you can archive it manually.

What happens if I manually add or overwrite information in fact sheets linked to a manifest file?

The manifest file is considered the source of truth for all the information it contains. Therefore, any manual changes made to fields owned by the integration will be overwritten on the next update to ensure consistency with the manifest file. However, you can manually update fields or relations not managed by the integration.

How do you ensure that two manifest files don't write information to the same fact sheet?

The GitHub integration uses the location of the leanix.yaml file as a unique identifier for the microservice, which is then used to create the external ID in the fact sheet. Therefore, even if you copy an identical manifest to two locations, it will create two separate fact sheets. However, if the name attribute in the manifest is identical, the integration will fail to process the second manifest file, as name in the manifest file corresponds to displayName in SAP LeanIX, which is unique.

I want to restructure a repository with synced leanix.yaml files. What should I consider before doing that?

The location of the manifest file in your repository serves as a unique identifier within SAP LeanIX. If you're planning to restructure a repository and move a leanix.yaml file, you should:

1. Move the leanix.yaml file to your desired new location. For example, if the file used to reside in acme-org/sample-repo/src, you might move it to acme-org/sample-repo/new_sub_directory.

2. Manually update the externalId in the manifest file to point to the old external ID (acme-org/sample-repo/src). This ensures the connection between the fact sheet and the moved leanix.yaml file remains intact.

   YAML

   version: 1
   metadata:
     name: disputes-service-v1
     externalId: acme-org/sample-repo/src
     description: |
       A microservice responsible for payment disputes.
       This service handles payment transaction disputes and is an integral part of our payment ecosystem.