Kato Java-Maven Project from Source

Overview

According to a large number of user feedback, we found that in the process of building a java-maven project from Kato source code, the most problematic part is the failure to obtain components.

Such as:

[ERROR] Failed to execute goal on project bq-insurance-third-party: Could not resolve dependencies for project···

Such an error.

This document will explain in detail how to make the correct configuration to obtain the required components of the project. Also, to answer the question “I can build locally, why can’t I build in Kato?”

Reading this document requires readers to have a certain understanding of Kato Principles of Building Java Maven Projects.

Download and Install Maven

Kato provides a variety of maven versions by default for users to choose. The corresponding versions and corresponding resource addresses are shown in the following table:

Maven versionGet address
3.3.1http://lang.gridworkz/jvm/maven/maven-3.3.1.tar.gz
3.0.5http://lang.gridworkz/jvm/maven/maven-3.0.5.tar.gz
3.1.1http://lang.gridworkz/jvm/maven/maven-3.1.1.tar.gz
3.2.5http://lang.gridworkz/jvm/maven/maven-3.2.5.tar.gz
3.3.9http://lang.gridworkz/jvm/maven/maven-3.3.9.tar.gz

If you encounter Maven build failure, please confirm the current Maven version first. If you are not sure, you can download the above resources and try to build it locally.

Clear the Build Cache

Kato provides a cache for the build environment of each service. The Maven project provides a cache for the maven installation directory, configuration directory, and local repository directory. Users can clear the cache through the following settings.

The user should always clear the build cache until the first build is successful. This prevents incomplete or wrong packages from being cached, causing the build to fail all the time. Remember, the build will prioritize getting components from the cache.

Default Maven build and run environment setting analysis

When Kato source code builds java-maven project, it provides default build and run environment settings.

Enable Clear Build Cache: Click to clear the build cache before each build. It is not turned on by default. Maven version: Select the Maven version, the default is 3.3.1. Disable Maven Mirror: Disable the Maven Mirror setting after clicking it, that is, no longer use the rbd-repo (Artifactory) service when obtaining components, and go directly to the central repository or the repository specified in pom.xml to obtain the components. The subsequent MAVEN MIRROR OF MAVEN MIRROR URL setting is invalid. It is not disabled by default. MAVEN MIRROR OF: After the mirror function is turned on, this parameter specifies which repositories are mirrored and cached. The default is the central repository (central). When designated as *, all repositories will be mirrored and cached. MAVEN MIRROR URL: Specify the mirror repository address, the default is maven.gridworkz (that is, the rbd-repo service address) that comes with Kato. If the user has a private server used by the company, it is recommended to directly specify the address, and the specified format is similar: http://IP:8081/nexus/content/groups/public/ Maven Build Parameters: The default setting is to ignore unit tests. The user sets it by himself according to the project situation. Maven component global parameters: The default value is clean dependency:list install. It should be noted that dependency:list needs to download a specific maven plugin. Therefore, when the user is in an offline environment and there is no corresponding component in the private server used, the build failure will inevitably occur. Please change to clean install. MAVEN build Java parameter configuration: The default configuration is -Xmx1024m. This option specifies the memory used when maven is built, and is set according to the user’s environment.

It should be pointed out that when specifying the MAVEN MIRROR OF parameter, you need to consider whether the specified repository can be identified. The repository name is specified in the settings.xml file used by maven, and only the above-mentioned configurations are added to the settings.xml used by Kato by default! ! ! Therefore, the custom repository name used by the user for daily construction will not be recognized. In this case, you can specify * to cache all components; or, use your own settings.xml file to replace the Kato default file.

Custom Settings.xml

Users can configure special environment variables to specify the settings.xml that they use when building the project locally. Once specified, the options in the default build environment configuration will be invalid.

Such a configuration will be an ultimate solution, users can build it locally, then it can also be built in Kato. Because after using the specified setting.xml file, everything in the Kato build environment is no longer different from the local one.

Users can put the settings.xml that they used in the past in the project source code directory. When the file is in the source code root directory, please do as follows: Set the environment variable BUILD_MAVEN_SETTINGS_PATH=/app/settings.xml to use this file.

When the Kato source code is built, all files in the source directory will be stored in the /app directory by default, so the path of the file becomes /app/settings.xml

If there is sensitive information in setting.xml, it should not appear in the source directory. Then you can upload it to a place such as object storage and provide the download address. then: Set the environment variable BUILD_MAVEN_SETTINGS_URL=http://somewhere/settings.xml to use this file.

Deploy local private server repository

Some users do not have private repositories in their companies, and they hope to use Kato source code to build maven projects in an offline environment. Then you need to use the rbd-repo (Artifactory) service to build your own repository private server and upload the dependent packages.

Visit http://management node IP:8081 and log in with the administrator account (admin/password).

Create a Maven repository of type Local. Example Create a Maven repository of type Local with the name repo-local

Upload your own jar package to the local repository repo-local

View dependency declaration information

Add repo-local to the libs-release virtual repository

Visit http://<management node>:8081/artifactory/list/libs-release/ or visit maven.gridworkz for management node to see if you can list your newly added components.

If the user already has a complete and usable repository folder, he can also use the import function to upload the entire repository to the repo-local local repository.

Two methods are provided for complete upload: mount import from directory, or upload zip. In the first method, it should be noted that the upload path is to be uploaded in the path in the rbd-repo container, so the path mounting needs to be performed in advance. In the second way, you need to pay attention to the file upload size limit, which can be set in admin-Configuration-General Configuration.

I can build locally, why can’t I build in Kato?

Reading through this document, we can find that there are many details that need to be paid attention to when executing a maven build. These tiny details may cause the build to fail.

But one thing for sure is that if it can be built locally, it can be built in a Kato environment with the same network conditions. Because when Kato executes source code builds, the principles used are no different from ordinary maven builds. What needs attention is the slight difference in the build environment.

The following are the ideas for troubleshooting when encountering such doubts:

  • Version difference: This difference includes the difference between the maven version and the version of the JDK (even if it is a different minor version under the same major version). If you encounter an error for which the cause cannot be determined, this will be the first aspect to be checked. For how to choose the JDK version and how to deal with the impact of version differences, please refer to the document: KATO source code build JAVA project selects JDK

  • GZIP STDIN NOT IN GZIP FORMAT: If the build log reports this error, it can basically be determined that it is a failure to obtain the JDK or Maven installation package, combined with the document Source code build prompt GZIP STDIN NOT IN GZIP FORMAT solve.

  • Build cache: Again, users should always clear the build cache until the build is successful until the first build is successful. This prevents incomplete or wrong packages from being cached, causing the build to fail all the time. Remember, the build will prioritize getting components from the cache.

  • Failed to obtain component: The answer to this question is not unique, and there are many possible situations.

    First, combine the component download address in the build log to determine whether to use the repository private server when obtaining the component fails, using the default private server (maven.gridworkz) or a user-defined private server (artifactory or nexus specified by the user) .

    If the Mirror function is disabled, the central repository will be used by default. At this time, it is necessary to determine whether the network can access the central repository, and whether the current component exists in the central repository.

    If the Mirror function is not disabled and the Kato default repository private server (maven.gridworkz) is used, the central repository will be the proxy by default. At this time, it is necessary to determine whether the network can access the central repository, and whether the current component exists in the central repository.

    If the Mirror function is not disabled and a user-defined private server is used. It is necessary to determine whether the network can access the designated repository private server, and whether the current component exists in the designated repository private server.

  • 401 authentication failure: If the build reports an error:

 [ERROR] Failed to execute goal org.apache.maven.plugins:maven-deploy-plugin:2.7:deploy (default-deploy) on project dx-id: Failed to deploy artifacts: Could not transfer artifact com.dx.application: dx-id:pom:0.0.1-20190727.012351-2 from/to snapshots (http://******:8081/artifactory/libs-release): Failed to transfer file: http://*** ****:8081/artifactory/libs-release/com/dx/application/dx-id/0.0.1-SNAPSHOT/dx-id-0.0.1-20190727.012351-2.pom. Return code is: 401, ReasonPhrase : Unauthorized. -> [Help 1]

Note that access to the repository private server specified by the user requires authentication information, and the authentication information is generally stored in the settings.xml file that the user uses daily. Therefore, the best way to solve this problem is to use the custom Settings.xml method mentioned above. Again, Custom Settings.xml exists as the ultimate solution, and it is also applicable to other component acquisition failures caused by the special settings of the user repository private server.

  • My repository is sufficient for the construction of my project, but it still reports that there are components not found: Kato’s default Maven component global parameter is clean dependency:list install. It should be noted that dependency:list needs to download a specific maven plugin. Therefore, when the user is in an offline environment and there is no corresponding component in the private server used, the build failure will inevitably occur. Please change to clean install.

Use rbd-repo to proxy other repository private servers: Users can use the rbd-repo component to proxy other remote repository private servers. However, strange and strange problems may occur in the transmission of components between different repository private servers. Therefore, we recommend that users use MAVEN MIRROR URL to directly specify the remote repository address instead of using the rbd-repo proxy.

Summary

I can build locally, why can’t I build in Kato environment? This question is the one that users most often ask us when there is a problem with the source code build function. There have even been instances where users abandon Kato because of this, and we feel sad about it. It is undeniable that the knowledge points involved in building this function from source code are more extensive, esoteric, and obscure than other functions.

But it should be pointed out that the construction principle used by Kato Java-Maven is consistent with the ordinary Maven construction principle. So in essence, it can be built locally, and it can definitely be built in the Kato environment, but the difference in the details of the setting of the build environment, how to obtain the components, etc., has a great impact on the build result. Therefore, the most important thing to use Kato source code to build a Java-Maven project is to find these subtle differences and smooth out the differences between the local environment and the Kato build environment.

This document describes in detail the various detailed settings and operations of the Kato source code to build a Java-Maven project. Many of the details have been listed, which can be regarded as a detailed summary of troubleshooting when the source code build fails. It is hoped that Kato users will have a deeper understanding of Kato’s source code construction function after reading the entire document.