Log Collection Services

This article mainly explains how to use the Alibaba Cloud log collection plug-in to connect to the Alibaba Cloud Log Service, and collect and send the logs generated during the operation of the components on the Kato platform to the Alibaba Cloud Log Service through the plug-in.

Prerequisites

  • There is a default Alibaba Cloud Log Service Collection Plugin in the team.
  • Runable components that persist logs to log files.
  • Available Alibaba Cloud accounts.

Plug-in Installation and Activation

On the My Plugin page in the team view, select Alibaba Cloud Log Service Collection Plugin, click Install, and the component can use the plugin after the installation is complete.

After the installation is complete, on the plug-in page of the component management page, in the list of unopened, find the Alibaba Cloud Log Service Collection Plug-in, and click the Activate button on the right to activate the plug-in. After that, the plug-in will appear in the opened list.

Parameter Configuration

You can click the View Configuration button on the right side of the plug-in to view the configuration parameter information of the plug-in.

| Parameter name | Default value | Description | | —————————— | —————— ————————————– | ———– ————————————————– ————————————————– —– | | ALIYUN_LOGTAIL_USER_ID | None | Aliyun user ID, please refer to Configure User ID for configuration. | | ALIYUN_LOGTAIL_USER_DEFINED_ID | None | Machine group user-defined ID, please refer to Create User-defined ID Machine Group for configuration. | | ALIYUN_LOGTAIL_CONFIG | /etc/ilogtail/conf/${cn-huhehaote}/ilogtail_config.json | Logtail collects service configuration parameters, which vary according to the location of the Log Service Project and the network type. | | ALIYUN_LOG_ENV_TAGS | _pod_name_|_pod_ip_|_namespace_|_node_name_|_node_ip_ | Tag configuration when collecting logs, specify specific values ​​through environment variables. |

ALIYUN_LOGTAIL_CONFIG needs to be set according to the memory network type where the Log Service Project is located. Users need to know the area where their Kato platform is deployed in Alibaba Cloud and what type of network they are using. If it is a public network, the format is: region-internet, for example: Virginia 1 (Virginia) is us-east-Internet. If it is an Alibaba Cloud intranet, the format is region. For example: Huadong 1 (Hangzhou) is cn-hangzhou.

After the configuration is complete, click Update Configuration, you need to update the restart component to make it take effect.

Shared Storage

The plug-in needs to share the log file directory of the component to collect logs, and the log file directory of the component needs to be shared with the plug-in. This can be achieved by mounting storage.

On the storage page of the component management page, add storage of type temporary storage, and fill in the mount path to the path where the component will generate the log file. Such as /var/log/nginx. **

After configuring the storage, you need to update and restart component to make it take effect.

Alibaba Cloud Log Service Configuration

In Alibaba Cloud Log Service Home, select Single Line-Log Text in the Access Data section to enter the configuration process of Log Service.

####Choose log space

  • Create a Project or use an existing Project. Need to pay attention to the selection of the project area, to be consistent with the area where Kato is located.
  • Create a log library LogStore or use an existing log library LogStore.

####Create machine group

Here choose self-built machine group. The reason is that the custom configuration of the self-built machine group is used to identify the log collection work of different projects and realize the isolation of log collection.

  • After selecting the self-built machine group, directly Confirm the installation is complete without any installation operations.
  • Fill in the machine group name, machine group topic information and user-defined identification.

Select Self-built machine group, and the machine group identification must select User-defined identification. Moreover, the user-defined identification configuration should be consistent with the value of the ALIYUN_LOGTAIL_USER_DEFINED_ID parameter in the plug-in configuration. **

####Machine group configuration

Select the machine group created in the previous step and move it from the source machine group to the application machine group list.

There will be a reminder of There is no machine configuration in the currently selected machine group, is it mandatory to skip, which can be ignored, just click skip.

####Logtail configuration

This step configures the log path to be collected and some high-level configurations.

Note that the log path configured here needs to be consistent with the path where the component shared storage is mounted. Otherwise, no logs can be collected. **

####Query analysis configuration

This step is used as a verification step for log collection, and a module of Preview Data will be provided for users to determine whether the log collection is correct. If the configuration is correct, the log list that appeared at that time will appear here. If there is no log preview data for a long time, please check whether the operation is wrong.

At this point, the log collection is completed by the Alibaba Cloud log service, and you can see that the logs have been normally collected in the Alibaba Cloud log service platform, and more complex analysis logic can be performed on the collected data.

Common Problems

Plug-in parameter configuration errors make it impossible to collect logs

You can check the plug-in parameter configuration, confirm whether ALIYUN_LOGTAIL_USER_DEFINED_ID is consistent with the identity set by the Alibaba Cloud machine group, and confirm whether the ALIYUN_LOGTAIL_CONFIG parameter is consistent with the Alibaba Cloud region where the Kato platform is running.

Failure to collect logs due to not configured shared storage

You can add temporary storage type storage on the storage page of the component management page, and make it take effect after re-update.

The configuration is correct, and it is created in the storage, but the logs cannot be collected

You can try to restart the component, confirm that all the configurations have taken effect, and then confirm whether the log collection is successful.