Resource management

Function introduction

Resource management is based on the system load management upgrade iteration, its function on the one handIt is based on the detailed query data in the system, aggregated according to different dimensions, and displays a series of indicators related to system usage and resource consumption in order and proportion. It aims to help system administrators understand the distribution of system resource consumption, locate abnormal queries, and terminate, diagnose, and optimize unreasonable queries. On the other hand, through the product data support, it helps the administrator to understand the business resource composition and the use of business resources of the Shenze system from a global perspective, and provides data insight and action guidance for customers to reduce costs and increase efficiency.

Query task: This part consists of a list of observable indicators based on the details of a query task and records the queries initiated by scheduled tasks or users. The specific indicators are as follows:

Statistical dimension: Query data by task level, login account, and project name
Indicators: Query count, number of queries, Average time consumed, Failure rate, CPU overhead, Memory Consumption, file scanning volume, and Time distribution

Business resources: This part provides statistics and inventory of table data, labels, clusters, overview, bookmarks and other data resources from a global perspective, and visually displays the composition of business resources and the use of business resources in the system. Specific indicators are as follows:

Meta event: stored in recent 30 days, referenced times in recent 30 days, received status, display status
Event attributes: Number of referenced events and display status in the last 30 days
User attributes: Number of user references and display status in the last 30 days
User group: number of queries in recent 30 days, number of references in recent 30 days, and update mode
User label: Number of queries in the last 30 days, number of references in the last 30 days, and update mode
Overview: Query times in the last 30 days, PV in the last 30 days, update status
Bookmarks: Queries in the last 30 days, PV in the last 30 days

entrance Limits of authority

entrance：Click the Resource Management button in the upper right corner of the navigation to enter the product interface.

permission：

SBP1.1: This function allows only the admin account and administrator role account to view the query data of all projects, and the administrator role account can only see the query data of authorized projects.

SBP1.4 and above：Controlled by the permission point View Resource Management,Prefabricated role: The administrator can view resource management by default. Other roles cannot view resource management by default.

Query task

Query task statistics

Query task overview

This section displays the query times, number of queries, average time consuming, and failure rate based on the query data delimit by time range, filtering criteria, and statistical dimensions, in order to provide users with a preview of resource usage.

When you select a day to query the time range, the day and week are provided. When you hover over the day and week, the specific comparison date is displayed.

Query task statistics

This part shows the trend change and proportion distribution of task statistics indicators based on the query data delimit by time range, screening conditions, and statistical dimensions.

By default, the observable indicators are query times. You can change the number of queries, average time consuming, and failure rate. You can hover over indicators to view their descriptions.
Hover over any point in the trend chart or a certain part of the pie chart to display the actual value of the corresponding statistical indicator at that point in time.
Click any point in the trend chart or a part of the pie chart to expand the query details and export the details.

Query Task System resource consumption statistics

This section shows the trend change and proportion distribution of resource consumption statistics indicators based on the query data specified in the time range, filtering conditions, and statistical dimensions.

The default observable indicators are CPU consumption. You can change the memory consumption and file scan amount. You can hover over the indicator to view its description.
Hover over any point in the trend chart or a certain part of the pie chart to display the actual value of the corresponding statistical indicator at that point in time.
Click any point in the trend chart or a part of the pie chart to expand the query details and export the details.

Query task distribution statistics

This part shows the time distribution of the query task based on the query data delimited by time range and filtering conditions.

Hover over the bar chart to display the actual value of the corresponding statistical indicator at this point in time.
Click the bar chart to expand the query details and export the details.
Time distribution scales default from less than 30 seconds to more than 600 seconds.

Query task Details

Query task details record all queries initiated by scheduled system tasks or users in the environment, including the queries in Progress and Completed.

Click the "question mark" icon at the head of the list table to see the specific meaning of the list fields.
Click "Export Details" : Export all the content after the list filter.
Click View in a query task. The detailed information about the diagnosis report is displayed.
Click Download for a single query task to download the diagnostic report.
For a running task, click the "Stop running" button of a single query task to terminate the running job. This function point SBP1.1 is only open to the admin account, and SBP1.4 is controlled by the "Stop Running" permission point. You can configure to allow the query initiated by all people to terminate, or you can configure to allow the query initiated by yourself to terminate.

Usage scenario

Scenario 1: How to use Extended Tags to analyze query tasks?

For the user feedback of "slow query", there are many reasons, many times because the user launched an unreasonable query caused. Extended tags are special identification of query tasks. They help system administrators pay attention to these tags during attribution analysis of slow queries, locate the cause of slow queries, and optimize improper configurations.

How do I use Failure Rate to analyze query tasks?

For example: As shown in the following figure, in a period of time, the query task is regular and the failure rate is as high as 100%. If you click the abnormal point on the line chart, you can see that it is the background task of the same account. You can consider that the query of the scheduled task is abnormal or the useless scheduled task wastes system resources, and then you can consider optimizing or deleting the query completely.

Scenario 3: How do I troubleshoot abnormal query tasks when a user reports that the query is slow in a certain period?

If it is the current time, some (several) unreasonably large queries in the running may occupy a large number of system resources, resulting in query queuing and resource shortage. At this time need: 1) enter < Query task details page > - < In progress > tab page 2) Terminate the abnormal job by combining the Peak Task Proportion and Task Time indicator

If it is a historical time, 1) The Query Task Execution Statistics Page is displayed. 2) Select a time range (the time range covers the node whose query time is slow, for example, 7 days before and after the query). 3) Filter the login accounts of users whose feedback is slow

Analyze the reasons: 1) If there is an obvious trend change in the line chart before and after the slow time node is queried, you can click the exception point to expand the job details, and locate the abnormal jobs through comprehensive statistical dimensions and indicators such as "task time", "CPU cost", "memory consumption", "Speed evaluation", and "extended label" to consider whether the abnormal jobs are due to the shortage of system resources at that time. Or the user itself initiated unreasonable query caused, and consider the optimization of the abnormal job configuration. 2) If there is no obvious trend change in the line charts before and after the slow query time nodes, you can determine that the system slow is not caused by the query operation.

Business resource

The Business Resources section provides statistics on resource usage for event tables, user tables, user tags, user clusters, bookmarks, and overview.

The statistics of user groups and user labels can be viewed only after licenses are issued.

Profile area

This section presents data metrics that help users determine whether each business resource under each project needs to be optimized and cleaned.

Detail data

This section displays the detailed data of each service resource, so that users can further determine the service resources that need to be optimized and cleaned.

Click "Export Details" : Export all the content after the list filter.
Hover over the question mark icon to view the indicator description.
Click on a single resource[View]: View the query or referenced data of a resource.
Click the "Enter Resource Page" of a single resource to enter the details page of the specific resource (only user groups, user tags, and overview types of resource support).

Tag deletion

This feature requires SA 2.5.3+ & SBP 1.4.2+ version support, and only resources like overview and bookmark have this feature. Take the overview as an example below:

Click "Mark for Deletion" for a single resource or select multiple resources and click "Bulk Mark for Deletion". The overview will be immediately moved to the Deletion Details list. It is expected to be permanently deleted after 30 days from entering the Deletion Details list. Meanwhile, the overview will not be visible on the Overview Management list and the overview board in the front end.

In the Deletion Details list, click "Mark for Recovery" for a single resource or select multiple resources and click "Bulk Mark for Recovery". The overview will be restored to the Overview Details list, and it will be visible in the Overview Management list and the overview board in the front end.