Management and Maintenance of Dataset

Overview

As an intermediate link for data access and visualization display, DataSet provides centralized management of data assets for enterprises, such as group management, search filtering, monitoring scheduling, and statistical applications. It helps data platform administrators effectively monitor the use and maintenance of data, prevent data leakage and abuse, and protect data security.

In addition, administrators can also archive and classify data for better management and maintenance, and quickly locate target data when necessary. It also helps monitor the scheduling of various models, helps administrators discover and solve problems in a timely manner, maximizes the reliability and stability of data, and supports the operation and development of internal business within the organization.

Dataset List

After entering the Dataset module, the overall list of business models under the current project is displayed, as shown in the figure below:

Business Dataset List Page Functions:

Group List

The entire interface of the DataSet list page can be divided into the left "Business Model Group" and the right "Business Model List"; when positioning to a specific group, the right side will display all the business models under that group and their basic information.

• Convenient for users to build a classification framework for data assets according to business lines, organizational structure, or data characteristics, which is conducive to the maintenance and accumulation of data assets
• Supports searching for group names, making it easy for users to locate and find data assets
• Supports pinning core-focused business model groups. The display order of the groups is arranged in descending order based on the pinning operation time. The pinning effect is effective for a single account
• Supports managing groups. For detailed configuration capabilities, please refer to ：Group Management

Set group members

Supports "Member Settings" for each group, controlling the data and management permissions of the current group and the business models under the group, as shown in the figure below:

• Group Administrator: Can manage all members in the group, including adding and deleting members, and assigning permission types to members; can publish personal business models to the group.
• Management permission: The group name and the names of business models within the group are visible in the public space; can publish personal business models to the group.
• Viewing permission: The group name and the names of business models within the group are visible in the public space.

Column Settings

Support hiding/showing settings for the display content of table columns, as shown in the figure below:

• Business Model Name, Operation Bar, Default selection, Not removable
• The selected order does not affect the display order of the column's display
• The effect of the settings applies to individual accounts

Refresh Data

When the update status of the business model is "update failed" or "data exception", a "refresh" button will appear next to the update status.

• Operation entrance: as shown in the figure below:
• Effect: After clicking "to refresh" the data, the data will be immediately updated to reacquire the data.

Scheduling Renewal

For routine updated business models, the system will prompt users "approaching expiration" when the update cycle is close to the end, as shown in the figure below; If the schedule has expired, the business model will no longer continue routine update tasks. If you want to continue using the latest data, it is recommended to renew the schedule as soon as possible.

1、Trigger Timing

• When the original update cycle exceeds 14 days, a reminder will be sent 7 days before expiration
• When the original update cycle is not more than 14 days, a reminder will be sent 3 days before expiration

2. Prompt Range:

• Creator of the business model
• Collaborative Manager
• Administrator

3. Operation Handling: After clicking "Renewal", a delay configuration popup will appear to extend the expiration date by days, default is 90 days, with a maximum of 180 days (configurable based on business requirements)

Dataset Management Center

The management center module provides the ability to centrally manage and maintain resources, helps users to manage the lifecycle of user data in a fine-grained manner, monitors data status in real time, understands the data application status, and provides effective data support for subsequent visualization display.

Operation Entry: Secondary directory under the visual menu

Group Management

In the group management module of the dataset, users can build a classification framework for data assets based on business lines, organizational structure, data characteristics, etc. They can perform operations such as adding, deleting, modifying, and querying business models in groups. It is flexible, convenient, and can quickly respond to business changes, which is conducive to the precipitation and maintenance of data assets.

1. Create a group:

• Default group: System pre-installed, exists during initialization, cannot change the name, and cannot be deleted
• Create a new group:
  • Default is 50, can be adjusted according to actual business needs
  • Group name cannot be duplicated
• Groups are sorted by default in reverse chronological order of creation

2. Delete:

• If there are no business models under the group, it can be deleted after confirmation.
• If there are business models under the group, the operator should be prompted to transfer them. Select an existing group from the drop-down list and search for the group name. After successful selection, the group can be deleted.

3. Rename:

• The group name must adhere to the naming convention for groups. The group name cannot be duplicated within the same project.
• After modifying the group name, the group information in the corresponding configuration drawer of the business model will be synchronized.

3. Member Settings:

Support "Member Settings" for each group, control the data and management permissions of the current group and the business models under the group, as shown in the figure below:

• Group Administrators: Can manage all members in the group, including adding and removing members, and assigning permission types to members; Can publish personal business models to the group.
• Management Permission: The group name and the names of business models within the group are visible in the public space. Can publish personal business models to the group.
• Viewing Permission: The group name and the names of business models within the group are visible in the public space.

Business Model Management

Business Model Management displays all public and personal business models under the project for administrators to manage all business models, as shown in the figure below:

Update Management

Update Record

Monitor the details of all data resource schedules; Focus on update tasks that are in abnormal state to ensure the real-time and accuracy of data.

Support searching for the name of business model and filtering based on "execution start time", "triggering method", and "update status" to quickly locate the target information of update records.

Update Plan

View upcoming data update tasks within a certain period of time, allowing users to adjust the frequency and triggering time of data updates based on the concurrency density of update tasks in different time periods, balancing the consumption and usage of scheduling system resources.

• Support viewing the business model of scheduled tasks in the next 36 hours.
• Support viewing the business model to quickly navigate to the corresponding business model detail page and view the specific details of the model.
• Support viewing the application statistics of the model, clicking to enter the reference list module, please refer to the following: Reference List Module

Update Due

The "Update Due" feature in the Settings allows for batch extension of expiring business models:

• Supports multiple selections, can batch extend routine update tasks for selected models
• Default extension is 90 days, with a maximum of 180 days (configurable based on business needs)
• Supports operations such as "View Data" and "Reference List", for details on viewing the reference list, refer to:Reference List Module
  

Reference List

The Dataset provides the capability to track data lineage. In the Application Statistics module in the Settings, we can quickly understand the dependency relationship between business models, reports and chart components; identify valuable data assets, which helps manage resources, troubleshoot issues, and assist administrators in making accurate judgments.

• Supports searching for business model, report, and component names
• Displays which reports the business model is used in and which components the applications are in.