Data Management Overview
Data management within the system is the structured process of organizing, storing, and controlling data. It plays a vital role in ensuring data is accurate, accessible, and secure. This functionality supports efficient data handling, enabling collaboration and analysis among teams. By organizing data properly, users can streamline workflows and maintain data integrity, leading to improved decision-making and operational efficiency.
The Benefits of Effective Data Management
Effective data management allows users to:
Organize large datasets systematically.
Ensure data consistency across teams and versions.
Protect data through controlled access and permissions.
Quickly retrieve and share accurate data for collaborative work.
Now, we can dive into the detailed explanations of Data Group, Data contract, Data Sample, and Run Outputs, as they form integral components of data management in the system.
Organizing Your Data
The system structures data into three key components to ensure effective management and organization:
Data Group
A Data Group acts as a container for Data contracts, providing a way to categorize data according to projects, teams, or departments. It helps users manage large datasets efficiently by grouping related tables together.
Key Characteristics:
Organizes multiple Data Contracts under one logical category.
Controls access and permissions for all data within the group.
Simplifies data sharing across teams by managing permissions at the group level.
Data Contract
A Data Contract defines the structure of the data, specifying columns, data types, and formats, without containing the actual data. It serves as a blueprint for storing and validating multiple Data Samples.
Key Characteristics:
Defines the data structure, including column names and types.
Requires a small data sample (up to 1 MB) to establish its format.
Supports multiple Data Samples that must be compatible with the same structure.
Data Sample
A Data Sample stores the actual data, aligned with the structure defined by the Data Contract. Each version represents a distinct snapshot of the data, allowing for changes and updates while maintaining consistency.
Key Characteristics:
Stores the actual data for analysis and use.
Must match the structure defined by the Data Contract.
Versions are linked to their Data Contract and cannot be shared independently but can be added directly to a process.
This structure allows for precise organization, version control, and secure sharing within teams.
How to Upload and Organize Data
Create a Data Group: Start by creating a data group, which acts as a folder for your data contracts.
Create a Data Contract: Within the data group, create a data contract and upload a small amount of data (up to 1 MB) to define its structure.
Create a Data Sample: Upload the actual data as a data sample, ensuring it matches the data contract's structure.
Using Managed Data for Process Creation
Once your data is structured into Data Groups, Tables, and Versions, you can leverage it to create efficient workflows and processes within the system. With organized data:
Users can track changes across different versions, maintaining a clear history of data iterations.
Teams can easily share structured data, facilitating collaboration.
Accurate and consistent data is readily available for analysis, process creation, and reporting.
By following these steps and utilizing effective data management practices, you ensure that your data remains organized, secure, and ready for any data-driven task.