Source

The Source Node serves as the entry point of any workflow, where data is imported and made available for subsequent steps. It facilitates the initial data intake, allowing it to be utilized in transformations, processes, or analyses within the workflow.

Adding the Source Node

To initiate a workflow, users must activate the Source Node, which can be done in the following ways:

Drag and Drop: Locate the Source Node in the left panel and drag it onto the Process Builder canvas.
Add Source Option: Click the Add Source button directly on the Process Builder canvas to place the Source Node.

Once the Source Node is on the canvas, it must be configured to define where and how the data is sourced.

Understanding Data Organization

The system organizes data into three main structures that work together:

Data Group: A Data Group acts as a container or folder for grouping related data contracts and their versions. It helps in organizing the data, making it easier to locate and manage.
Data Contract: A Data Contract defines the structure of the dataset, such as the column names, data types, and any constraints that need to be applied.
Data Sample: A Data Sample refers to a specific instance of the data stored under a Data Contract. It ensures that you have access to historical data samples for reference or processing.

Configuration

The Source Node offers three configuration options for specifying how and where the data will be retrieved:

Existing Source
File Upload
Server Connection

Each configuration method is designed to meet specific needs in your workflow, so understanding their differences can help you choose the best approach.

Existing Source

The Existing Source configuration allows you to reuse pre-defined and structured data available within the system. This option is ideal for workflows requiring consistent and validated datasets.

Let's see the steps to configure:

Data Group: Select an existing Data Group from the dropdown menu.
Data Contract: Choose a Data Contract within the selected Data Group.
Data Sample: Pick a specific Data Sample from the selected Data Contract.

File Upload

The File Upload configuration lets you upload new data directly into the system. This is useful for creating entirely new datasets.

Let's see the steps to configure:

New Data Group:
Provide a name for the Data Group to organize the data.
New Data Contract:
Enter a name for the Data Contract and define its structure using a sample dataset.
New Data Sample:
Specify a name for the Data Sample to store the actual data.
Upload File:
Browse or drag-and-drop an XLSX or CSV file into the upload area. The file must adhere to the following requirements:
- File type: XLSX or CSV
- Maximum size: 1 MB
- Matches the structure of the Data Contract

Server Connection

The Server Connection configuration enables you to connect to external databases, allowing seamless integration with larger or dynamic datasets.

Let's see the steps to configure:

Data Group: Select a Data Group to organize the imported data.
Connection Name: Choose an existing connection available within your team.
Database Name: Select the database associated with the chosen connection.
Retrieve Data Options:
- Using Table:
  - Provide the Schema Name and Table Name from the database.
  - Add a Condition to filter data as needed (reference the table structure for clarity).
  - Click Validate to ensure the condition is correct:
    - Success: A message “Condition validated successfully!” will appear.
    - Failure: An error message will display, prompting you to correct the condition.
  - Once validated, click Preview Run to view the results.
- Using Query:
  - Enter a custom SQL query in the Query Builder field.
  - Click Validate to check query correctness.
  - After validation, click Preview Run to view the query results.

Demo Video

For a visual walkthrough of the Source Node configuration, watch the demo below:
Watch the Demo

Last modified: 21 February 2025