Match

🔄 Merge: Combining and Controlling File Outputs

📌 Concept

Merge is a configurable operation used to combine, unify, or enrich datasets—such as customer, transaction, or address files—into a single structured output. It supports logic for joining, appending, or excluding records based on defined rules.

Merge can also enforce constraints such as data type, processing time window, or client-specific rules, making it a flexible tool for managing how incoming files are consolidated.

🔗 Configure Record Handling (Join / Select / Omit)

In the Output Schema, navigate to the Match tab and click the Add button to insert new rows. For each row, select the appropriate Output Schema and Match Type to define how records will be Joined, Selected, or Omitted.

🛠️ Key Functionalities

1. File Combination & Unification

• Combine multiple related files (e.g., customer, transaction, and item files) into a single output.
• Supports joins across files that may differ in length, order, or structure.
• Ensures distinct, unified outputs by aligning on shared keys or logic.

2. Select and Omit Logic

• Select: Keep records when a join or condition is met.
• Omit: Drop records that match a specified condition.

This allows precise control over which records make it into the final output, based on defined join behavior.

3. Scoped Processing by Type, Time, and Client

Merge configurations can be scoped to:

• Only run when files are processed as a specific type (e.g., customer, transaction)
• Operate within a specific time window
• Apply to data for a specific client

This allows targeted and controlled processing across large, complex data flows.

4. Bulk Configuration Support

• Merge jobs can be created from the Merge screen manually.
• Alternatively, select multiple files in the DataOps screen and use Bulk > Add to Merge Job to configure the merge.

This enables both automated and hands-on management of merge logic across datasets.

📊 Example Use Cases

• Multi-File Join

Combine files like customers.csv, transactions.csv, and items.csv into one master file, repeating records as needed based on shared keys.

• Union of Like-Type Files

Receive multiple customer files from different sources and union them into one standardized customer dataset.

• Suppression Join

Combine customer records with a suppression flag file (e.g., IDs with DNM/DNR indicators), adding necessary flags to the output without exposing PII.

✅ Summary

• Merge enables flexible, rule-based unification of multiple files into a structured output.
• Supports joining, unioning, and suppressing records via Select and Omit rules.
• Can be configured for specific file types, time windows, and clients.
• Usable both manually and in bulk for streamlined, large-scale data operations.