Ingestion

Data Ingestion is the process of streaming-in massive amounts of data into the system from different external sources for running analytics and other operations required by the business.

Click the Ingestion tab to view and manage Data Ingestion Settings including Indexer and Processing.

Indexer	Description	Default
Enabled	Enable/disable file analysis and processing (files will queue)	Enabled
Allowed Route Failures	Enter Number of sequential failures a route can have before being disabled	3
Processing
Enabled	Enable/disable file analysis and processing (files will queue)	Enabled
Enable Smart Types	Enable/disable smart detection of Output Schemas	Disabled
Forgive Non-Zero PGP Exit Enabled	If enabled, PGP decryption will continue on ERROR and if an output is produced. Enable at your own risk.	Empty
Enable Pre-Processing Hook	Enable a pre-processing script to customize file formatting, encoding or content	Disabled
Force Processing of Duplicates	Duplicates will be processed as normal files with only a warning. Requires user to set ‘duplicate’ as Type and mark as ‘Done’ in order to move file through the process. Note that if this feature is activated, the Progress Duplicates feature becomes unavailable (even if previously checked). Also, note that if you force process a parent, the children will force process as well.	Disabled
Copy Parent Tags	Copy a parent’s custom attributes to all of it’s derived child files (applies to file only)	Disabled
Virtual Files: Change Owner	Allow users to change the owner when creating virtual files	Disabled
Excel: Do Not Process Hidden Tabs	Set this to true to disable processing of hidden Excel tabs	Disabled
Split Large Files: Minimum	Minimum Bytes/Records to Split. Set this to 0 to disable splitting files.	Blank
Global Output Schema Mappings	Allows the use of a Regex pattern and a qualifier for a global output schema	Blank

Be sure to click the Save button for each Ingestion Setting where entries or changes have been made.