This example demonstrates how to use this Transformation to flag records with missing values, allowing those records to be filtered out to ensure that only complete records are sent for further processing.
Pump temperature readings are evaluated for missing values. The missing values are printed and the other events are passed to the filter, with the addition of a boolean to indicate missing values. The filter passes the events without missing data to the data conversion, and these events are also printed.
Drag the Missing Values Detector Transformation onto the canvas. Link the input endpoint to the pump data, the output endpoint to the filter, and the missing only endpoint to the printer. Save the Data Stream.
Select the Agent and click Configure. Keep the default Collection.
Select the attributes to monitor. In this case, PumpId and Temperature. Apply the changes and save the Data Stream.
Select the Filter and click Configure. Note the filter is configured to allow events where the MissingValue attribute is equal to false.
Publish the Data Stream and open the Live Data View to view the results.
The Event Printer prints events that have values for both PumpId and Temperature, while the Event Printer (missing) prints events that have one or both values missing.
Missing Values Detector CSV.csv