Namespace ETLBox.DataFlow.Transformations

Classes

Aggregation

Aggregates data by the given aggregation methods. The aggregate is a partial-blocking transformation - only the aggregation values are stored in separate memory objects. When all rows have been processed by the aggregation, the aggregated values are written into the output.

Aggregation<TInput, TOutput>

Aggregates data by the given aggregation methods. The aggregate is a partial-blocking transformation - only the aggregation values are stored in separate memory objects. When all rows have been processed by the aggregation, the aggregated values are written into the output.

BatchTransformation

A batch transformation will transform batches of data. The default batch size are 100000 rows. The batch transformation function allows you to process and modify each batch of data. You can use the BatchSize property to choose a smaller batch size. The batch size must always be smaller than the max buffer size. The default batch size are 1000 rows per batch. The batch transformation is a partial blocking transformation - it will always need at least enough memory to store a whole batch.

BatchTransformation<TInput>

A batch transformation will transform batches of data. The default batch size are 100000 rows. The batch transformation function allows you to process and modify each batch of data. You can use the BatchSize property to choose a smaller batch size. The batch size must always be smaller than the max buffer size. The default batch size are 1000 rows per batch. The batch transformation is a partial blocking transformation - it will always need at least enough memory to store a whole batch.

BatchTransformation<TInput, TOutput>

A batch transformation will transform batches of data. The default batch size are 100000 rows. The batch transformation function allows you to process and modify each batch of data. You can use the BatchSize property to choose a smaller batch size. The batch size must always be smaller than the max buffer size. The default batch size are 1000 rows per batch. The batch transformation is a partial blocking transformation - it will always need at least enough memory to store a whole batch.

BlockTransformation

A block transformation will wait for all data from the flow to be loaded into its buffer. After all data is in the buffer, the transformation function is executed for the complete data and the result posted into the targets. The block transformations allows you to access all data in the flow in one generic collection. But as this block any processing until all data is buffered, it will also need to store the whole data in memory.

BlockTransformation<TInput>

A block transformation will wait for all data from the flow to be loaded into its buffer. After all data is in the buffer, the transformation function is executed for the complete data and the result posted into the targets. The block transformations allows you to access all data in the flow in one generic collection. But as this block any processing until all data is buffered, it will also need to store the whole data in memory.

BlockTransformation<TInput, TOutput>

A block transformation will wait for all data from the flow to be loaded into its buffer. After all data is in the buffer, the transformation function is executed for the complete data and the result posted into the targets. The block transformations allows you to access all data in the flow in one generic collection. But as this block any processing until all data is buffered, it will also need to store the whole data in memory.

CachedBatchTransformation

CachedBatchTransformation<TInput>

A batch transformation will transform batches of data. The default batch size are 100000 rows. The batch transformation function allows you to process and modify each batch of data. You can use the BatchSize property to choose a smaller batch size. The batch size must always be smaller than the max buffer size. The default batch size are 1000 rows per batch. The batch transformation is a partial blocking transformation - it will always need at least enough memory to store a whole batch.

CachedBatchTransformation<TInput, TOutput>

A batch transformation will transform batches of data. The default batch size are 100000 rows. The batch transformation function allows you to process and modify each batch of data. You can use the BatchSize property to choose a smaller batch size. The batch size must always be smaller than the max buffer size. The default batch size are 1000 rows per batch. The batch transformation is a partial blocking transformation - it will always need at least enough memory to store a whole batch.

CachedBatchTransformation<TInput, TOutput, TCache>

A batch transformation will transform batches of data. The default batch size are 100000 rows. The batch transformation function allows you to process and modify each batch of data. You can use the BatchSize property to choose a smaller batch size. The batch size must always be smaller than the max buffer size. The default batch size are 1000 rows per batch. The batch transformation is a partial blocking transformation - it will always need at least enough memory to store a whole batch.

CachedRowTransformation

Implemented by transformations that have one or more inputs of the same type and one or more outputs of the same type

CachedRowTransformation<TInput>

Implemented by transformations that have one or more inputs of the same type and one or more outputs of the same type

CachedRowTransformation<TInput, TOutput>

Implemented by transformations that have one or more inputs of the same type and one or more outputs of the same type

CachedRowTransformation<TInput, TOutput, TCache>

ColumnRename

ColumnRename allows you to rename the column or properties names of your ingoing data. This transformation works with objects, ExpandoObjects and arrays as input data type.
Provide a column mapping with the old and the new name. The mapping can also be automatically retrieved from existing ColumnMap attributes. For arrays provide the array index and the new name.

ColumnRename<TInput>

ColumnRename allows you to rename the column or properties names of your ingoing data. This transformation works with objects, ExpandoObjects and arrays as input data type.
Provide a column mapping with the old and the new name. The mapping can also be automatically retrieved from existing ColumnMap attributes. For arrays provide the array index and the new name.

CrossJoin

The CrossJoin allows you to combine every record from one input with every record from the other input. The input for the first table will be loaded into memory before join starts. Then every incoming row will be joined with every row of the InMemory-Table using the CrossJoinFunc function. The InMemory target should always be the target of the smaller amount of data to reduce memory consumption and processing time.

CrossJoin<TInput>

The CrossJoin allows you to combine every record from one input with every record from the other input. The input for the first table will be loaded into memory before join starts. Then every incoming row will be joined with every row of the InMemory-Table using the CrossJoinFunc function. The InMemory target should always be the target of the smaller amount of data to reduce memory consumption and processing time.

CrossJoin<TInput1, TInput2, TOutput>

The CrossJoin allows you to combine every record from one input with every record from the other input. The input for the first table will be loaded into memory before join starts. Then every incoming row will be joined with every row of the InMemory-Table using the CrossJoinFunc function. The InMemory target should always be the target of the smaller amount of data to reduce memory consumption and processing time.

CrossJoin<TInput1, TInput2, TOutput>.InMemoryDestination<TInput>

Distinct

Distinct<TInput>

LookupTransformation

The lookup transformation enriches the incoming data with data from the lookup source. Data from the lookup source is read into memory when the first record arrives. For each incoming row, the lookup tries to find a matching record in the loaded source data and uses this record to enrich the ingoing data.

LookupTransformation<TInput, TSource>

The lookup transformation enriches the incoming data with data from the lookup source. Data from the lookup source is read into memory when the first record arrives. For each incoming row, the lookup tries to find a matching record in the loaded source data and uses this record to enrich the ingoing data.

LookupTransformation<TInput, TSource>.PartialDbCacheSettings

Defines properties to configure a partial cache in a Lookup.

MergeJoin

Will join data from the two inputs into one output. Make sure both inputs are sorted or in the right order. Each row from the left join target will be merged with a row from the right join target. If the amount of ingoing data is unevenly distributed, the last rows will be joined with null values.

You can define a match condition that let you only merge matching records. This will change the match behavior a little bit. By assuming that the input is sorted, not matching records will be joined with null then. This can be compared with a left or right join.

MergeJoin<TInput>

Will join data from the two inputs into one output. Make sure both inputs are sorted or in the right order. Each row from the left join target will be merged with a row from the right join target. If the amount of ingoing data is unevenly distributed, the last rows will be joined with null values.

You can define a match condition that let you only merge matching records. This will change the match behavior a little bit. By assuming that the input is sorted, not matching records will be joined with null then. This can be compared with a left or right join.

MergeJoin<TInput1, TInput2, TOutput>

Will join data from the two inputs into one output. Make sure both inputs are sorted or in the right order. Each row from the left join target will be merged with a row from the right join target. If the amount of ingoing data is unevenly distributed, the last rows will be joined with null values.

You can define a match condition that let you only merge matching records. This will change the match behavior a little bit. By assuming that the input is sorted, not matching records will be joined with null then. This can be compared with a left or right join.

Multicast

A multicast broadcast data from the input into two or more outputs. Every linked component will receive a copy of the rows that the Multicast receives. There is no limit how many target the Multicast can be linked to.

Multicast<TInput>

A multicast broadcast data from the input into two or more outputs. Every linked component will receive a copy of the rows that the Multicast receives. There is no limit how many target the Multicast can be linked to.

RowDuplication

Creates one or more duplicates of your incoming row. Use the CanDuplicate property if you want to duplicate only particular rows.

RowDuplication<TInput>

Creates one or more duplicates of your incoming row. Use the CanDuplicate property if you want to duplicate only particular rows.

RowMultiplication

This transformation allow you to transform one row of your input data into multiple rows.

RowMultiplication<TInput>

This transformation allow you to transform one row of your input data into multiple rows.

RowMultiplication<TInput, TOutput>

This transformation allow you to transform one row of your input data into multiple rows.

RowTransformation

The RowTransformation will apply the transformation function to each row of data.

RowTransformation<TInput>

The RowTransformation will apply the transformation function to each row of data.

RowTransformation<TInput, TOutput>

The RowTransformation will apply the transformation function to each row of data.

Sort

Sorts the incoming data with by the given comparison function. This is a blocking transformation - no output will be produced until all input data has arrived in the transformation.

Sort<TInput>

Sorts the incoming data with by the given comparison function. This is a blocking transformation - no output will be produced until all input data has arrived in the transformation.

XmlSchemaValidation

Validate XML code in your incoming data against a XML schema definition. You need to define how the XML string can be read from your data row and the schema definition. Rows with a schema that is not valid are send to the error output..

XmlSchemaValidation<TInput>

Validate XML code in your incoming data against a XML schema definition. You need to define how the XML string can be read from your data row and the schema definition. Rows with a schema that is not valid are send to the error output..

Enums

AggregationMethod

CacheMode