Class BatchTransformation<TInput, TOutput>
A batch transformation will transform batches of data. The default batch size are 100000 rows.
The batch transformation function allows you to process and modify each batch of data.
You can use the BatchSize property to choose a smaller batch size. The batch size must always be smaller
than the max buffer size. The default batch size are 1000 rows per batch.
The batch transformation is a partial blocking transformation - it will always need at least enough
memory to store a whole batch.
Inheritance
BatchTransformation<TInput, TOutput>
Namespace: ETLBox.DataFlow
Assembly: ETLBox.dll
public class BatchTransformation<TInput, TOutput> : DataFlowTransformation<TInput, TOutput>, IDataFlowLogging, IDataFlowTransformation<TInput, TOutput>, IDataFlowSource<TOutput>, IDataFlowSource, IDataFlowDestination<TInput>, IDataFlowDestination, IDataFlowComponent, ILoggableTask
Type Parameters
| Name | Description |
|---|
| TInput | Type of ingoing data. |
| TOutput | Type of outgoing data. |
Constructors
Declaration
public BatchTransformation()
Declaration
public BatchTransformation(int batchSize)
Parameters
| Type | Name | Description |
|---|
| int | batchSize | The size of each batch |
Declaration
public BatchTransformation(int batchSize, Func<TInput[], TOutput[]> batchTransformationFunc)
Parameters
Fields
Declaration
protected bool WasInitActionInvoked
Field Value
Properties
Declaration
public virtual int BatchSize { get; set; }
Property Value
The transformation Func that is executed on each array of input data. It returns
another array as output data - the output array can have a different length than
the input array.
Declaration
public Func<TInput[], TOutput[]> BatchTransformationFunc { get; set; }
Property Value
| Type | Description |
|---|
| Func<TInput[], TOutput[]> | |
The init action is executed shortly before the first data row is processed.
Declaration
public Action InitAction { get; set; }
Property Value
SourceBlock from the underlying TPL.Dataflow which is used as output buffer for the component.
Declaration
public override ISourceBlock<TOutput> SourceBlock { get; }
Property Value
Overrides
By default, all null values in the batch returned from the batch transformation func
are filtered out. Set this option to true to avoid this behavior.
Declaration
public bool SuppressNullValueFilter { get; set; }
Property Value
TargetBlock from the underlying TPL.Dataflow which is used as input buffer for the component.
Declaration
public override ITargetBlock<TInput> TargetBlock { get; }
Property Value
Overrides
Methods
Declaration
protected void CheckAndAdjustBatchSize()
Declaration
protected override void CheckParameter()
Overrides
Declaration
protected override void CleanUpOnFaulted(Exception e)
Parameters
Overrides
Declaration
protected override void CleanUpOnSuccess()
Overrides
Declaration
protected override void InitComponent()
Overrides
Declaration
protected virtual TOutput[] InvokeBatchTransformationFunc(TInput[] batch)
Parameters
| Type | Name | Description |
|---|
| TInput[] | batch | |
Returns
Declaration
protected virtual void InvokeInitActionOnce()
Declaration
protected override void Reset()
Overrides
Implements