Class BatchTransformation<TInput, TOutput>
A batch transformation will transform batches of data. The default batch size are 100000 rows.
The batch transformation function allows you to process and modify each batch of data.
You can use the BatchSize property to choose a smaller batch size. The batch size must always be smaller
than the max buffer size. The default batch size are 1000 rows per batch.
The batch transformation is a partial blocking transformation - it will always need at least enough
memory to store a whole batch.
Inheritance
object
BatchTransformation<TInput, TOutput>
Inherited Members
object.Equals(object)
object.Equals(object, object)
object.GetHashCode()
object.GetType()
object.MemberwiseClone()
object.ReferenceEquals(object, object)
object.ToString()
Namespace: ETLBox.DataFlow
Assembly: ETLBox.dll
public class BatchTransformation<TInput, TOutput> : DataFlowTransformation<TInput, TOutput>, ILoggableTask, IDataFlowLogging, IDataFlowTransformation<TInput, TOutput>, IDataFlowSource<TOutput>, IDataFlowSource, IDataFlowDestination<TInput>, IDataFlowDestination, IDataFlowComponent
Type Parameters
| Name | Description |
|---|
| TInput | Type of ingoing data. |
| TOutput | Type of outgoing data. |
Constructors
Declaration
public BatchTransformation()
Declaration
public BatchTransformation(int batchSize, Func<TInput[], TOutput[]> batchTransformationFunc)
Parameters
| Type | Name | Description |
|---|
| int | batchSize | The size of each batch |
| System.Func<T, TResult><TInput[], TOutput[]> | batchTransformationFunc | Sets the BatchTransformationFunc |
Declaration
public BatchTransformation(int batchSize)
Parameters
| Type | Name | Description |
|---|
| int | batchSize | The size of each batch |
Fields
Declaration
protected bool WasInitActionInvoked
Field Value
Properties
Declaration
public virtual int BatchSize { get; set; }
Property Value
The transformation Func that is executed on each array of input data. It returns
another array as output data - the output array can have a different length than
the input array.
Declaration
public Func<TInput[], TOutput[]> BatchTransformationFunc { get; set; }
Property Value
| Type | Description |
|---|
| System.Func<T, TResult><TInput[], TOutput[]> | |
The init action is executed shortly before the first data row is processed.
Declaration
public Action InitAction { get; set; }
Property Value
| Type | Description |
|---|
| System.Action | |
SourceBlock from the underlying TPL.Dataflow which is used as output buffer for the component.
Declaration
public override ISourceBlock<TOutput> SourceBlock { get; }
Property Value
| Type | Description |
|---|
| ISourceBlock<><TOutput> | |
Overrides
ETLBox.DataFlow.DataFlowSource<TOutput>.SourceBlock
By default, all null values in the batch returned from the batch transformation func
are filtered out. Set this option to true to avoid this behavior.
Declaration
public bool SuppressNullValueFilter { get; set; }
Property Value
TargetBlock from the underlying TPL.Dataflow which is used as input buffer for the component.
Declaration
public override ITargetBlock<TInput> TargetBlock { get; }
Property Value
| Type | Description |
|---|
| ITargetBlock<><TInput> | |
Overrides
ETLBox.DataFlow.DataFlowTransformation<TInput, TOutput>.TargetBlock
A name to identify the task or component. Every component or task comes
with a default name that can be overwritten.
Declaration
public override string TaskName { get; set; }
Property Value
Overrides
Methods
Declaration
protected void CheckAndAdjustBatchSize()
Declaration
protected override void CheckParameter()
Overrides
Declaration
protected override void CleanUpOnFaulted(Exception e)
Parameters
| Type | Name | Description |
|---|
| System.Exception | e | |
Overrides
Declaration
protected override void CleanUpOnSuccess()
Overrides
Declaration
protected override void InitComponent()
Overrides
Declaration
protected virtual TOutput[] InvokeBatchTransformationFunc(TInput[] batch)
Parameters
| Type | Name | Description |
|---|
| TInput[] | batch | |
Returns
Declaration
protected virtual void InvokeInitActionOnce()
Implements