Class CachedBatchTransformation<TInput, TOutput, TCache>
A batch transformation will transform batches of data. The default batch size are 100000 rows. The batch transformation function allows you to process and modify each batch of data. You can use the BatchSize property to choose a smaller batch size. The batch size must always be smaller than the max buffer size. The default batch size are 1000 rows per batch. The batch transformation is a partial blocking transformation - it will always need at least enough memory to store a whole batch.
Inheritance
Implements
Inherited Members
Namespace: ETLBox.DataFlow
Assembly: ETLBox.dll
Syntax
public class CachedBatchTransformation<TInput, TOutput, TCache> : BatchTransformation<TInput, TOutput>, ILoggableTask, IDataFlowLogging, IDataFlowTransformation<TInput, TOutput>, IDataFlowSource<TOutput>, IDataFlowSource, IDataFlowDestination<TInput>, IDataFlowDestination, IDataFlowComponent
Type Parameters
| Name | Description |
|---|---|
| TInput | Type of ingoing data. |
| TOutput | Type of outgoing data. |
| TCache | Type of data in the cache. |
Constructors
CachedBatchTransformation()
Declaration
public CachedBatchTransformation()
CachedBatchTransformation(int, Func<TInput[], IEnumerable<TCache>, TOutput[]>)
Declaration
public CachedBatchTransformation(int batchSize, Func<TInput[], IEnumerable<TCache>, TOutput[]> batchTransformationFunc)
Parameters
| Type | Name | Description |
|---|---|---|
| int | batchSize | The size of each batch |
| System.Func<T1, T2, TResult><TInput[], System.Collections.Generic.IEnumerable<T><TCache>, TOutput[]> | batchTransformationFunc | Sets the BatchTransformationFunc |
CachedBatchTransformation(int)
Declaration
public CachedBatchTransformation(int batchSize)
Parameters
| Type | Name | Description |
|---|---|---|
| int | batchSize | The size of each batch |
Properties
BatchTransformationFunc
Declaration
public Func<TInput[], IEnumerable<TCache>, TOutput[]> BatchTransformationFunc { get; set; }
Property Value
| Type | Description |
|---|---|
| System.Func<T1, T2, TResult><TInput[], System.Collections.Generic.IEnumerable<T><TCache>, TOutput[]> |
CacheManager
The CacheManager to use when caching data
Declaration
public ICacheManager<TInput, TCache> CacheManager { get; }
Property Value
| Type | Description |
|---|---|
| ICacheManager<TInput, TCache> |
FillCacheAfterTranformation
If set to true, the incoming row will be added to the cache after the transformation func has been invoked.
Declaration
public bool FillCacheAfterTranformation { get; set; }
Property Value
| Type | Description |
|---|---|
| bool |
MaxCacheSize
The maximum amount of previously records to store
Declaration
public int MaxCacheSize { get; set; }
Property Value
| Type | Description |
|---|---|
| int |
TaskName
A name to identify the task or component. Every component or task comes with a default name that can be overwritten.
Declaration
public override string TaskName { get; set; }
Property Value
| Type | Description |
|---|---|
| string |
Overrides
Methods
CheckParameter()
Declaration
protected override void CheckParameter()
Overrides
InvokeBatchTransformationFunc(TInput[])
Declaration
protected override TOutput[] InvokeBatchTransformationFunc(TInput[] batch)
Parameters
| Type | Name | Description |
|---|---|---|
| TInput[] | batch |
Returns
| Type | Description |
|---|---|
| TOutput[] |
Overrides
InvokeInitActionOnce()
Declaration
protected override void InvokeInitActionOnce()