# LookupTransformation {{< rawhtml >}}

Class LookupTransformation<TInput, TSource>

The lookup transformation enriches the incoming data with data from the lookup source. Data from the lookup source is read into memory when the first record arrives. For each incoming row, the lookup tries to find a matching record in the loaded source data and uses this record to enrich the ingoing data.

Inheritance
object
LoggableTask
DataFlowComponent
DataFlowSource<TInput>
DataFlowTransformation<TInput, TInput>
LookupTransformation<TInput, TSource>
LookupTransformation
LookupTransformation
Implements
IDataFlowTransformation<TInput, TInput>
IDataFlowSource<TInput>
IDataFlowSource
IDataFlowDestination<TInput>
IDataFlowDestination
IDataFlowComponent
ILoggableTask
Inherited Members
DataFlowTransformation<TInput, TInput>.TargetBlock
DataFlowTransformation<TInput, TInput>.CompleteBuffer()
DataFlowTransformation<TInput, TInput>.FaultBuffer(Exception)
DataFlowSource<TInput>.LinkTo(IDataFlowDestination<TInput>)
DataFlowSource<TInput>.LinkTo(IDataFlowDestination<TInput>, Predicate<TInput>)
DataFlowSource<TInput>.LinkTo(IDataFlowDestination<TInput>, Predicate<TInput>, Predicate<TInput>)
DataFlowSource<TInput>.LinkTo<TConvert>(IDataFlowDestination<TInput>)
DataFlowSource<TInput>.LinkTo<TConvert>(IDataFlowDestination<TInput>, Predicate<TInput>)
DataFlowSource<TInput>.LinkTo<TConvert>(IDataFlowDestination<TInput>, Predicate<TInput>, Predicate<TInput>)
DataFlowComponent.MaxBufferSize
DataFlowComponent.Predecessors
DataFlowComponent.Successors
DataFlowComponent.Completion
DataFlowComponent.BufferCancellationSource
DataFlowComponent.BufferCancellationToken
DataFlowComponent.WasComponentInitialized
DataFlowComponent.IsReadyForProcessing
DataFlowComponent.SetParentComponent(DataFlowComponent)
DataFlowComponent.InternalLinkTo<T>(IDataFlowDestination, object, object)
DataFlowComponent.AddComplementVoidLinkToOtherTargets(Func<Type, IDataFlowDestination>, Delegate)
DataFlowComponent.ResetComponent()
DataFlowComponent.InitBufferObjects(CancellationToken?)
DataFlowComponent.InitCheckedParameter()
DataFlowComponent.OnInitialization
DataFlowComponent.OnCompletion
DataFlowComponent.OnException
DataFlowComponent.CompleteOrFaultBufferOnPredecessorCompletion(Task)
DataFlowComponent.Exception
DataFlowComponent.ErrorSource
DataFlowComponent.CancelBufferAndThrowOperationCanceledException()
DataFlowComponent.InternalLinkErrorTo(IDataFlowDestination<ETLBoxError>)
DataFlowComponent.ThrowOrRedirectError(Exception, string, string)
DataFlowComponent.ThrowErrorAndFaultNetwork(Exception, string)
DataFlowComponent.CancelComponent()
DataFlowComponent.ErrorCount
DataFlowComponent.OnProgress
DataFlowComponent.ExecutionStartTime
DataFlowComponent.ExecutionEndTime
DataFlowComponent.Tag
DataFlowComponent.LoggingStartOnce()
DataFlowComponent.LoggingFinishOnce()
DataFlowComponent.LogProgressBatch_AfterThrowOrRedirectError(int)
DataFlowComponent.LogProgress_AfterThrowOrRedirectError()
LoggableTask.TaskType
LoggableTask.TaskName
LoggableTask.DisableLogging
LoggableTask.LogThreshold
LoggableTask.TaskHash
LoggableTask.LogInstance
LoggableTask.ParentTask
LoggableTask.LogTrace(string, params object[])
LoggableTask.LogDebug(string, params object[])
LoggableTask.LogInfo(string, params object[])
LoggableTask.LogInfoOrDebug(string, params object[])
LoggableTask.LogWarn(string, params object[])
LoggableTask.LogError(string, params object[])
LoggableTask.LogFatal(string, params object[])
object.Equals(object)
object.Equals(object, object)
object.GetHashCode()
object.GetType()
object.MemberwiseClone()
object.ReferenceEquals(object, object)
object.ToString()
Namespace: ETLBox.DataFlow
Assembly: ETLBox.dll
Syntax
{{< /rawhtml >}} ```C# public class LookupTransformation : DataFlowTransformation, IDataFlowTransformation, IDataFlowSource, IDataFlowSource, IDataFlowDestination, IDataFlowDestination, IDataFlowComponent, ILoggableTask ``` {{< rawhtml >}}
Type Parameters
Name Description
TInput

Type of ingoing and outgoing data.
TSource

Type of data used in the lookup source.

Constructors

LookupTransformation()

Declaration
{{< /rawhtml >}} ```C# public LookupTransformation() ``` {{< rawhtml >}}

LookupTransformation(IDataFlowExecutableSource<TSource>)

Declaration
{{< /rawhtml >}} ```C# public LookupTransformation(IDataFlowExecutableSource source) ``` {{< rawhtml >}}
Parameters
Type Name Description
IDataFlowExecutableSource<TSource> source

Sets the Source of the lookup.

LookupTransformation(IDataFlowExecutableSource<TSource>, Func<TInput, CachedData<TSource>, TInput>)

Declaration
{{< /rawhtml >}} ```C# public LookupTransformation(IDataFlowExecutableSource source, Func, TInput> applyRetrievedCacheToInput) ``` {{< rawhtml >}}
Parameters
Type Name Description
IDataFlowExecutableSource<TSource> source

Sets the Source of the lookup.
Func<TInput, CachedData<TSource>, TInput> applyRetrievedCacheToInput

Sets the ApplyRetrievedCacheToInput function.

Properties

ApplyRetrievedCacheForMultipleOutputs

This function describes how the ingoing data can be enriched with the already pre-read data from the Source, but can return multiple rows for every input row. This is necessary if PermitMultipleEntriesPerKey is set to true, which allows the cache lookup to load multiple rows for every key.

Declaration
{{< /rawhtml >}} ```C# public Func, TInput[]> ApplyRetrievedCacheForMultipleOutputs { get; set; } ``` {{< rawhtml >}}
Property Value
Type Description
Func<TInput, CachedData<TSource>, TInput[]>

ApplyRetrievedCacheToInput

This function describes how the ingoing data can be enriched with the already pre-read data from the Source.

Declaration
{{< /rawhtml >}} ```C# public Func, TInput> ApplyRetrievedCacheToInput { get; set; } ``` {{< rawhtml >}}
Property Value
Type Description
Func<TInput, CachedData<TSource>, TInput>

CacheMode

The cache mode used for loading data into the lookup cache. By default, all data is loaded into memory from the lookup source when the first records arrives at the lookup.

Declaration
{{< /rawhtml >}} ```C# public CacheMode CacheMode { get; set; } ``` {{< rawhtml >}}
Property Value
Type Description
CacheMode

CachedBatchTransformation

Declaration
{{< /rawhtml >}} ```C# public CachedBatchTransformation CachedBatchTransformation { get; } ``` {{< rawhtml >}}
Property Value
Type Description
CachedBatchTransformation<TInput, TInput, TSource>

InputKeySelector

This function describe how the key is generated for an ingoing row. If the result of this function match with SourceKeySelector, both records can be identified as a match.

Declaration
{{< /rawhtml >}} ```C# public Func InputKeySelector { get; set; } ``` {{< rawhtml >}}
Property Value
Type Description
Func<TInput, object>

MatchColumns

This collection will be used to define the matching columns - will also work with ExpandoObject.

Declaration
{{< /rawhtml >}} ```C# public ICollection MatchColumns { get; set; } ``` {{< rawhtml >}}
Property Value
Type Description
ICollection<MatchColumn>

OnNoMatchFound

Specifies the function to be invoked when a lookup operation fails to find a matching key in the source. Will only be invoked if ApplyRetrievedCacheToInput or ApplyRetrievedCacheForMultipleOutputs is not set! This action contains the current input row and the lookup key for which no match was found.

Declaration
{{< /rawhtml >}} ```C# public Action OnNoMatchFound { get; set; } ``` {{< rawhtml >}}
Property Value
Type Description
Action<TInput, object>

PartialCacheSettings

Only need if you set the CacheMode to PartialDbCache. This will only work if you use a DbSource as lookup source.

Declaration
{{< /rawhtml >}} ```C# public LookupTransformation.PartialDbCacheSettings PartialCacheSettings { get; set; } ``` {{< rawhtml >}}
Property Value
Type Description
LookupTransformation<TInput, TSource>.PartialDbCacheSettings

PermitMultipleEntriesPerKey

If set to true, multiple entries can be loaded into the cache for every key. Otherwise, only the first occurence will be loaded into the cache.

Declaration
{{< /rawhtml >}} ```C# public bool PermitMultipleEntriesPerKey { get; set; } ``` {{< rawhtml >}}
Property Value
Type Description
bool

ProgressCount

Declaration
{{< /rawhtml >}} ```C# public int ProgressCount { get; } ``` {{< rawhtml >}}
Property Value
Type Description
int

RemoveUnmatchedRows

If no match is found in the lookup source, the row will be removed from the output. Will only be invoked if ApplyRetrievedCacheToInput or ApplyRetrievedCacheForMultipleOutputs is not set!

Declaration
{{< /rawhtml >}} ```C# public bool RemoveUnmatchedRows { get; set; } ``` {{< rawhtml >}}
Property Value
Type Description
bool

RetrieveColumns

This collection will be used to define the retrieve columns - will also work with ExpandoObject.

Declaration
{{< /rawhtml >}} ```C# public ICollection RetrieveColumns { get; set; } ``` {{< rawhtml >}}
Property Value
Type Description
ICollection<RetrieveColumn>

ShouldSkipRow

Specifies if a row should be skipped based on the input row. If this function returns true, the row will be skipped and not be processed by the lookup. Will only be invoked if ApplyRetrievedCacheToInput or ApplyRetrievedCacheForMultipleOutputs is not set!

Declaration
{{< /rawhtml >}} ```C# public Predicate ShouldSkipRow { get; set; } ``` {{< rawhtml >}}
Property Value
Type Description
Predicate<TInput>

Source

The source component from which the lookup data is retrieved. E.g. a DbSource or a MemorySource.

Declaration
{{< /rawhtml >}} ```C# public IDataFlowExecutableSource Source { get; set; } ``` {{< rawhtml >}}
Property Value
Type Description
IDataFlowExecutableSource<TSource>

SourceBlock

SourceBlock from the underlying TPL.Dataflow which is used as output buffer for the component.

Declaration
{{< /rawhtml >}} ```C# public override ISourceBlock SourceBlock { get; } ``` {{< rawhtml >}}
Property Value
Type Description
ISourceBlock<TInput>
Overrides

SourceKeySelector

This function describe how the key is generated for a row from the lookup source. This key is used to store the data in the lookup dictionary.

Declaration
{{< /rawhtml >}} ```C# public Func SourceKeySelector { get; set; } ``` {{< rawhtml >}}
Property Value
Type Description
Func<TSource, object>

TargetBlock

TargetBlock from the underlying TPL.Dataflow which is used as input buffer for the component.

Declaration
{{< /rawhtml >}} ```C# public override ITargetBlock TargetBlock { get; } ``` {{< rawhtml >}}
Property Value
Type Description
ITargetBlock<TInput>
Overrides

UseExactNumericTypes

If set to true, data types are ignored when comparing data to identify matching records. E.g. property that stores the integer value 7 would match with a string "7".

Declaration
{{< /rawhtml >}} ```C# public bool UseExactNumericTypes { get; set; } ``` {{< rawhtml >}}
Property Value
Type Description
bool

Methods

CheckParameter()

Declaration
{{< /rawhtml >}} ```C# protected override void CheckParameter() ``` {{< rawhtml >}}
Overrides

CleanUpOnFaulted(Exception)

Declaration
{{< /rawhtml >}} ```C# protected override void CleanUpOnFaulted(Exception e) ``` {{< rawhtml >}}
Parameters
Type Name Description
Exception e
Overrides

CleanUpOnSuccess()

Declaration
{{< /rawhtml >}} ```C# protected override void CleanUpOnSuccess() ``` {{< rawhtml >}}
Overrides

CompleteBuffer()

Declaration
{{< /rawhtml >}} ```C# protected override void CompleteBuffer() ``` {{< rawhtml >}}
Overrides

FaultBuffer(Exception)

Declaration
{{< /rawhtml >}} ```C# protected override void FaultBuffer(Exception e) ``` {{< rawhtml >}}
Parameters
Type Name Description
Exception e
Overrides

InitComponent()

Declaration
{{< /rawhtml >}} ```C# protected override void InitComponent() ``` {{< rawhtml >}}
Overrides

LinkErrorTo(IDataFlowDestination<ETLBoxError>)

If an error occurs in the component, by default the component will throw an exception and stop execution. If you use the error linking, any erroneous records will be caught and redirected.

Declaration
{{< /rawhtml >}} ```C# public override IDataFlowSource LinkErrorTo(IDataFlowDestination target) ``` {{< rawhtml >}}
Parameters
Type Name Description
IDataFlowDestination<ETLBoxError> target

The target for erroneous rows.
Returns
Type Description
IDataFlowSource<ETLBoxError>

The linked component.
Overrides

PrepareParameterForCheck()

Declaration
{{< /rawhtml >}} ```C# protected override void PrepareParameterForCheck() ``` {{< rawhtml >}}
Overrides

Reset()

Declaration
{{< /rawhtml >}} ```C# protected override void Reset() ``` {{< rawhtml >}}
Overrides

Implements

{{< /rawhtml >}}