Class EmbeddingTransformation<TInput>

Creates an embedding for each row and writes the vector back into the same row object. The text is read via TextSelector (or the EmbeddingText attribute), the vector is written via VectorSetter (or the EmbeddingVector attribute).

Inheritance
DataFlowTransformation<TInput, TInput>
EmbeddingTransformation<TInput>
Inherited Members
Namespace: ETLBox.AI
Assembly: ETLBox.AI.dll
Syntax
    public class EmbeddingTransformation<TInput> : DataFlowTransformation<TInput, TInput>, IDataFlowTransformation<TInput, TInput>, IDataFlowSource<TInput>, IDataFlowSource, IDataFlowDestination<TInput>, IDataFlowDestination, IDataFlowComponent, ILoggableTask
Type Parameters
NameDescription
TInput

Type of ingoing and outgoing data.

Constructors

EmbeddingTransformation()

Declaration
    public EmbeddingTransformation()

EmbeddingTransformation(IEmbeddingGenerator<string, Embedding<float>>)

Declaration
    public EmbeddingTransformation(IEmbeddingGenerator<string, Embedding<float>> embeddingGenerator)
Parameters
TypeNameDescription
IEmbeddingGenerator<string, Embedding<float>>embeddingGenerator

Properties

EmbeddingGenerationOptions

Optional options passed to the embedding generator, e.g. model id or dimensions.

Declaration
    public EmbeddingGenerationOptions EmbeddingGenerationOptions { get; set; }
Property Value
TypeDescription
EmbeddingGenerationOptions

EmbeddingGenerator

Embedding generator created by the caller.

Declaration
    public IEmbeddingGenerator<string, Embedding<float>> EmbeddingGenerator { get; set; }
Property Value
TypeDescription
IEmbeddingGenerator<string, Embedding<float>>

SourceBlock

SourceBlock from the underlying TPL.Dataflow which is used as output buffer for the component.

Declaration
    public override ISourceBlock<TInput> SourceBlock { get; }
Property Value
TypeDescription
ISourceBlock<TInput>
Overrides

TargetBlock

TargetBlock from the underlying TPL.Dataflow which is used as input buffer for the component.

Declaration
    public override ITargetBlock<TInput> TargetBlock { get; }
Property Value
TypeDescription
ITargetBlock<TInput>
Overrides

TextPropertyName

Only for ExpandoObject: property that contains the text to embed. Default is "Text".

Declaration
    public string TextPropertyName { get; set; }
Property Value
TypeDescription
string

TextSelector

Selects the text that is sent to the embedding generator.

Declaration
    public Func<TInput, string> TextSelector { get; set; }
Property Value
TypeDescription
Func<TInput, string>

VectorPropertyName

Only for ExpandoObject: property that receives the vector. Default is "Embedding".

Declaration
    public string VectorPropertyName { get; set; }
Property Value
TypeDescription
string

VectorSetter

Writes the returned vector into the row.

Declaration
    public Action<TInput, ReadOnlyMemory<float>> VectorSetter { get; set; }
Property Value
TypeDescription
Action<TInput, ReadOnlyMemory<float>>

Methods

CheckParameter()

Declaration
    protected override void CheckParameter()
Overrides

CleanUpOnFaulted(Exception)

Declaration
    protected override void CleanUpOnFaulted(Exception e)
Parameters
TypeNameDescription
Exceptione
Overrides

CleanUpOnSuccess()

Declaration
    protected override void CleanUpOnSuccess()
Overrides

InitComponent()

Declaration
    protected override void InitComponent()
Overrides

PrepareParameterForCheck()

Declaration
    protected override void PrepareParameterForCheck()
Overrides

Reset()

Declaration
    protected override void Reset()
Overrides

Implements