Class EmbeddingBatchTransformation<TInput> Creates embeddings for a batch of rows with one generator call and writes each vector
back into the corresponding row object. The default batch size is 64.
Inheritance EmbeddingBatchTransformation<TInput>
Namespace : ETLBox.AIAssembly : ETLBox.AI.dll public class EmbeddingBatchTransformation < TInput > : DataFlowTransformation < TInput , TInput >, IDataFlowTransformation < TInput , TInput >, IDataFlowSource < TInput >, IDataFlowSource , IDataFlowDestination < TInput >, IDataFlowDestination , IDataFlowComponent , ILoggableTask Type Parameters Name Description TInput Type of ingoing and outgoing data.
Constructors
Declaration public EmbeddingBatchTransformation ()
Declaration public EmbeddingBatchTransformation ( IEmbeddingGenerator < string , Embedding < float >> embeddingGenerator ) Parameters Type Name Description IEmbeddingGenerator <string , Embedding <float >>embeddingGenerator
Declaration public EmbeddingBatchTransformation ( IEmbeddingGenerator < string , Embedding < float >> embeddingGenerator , int batchSize ) Parameters Type Name Description IEmbeddingGenerator <string , Embedding <float >>embeddingGenerator int batchSize
Properties The number of rows sent to the embedding generator in one call. Default is 64.
Declaration public int BatchSize { get ; set ; } Property Value Optional options passed to the embedding generator, e.g. model id or dimensions.
Declaration public EmbeddingGenerationOptions EmbeddingGenerationOptions { get ; set ; } Property Value Type Description EmbeddingGenerationOptions
Embedding generator created by the caller.
Declaration public IEmbeddingGenerator < string , Embedding < float >> EmbeddingGenerator { get ; set ; } Property Value Type Description IEmbeddingGenerator <string , Embedding <float >>
If true, the generator must return exactly one vector per input row, otherwise the batch fails.
If false (default), rows are only emitted for the returned vectors.
Declaration public bool RequireSameItemCount { get ; set ; } Property Value SourceBlock from the underlying TPL.Dataflow which is used as output buffer for the component.
Declaration public override ISourceBlock < TInput > SourceBlock { get ; } Property Value Overrides TargetBlock from the underlying TPL.Dataflow which is used as input buffer for the component.
Declaration public override ITargetBlock < TInput > TargetBlock { get ; } Property Value Overrides TextPropertyName# Only for ExpandoObject : property that contains the text to embed. Default is "Text".
Declaration public string TextPropertyName { get ; set ; } Property Value TextSelector# Selects the text that is sent to the embedding generator.
Declaration public Func < TInput , string > TextSelector { get ; set ; } Property Value Only for ExpandoObject : property that receives the vector. Default is "Embedding".
Declaration public string VectorPropertyName { get ; set ; } Property Value Writes the returned vector into the row.
Declaration public Action < TInput , ReadOnlyMemory < float >> VectorSetter { get ; set ; } Property Value Methods
Declaration protected override void CheckParameter () Overrides
Declaration protected override void CleanUpOnFaulted ( Exception e ) Parameters Overrides
Declaration protected override void CleanUpOnSuccess () Overrides
Declaration protected override void CompleteBuffer () Overrides
Declaration protected override void FaultBuffer ( Exception e ) Parameters Overrides
Declaration protected override void InitComponent () Overrides
Declaration protected override void PrepareParameterForCheck () Overrides
Declaration protected override void Reset () Overrides Implements#
ETLBox