Class DataFlowStreamDestination<TInput>

The base implementation for a destination that allows writing of data in a stream.

Inherited Members
Namespace: ETLBox.DataFlow
Assembly: ETLBox.dll
Syntax
    public abstract class DataFlowStreamDestination<TInput> : DataFlowDestination<TInput>, IDataFlowLogging, IDataFlowStreamDestination<TInput>, IDataFlowDestination<TInput>, IDataFlowStreamDestination, IDataFlowDestination, IDataFlowComponent, ILoggableTask
Type Parameters
NameDescription
TInput

Type of ingoing data

Properties

AzureBlobStorage

Use this to pass your AzureBlobStorage connection information. There are different methods to connect with Azure Blob Storage: Option 1: provide an existing BlockBlobClient Option 2: create BlockBlobClient based on the Uri and optional credential tokens - the Uri needs to be the blob url Option 3: create BlobkBlobClient with provided ConnectionString and ContainerName, the Uri is then the name of the blob

Declaration
    public AzureBlobStorageConfiguration AzureBlobStorage { get; set; }
Property Value
TypeDescription
AzureBlobStorageConfiguration

CreateStreamWriter

By default, stream are created internally based on Uri & ResourceType. If you already have an existing stream, use this function to returns a new StreamWriter based on this stream.

Declaration
    public Func<string, StreamWriter> CreateStreamWriter { get; set; }
Property Value
TypeDescription
Func<string, StreamWriter>

Encoding

Encoding used to write data into the source file or web request.

Declaration
    public Encoding Encoding { get; set; }
Property Value
TypeDescription
Encoding

GetNextUri

This function returns the next url that is used for writing data. It will used until HasNextUri returns false. The incoming StreamMetaData holds information about the current progress and currently processed data row. This property can be used if you want to write into multiple files or send data into different web services.

Declaration
    public Func<StreamMetaData, TInput, string> GetNextUri { get; set; }
Property Value
TypeDescription
Func<StreamMetaData, TInput, string>

HasNextUri

This func determines if the next data record should be written into another location. StreamMetaData has information about the current progress and currently processed data row.

Declaration
    public Func<StreamMetaData, TInput, bool> HasNextUri { get; set; }
Property Value
TypeDescription
Func<StreamMetaData, TInput, bool>

HttpClient

The System.Net.HttpClient used to connect with the destination (only needed when the ResourceType is Http.

Declaration
    public HttpClient HttpClient { get; set; }
Property Value
TypeDescription
HttpClient

HttpContentType

The content type used when sending the http request content.

Declaration
    public virtual string HttpContentType { get; set; }
Property Value
TypeDescription
string

HttpRequestMessage

The System.Net.Http.HttpRequestMessage used for the request in the HttpClient. Use this property to add additional header data type or to change the http method (e.g. for POST requests).

Declaration
    public HttpRequestMessage HttpRequestMessage { get; set; }
Property Value
TypeDescription
HttpRequestMessage

HttpResponseMessage

The System.Net.HttpResponseMessage returned from the current request. The response message is avaiable after all data rows were sent to the destination.

Declaration
    public HttpResponseMessage HttpResponseMessage { get; }
Property Value
TypeDescription
HttpResponseMessage

NewMetaDataObject

Declaration
    protected StreamMetaData NewMetaDataObject { get; }
Property Value
TypeDescription
StreamMetaData

ResourceType

Specifies the resource type. Use ResourceType.File if you want to write into a file (default) Use ResourceType.Http to write into a web endpoint.

Declaration
    public ResourceType ResourceType { get; set; }
Property Value
TypeDescription
ResourceType

StreamWriter

The stream writer used for the current request.

Declaration
    public StreamWriter StreamWriter { get; protected set; }
Property Value
TypeDescription
StreamWriter

Uri

The Url of the webservice (e.g. https://test.com/foo) or the file name (relative or absolute)

Declaration
    public string Uri { get; set; }
Property Value
TypeDescription
string

WriteUri

The Uri used for the current write operation- this will only be different to the Uri if you use the GetNextUri and HasNextUri functions.

Declaration
    public string WriteUri { get; protected set; }
Property Value
TypeDescription
string

Methods

CheckParameter()

Declaration
    protected override void CheckParameter()
Overrides

CleanUpOnFaulted(Exception)

Declaration
    protected override void CleanUpOnFaulted(Exception e)
Parameters
TypeNameDescription
Exceptione
Overrides

CleanUpOnSuccess()

Declaration
    protected override void CleanUpOnSuccess()
Overrides

CloseStream()

Declaration
    protected abstract void CloseStream()

InitComponent()

Declaration
    protected override void InitComponent()
Overrides

InitStream()

Declaration
    protected abstract void InitStream()

PrepareParameterForCheck()

Declaration
    protected override void PrepareParameterForCheck()
Overrides

PrepareStreamForBatchProcessing(TInput[])

Declaration
    protected void PrepareStreamForBatchProcessing(TInput[] data)
Parameters
TypeNameDescription
TInput[]data

WriteData(TInput)

Declaration
    protected void WriteData(TInput data)
Parameters
TypeNameDescription
TInputdata

WriteIntoStream(TInput)

Declaration
    protected abstract void WriteIntoStream(TInput data)
Parameters
TypeNameDescription
TInputdata

Implements