Class DataFlowStreamDestination<TInput>
The base implementation for a destination that allows writing of data in a stream.
Inheritance
object
DataFlowStreamDestination<TInput>
Inherited Members
object.Equals(object)
object.Equals(object, object)
object.GetHashCode()
object.GetType()
object.MemberwiseClone()
object.ReferenceEquals(object, object)
object.ToString()
Namespace: ETLBox.DataFlow
Assembly: ETLBox.dll
Syntax
public abstract class DataFlowStreamDestination<TInput> : DataFlowDestination<TInput>, ILoggableTask, IDataFlowLogging, IDataFlowStreamDestination<TInput>, IDataFlowDestination<TInput>, IDataFlowDestination, IDataFlowComponent, IDataFlowStreamDestination
Type Parameters
| Name | Description |
|---|
| TInput | Type of ingoing data |
Properties
AzureBlobStorage
Use this to pass your AzureBlobStorage connection information.
There are different methods to connect with Azure Blob Storage:
Option 1: provide an existing BlockBlobClient
Option 2: create BlockBlobClient based on the Uri and optional credential tokens - the Uri needs to be the blob url
Option 3: create BlobkBlobClient with provided ConnectionString and ContainerName, the Uri is then the name of the blob
Declaration
public AzureBlobStorageConfiguration AzureBlobStorage { get; set; }
Property Value
CreateStreamWriter
By default, stream are created internally based on Uri & ResourceType.
If you already have an existing stream, use this function to returns a new StreamWriter based on this stream.
Declaration
public Func<string, StreamWriter> CreateStreamWriter { get; set; }
Property Value
| Type | Description |
|---|
| System.Func<T, TResult><string, System.IO.StreamWriter> | |
Encoding
Encoding used to write data into the source file or web request.
Declaration
public Encoding Encoding { get; set; }
Property Value
| Type | Description |
|---|
| System.Text.Encoding | |
GetNextUri
This function returns the next url that is used for writing data. It will used until HasNextUri returns false.
The incoming StreamMetaData holds information about the current progress and currently processed data row.
This property can be used if you want to write into multiple files or send data into different web services.
Declaration
public Func<StreamMetaData, TInput, string> GetNextUri { get; set; }
Property Value
| Type | Description |
|---|
| System.Func<T1, T2, TResult><StreamMetaData, TInput, string> | |
HasNextUri
This func determines if the next data record should be written into another location.
StreamMetaData has information about the current progress and currently processed data row.
Declaration
public Func<StreamMetaData, TInput, bool> HasNextUri { get; set; }
Property Value
| Type | Description |
|---|
| System.Func<T1, T2, TResult><StreamMetaData, TInput, bool> | |
HttpClient
The System.Net.HttpClient used to connect with the destination (only needed when the ResourceType is Http.
Declaration
public HttpClient HttpClient { get; set; }
Property Value
| Type | Description |
|---|
| System.Net.Http.HttpClient | |
HttpContentType
The content type used when sending the http request content.
Declaration
public virtual string HttpContentType { get; set; }
Property Value
HttpRequestMessage
The System.Net.Http.HttpRequestMessage used for the request in the HttpClient. Use this property
to add additional header data type or to change the http method (e.g. for POST requests).
Declaration
public HttpRequestMessage HttpRequestMessage { get; set; }
Property Value
| Type | Description |
|---|
| System.Net.Http.HttpRequestMessage | |
HttpResponseMessage
The System.Net.HttpResponseMessage returned from the current request. The response message is avaiable
after all data rows were sent to the destination.
Declaration
public HttpResponseMessage HttpResponseMessage { get; }
Property Value
| Type | Description |
|---|
| System.Net.Http.HttpResponseMessage | |
Declaration
protected StreamMetaData NewMetaDataObject { get; }
Property Value
ResourceType
Specifies the resource type.
Use ResourceType.File if you want to write into a file (default)
Use ResourceType.Http to write into a web endpoint.
Declaration
public ResourceType ResourceType { get; set; }
Property Value
StreamWriter
The stream writer used for the current request.
Declaration
public StreamWriter StreamWriter { get; protected set; }
Property Value
| Type | Description |
|---|
| System.IO.StreamWriter | |
Uri
Declaration
public string Uri { get; set; }
Property Value
WriteUri
The Uri used for the current write operation- this will only be different to the Uri
if you use the GetNextUri and HasNextUri functions.
Declaration
public string WriteUri { get; protected set; }
Property Value
Methods
CheckParameter()
Declaration
protected override void CheckParameter()
Overrides
CleanUpOnFaulted(Exception)
Declaration
protected override void CleanUpOnFaulted(Exception e)
Parameters
| Type | Name | Description |
|---|
| System.Exception | e | |
Overrides
CleanUpOnSuccess()
Declaration
protected override void CleanUpOnSuccess()
Overrides
CloseStream()
Declaration
protected abstract void CloseStream()
InitComponent()
Declaration
protected override void InitComponent()
Overrides
ETLBox.DataFlow.DataFlowDestination<TInput>.InitComponent()
InitStream()
Declaration
protected abstract void InitStream()
PrepareParameter()
Declaration
protected override void PrepareParameter()
Overrides
PrepareStreamForBatchProcessing(TInput[])
Declaration
protected void PrepareStreamForBatchProcessing(TInput[] data)
Parameters
| Type | Name | Description |
|---|
| TInput[] | data | |
WriteData(TInput)
Declaration
protected void WriteData(TInput data)
Parameters
| Type | Name | Description |
|---|
| TInput | data | |
WriteIntoStream(TInput)
Declaration
protected abstract void WriteIntoStream(TInput data)
Parameters
| Type | Name | Description |
|---|
| TInput | data | |
Implements