ETLBox ETLBox

Class DataFlowStreamSource<TOutput>

Inheritance
System.Object
LoggableTask
DataFlowComponent
DataFlowSource<TOutput>
DataFlowExecutableSource<TOutput>
DataFlowStreamSource<TOutput>
CsvSource<TOutput>
ExcelSource<TOutput>
JsonSource<TOutput>
TextSource<TOutput>
XmlSource<TOutput>
Inherited Members
System.Object.Equals(System.Object)
System.Object.Equals(System.Object, System.Object)
System.Object.GetHashCode()
System.Object.GetType()
System.Object.MemberwiseClone()
System.Object.ReferenceEquals(System.Object, System.Object)
System.Object.ToString()
Namespace: ETLBox.DataFlow
Assembly: ETLBox.dll
Syntax
    public abstract class DataFlowStreamSource<TOutput> : DataFlowExecutableSource<TOutput>, ILoggableTask, IDataFlowLogging, IDataFlowStreamSource<TOutput>, IDataFlowExecutableSource<TOutput>, IDataFlowSource<TOutput>, IDataFlowSource, IDataFlowComponent, IDataFlowExecutableSource, IDataFlowStreamSource
Type Parameters
NameDescription
TOutput

Properties

CurrentRequestUri

Declaration
    protected string CurrentRequestUri { get; set; }
Property Value
TypeDescription
System.String

Encoding

Encoding used to read data from the source file or web request.

Declaration
    public Encoding Encoding { get; set; }
Property Value
TypeDescription
System.Text.Encoding

GetNextUri

This func returns the next url that is used for reading data. It will be called until HasNextUri returns false. The incoming StreamMetaData holds information about the current progress and other meta data from the response, like unparsed json data that contains references to the next page of the response. This property can be used if you want to read multiple files or if you want to paginate through web responses.

Declaration
    public Func<StreamMetaData, string> GetNextUri { get; set; }
Property Value
TypeDescription
System.Func<StreamMetaData, System.String>

HasNextUri

This func determines if another request is started to read additional data from the next uri. StreamMetaData has information about the current progress and other meta data from the response.

Declaration
    public Func<StreamMetaData, bool> HasNextUri { get; set; }
Property Value
TypeDescription
System.Func<StreamMetaData, System.Boolean>

HttpClient

The System.Net.Http.HttpClient uses for the request. Use this client if you want to add or change the http request data, e.g. you can add your authorization information here.

Declaration
    public HttpClient HttpClient { get; set; }
Property Value
TypeDescription
System.Net.Http.HttpClient

HttpRequestMessage

The System.Net.Http.HttpRequestMessage use for the request from the HttpClient. Add your request message here, e.g. your POST body.

Declaration
    public HttpRequestMessage HttpRequestMessage { get; set; }
Property Value
TypeDescription
System.Net.Http.HttpRequestMessage

ResourceType

Specifies the resource type. By default requests are made with HttpClient. Specify ResourceType.File if you want to read from a json file.

Declaration
    public ResourceType ResourceType { get; set; }
Property Value
TypeDescription
ResourceType

SkipRows

Number of rows to skip before starting reading the header and csv data

Declaration
    public int SkipRows { get; set; }
Property Value
TypeDescription
System.Int32

StreamReader

Declaration
    protected StreamReader StreamReader { get; set; }
Property Value
TypeDescription
System.IO.StreamReader

UnparsedData

Declaration
    public string UnparsedData { get; }
Property Value
TypeDescription
System.String

UnparsedDataBuilder

Declaration
    protected StringBuilder UnparsedDataBuilder { get; set; }
Property Value
TypeDescription
System.Text.StringBuilder

Uri

The Url of the webservice (e.g. https://test.com/foo) or the file name (relative or absolute).

Declaration
    public string Uri { get; set; }
Property Value
TypeDescription
System.String

Methods

CheckParameter()

Declaration
    protected override void CheckParameter()
Overrides

CleanUpOnFaulted(Exception)

Declaration
    protected override void CleanUpOnFaulted(Exception e)
Parameters
TypeNameDescription
System.Exceptione
Overrides

CleanUpOnSuccess()

Declaration
    protected override void CleanUpOnSuccess()
Overrides

CloseReader()

Declaration
    protected abstract void CloseReader()

InitReader()

Declaration
    protected abstract void InitReader()

OnExecutionDoAsyncWork()

Declaration
    protected override void OnExecutionDoAsyncWork()
Overrides
ETLBox.DataFlow.DataFlowExecutableSource<TOutput>.OnExecutionDoAsyncWork()

OnExecutionDoSynchronousWork()

Declaration
    protected override void OnExecutionDoSynchronousWork()
Overrides
ETLBox.DataFlow.DataFlowExecutableSource<TOutput>.OnExecutionDoSynchronousWork()

ReadAllRecords()

Declaration
    protected abstract void ReadAllRecords()

Implements