Class ParquetSource<TOutput>
Reads data from a parquet source. While reading the data from the file, data is also asynchronously posted into the targets.
Inheritance
object
DataFlowSource<TOutput>
DataFlowExecutableSource<TOutput>
DataFlowStreamSource<TOutput>
ParquetSource<TOutput>
Inherited Members
object.Equals(object)
object.Equals(object, object)
object.GetHashCode()
object.GetType()
object.MemberwiseClone()
object.ReferenceEquals(object, object)
object.ToString()
Namespace: ETLBox.Parquet
Assembly: ETLBox.Parquet.dll
Syntax
public class ParquetSource<TOutput> : DataFlowStreamSource<TOutput>, ILoggableTask, IDataFlowLogging, IDataFlowStreamSource<TOutput>, IDataFlowExecutableSource<TOutput>, IDataFlowSource<TOutput>, IDataFlowStreamSource, IDataFlowExecutableSource, IDataFlowSource, IDataFlowComponent
Type Parameters
| Name | Description |
|---|---|
| TOutput | Type of outgoing data |
Constructors
ParquetSource()
Declaration
public ParquetSource()
ParquetSource(string, ResourceType)
Declaration
public ParquetSource(string uri, ResourceType resourceType)
Parameters
| Type | Name | Description |
|---|---|---|
| string | uri | The source parquet file name or uri |
| ResourceType | resourceType | Specifies if data is loaded from a file, a web endpoint or other storage types (e.g. Azure Blob Storage) |
ParquetSource(string)
Declaration
public ParquetSource(string uri)
Parameters
| Type | Name | Description |
|---|---|---|
| string | uri | The source parquet file name or uri |
Fields
SupportedDataTypes
Currently the ParquetDestination only supports 'basics' data types. Access this property to get all Clr types that are supported.
Declaration
public Type[] SupportedDataTypes
Field Value
| Type | Description |
|---|---|
| System.Type[] |
Properties
ParquetColumns
List of mappings between columns names in parquet file and property name in object.
Declaration
public ICollection<ParquetColumn> ParquetColumns { get; set; }
Property Value
| Type | Description |
|---|---|
| System.Collections.Generic.ICollection<T><ParquetColumn> |
ParquetOptions
Options used when creating the ParquetReader
Declaration
public ParquetOptions ParquetOptions { get; set; }
Property Value
| Type | Description |
|---|---|
| ParquetOptions |
TaskName
A name to identify the task or component. Every component or task comes with a default name that can be overwritten.
Declaration
public override string TaskName { get; set; }
Property Value
| Type | Description |
|---|---|
| string |
Overrides
Methods
CheckParameter()
Declaration
protected override void CheckParameter()
Overrides
ETLBox.DataFlow.DataFlowStreamSource<TOutput>.CheckParameter()
CloseReader()
Declaration
protected override void CloseReader()
Overrides
ETLBox.DataFlow.DataFlowStreamSource<TOutput>.CloseReader()
FindPropNameMapping(string)
Declaration
public string FindPropNameMapping(string columnName)
Parameters
| Type | Name | Description |
|---|---|---|
| string | columnName |
Returns
| Type | Description |
|---|---|
| string |
InitReader()
Declaration
protected override void InitReader()
Overrides
ETLBox.DataFlow.DataFlowStreamSource<TOutput>.InitReader()
PrepareParameter()
Declaration
protected override void PrepareParameter()
Overrides
ETLBox.DataFlow.DataFlowStreamSource<TOutput>.PrepareParameter()
ReadAllRecords()
Declaration
protected override void ReadAllRecords()
Overrides
ETLBox.DataFlow.DataFlowStreamSource<TOutput>.ReadAllRecords()