Title here
Summary here
Reads data from a parquet source. While reading the data from the file, data is also asynchronously posted into the targets.
public class ParquetSource<TOutput> : DataFlowStreamSource<TOutput>, IDataFlowLogging, IDataFlowStreamSource<TOutput>, IDataFlowExecutableSource<TOutput>, IDataFlowSource<TOutput>, IDataFlowStreamSource, IDataFlowExecutableSource, IDataFlowSource, IDataFlowComponent, ILoggableTask
Name | Description |
---|---|
TOutput | Type of outgoing data |
public ParquetSource()
public ParquetSource(string uri)
Type | Name | Description |
---|---|---|
string | uri | The source parquet file name or uri |
public ParquetSource(string uri, ResourceType resourceType)
Type | Name | Description |
---|---|---|
string | uri | The source parquet file name or uri |
ResourceType | resourceType | Specifies if data is loaded from a file, a web endpoint or other storage types (e.g. Azure Blob Storage) |
Currently the ParquetDestination only supports 'basic' data types. Access this property to get all Clr types that are supported.
public Type[] SupportedDataTypes
Type | Description |
---|---|
Type[] |
List of mappings between columns names in parquet file and property name in object.
public ICollection<ParquetColumn> ParquetColumns { get; set; }
Type | Description |
---|---|
ICollection<ParquetColumn> |
Options used when creating the ParquetReader
public ParquetOptions ParquetOptions { get; set; }
Type | Description |
---|---|
ParquetOptions |
protected override void CheckParameter()
protected override void CloseReader()
public string FindPropNameMapping(string columnName)
Type | Name | Description |
---|---|---|
string | columnName |
Type | Description |
---|---|
string |
protected override void InitReader()
protected override void PrepareParameterForCheck()
protected override void ReadAllRecords()
protected override void Reset()