pypeliner.readers.stream_readers.file_readers

File readers module, defines stream reading for text and csv files (line by line).

Classes

BaseReader	A base class for readers
FileLinesReader	A reader used to read a file line by line using a generator.
CSVFileStreamReader	A reader used to read a stream of rows from a csv file.

Functions

convert_row_values_to_numbers(→ List[Union[int, float, ...)

A function to convert a csv file row string values to numbers if possible.

Module Contents

class pypeliner.readers.stream_readers.file_readers.BaseReader

A base class for readers

abstract read() → Any

An abstract method that defines reading logic.

Returns:

data to be processed.

pypeliner.readers.stream_readers.file_readers.convert_row_values_to_numbers(row: List[str]) → List[int | float | str]

A function to convert a csv file row string values to numbers if possible.

e.g:

input: [“1”, “2”, “a”, “b”] output: [1, 2, “a”, “b”]

Args:

row: a row from a csv file.

Returns:

the row with number values instead of strings

class pypeliner.readers.stream_readers.file_readers.FileLinesReader(file_path: str)

Bases: pypeliner.readers.base.BaseReader

A reader used to read a file line by line using a generator. Args:

file_path: file directory.

read() → str

An abstract method that defines reading logic.

Returns:

data to be processed.

class pypeliner.readers.stream_readers.file_readers.CSVFileStreamReader(file_path: str, load_titles_row: bool = True, delimiter: str = ',', detect_numbers: bool = True)

Bases: pypeliner.readers.base.BaseReader

A reader used to read a stream of rows from a csv file.

Args:

file_path: file directory. load_titles_row: read the first line from the file to consider that as the columns names. delimiter: rows delimiter. detect_numbers: convert string number values to int/float.

read() → List[str]

An abstract method that defines reading logic.

Returns:

data to be processed.