components package

Submodules

components.config module

class components.config.Config(cli_reports_list_path: str, cli_report: str, cli_path: str, cli_threads: int)[source]

Bases: object

Concrete class representing Config object. Contains entire configuration required for a program.

static load_env_file() → None[source]

Loads .env config file from root folder.

class components.config.ConfigProtocol(*args, **kwargs)[source]

Bases: Protocol

Protocol class for config object.

Parameters:

  • cli_reports_list_path (str) – CLI argument for input report list path.

  • cli_report (str) – CLI argument for single report params.

  • cli_path (str) – CLI argument for save location path override.

  • cli_threads (int) – CLI argument for number of threads to use.

cli_path: str
cli_report: str
cli_reports_list_path: str
cli_threads: int
static load_env_file() → None[source]

Loads system environment variables.

components.connectors module

class components.connectors.Connector(*args, **kwargs)[source]

Bases: Protocol

Protocol class for connector object.

Parameters:

  • queue (Queue) – Shared queue object.

  • verbose (bool) – CLI parameter used as switch between progress bar and logging to stdout on INFO level.

  • timeout (int) – Request’s timeout value in seconds.

  • headers (dict[str, str]) – Headers required to establish the connection.

check_connection() → bool[source]

Checks connection with given domain.

Returns:

Flag, True if connection is established, False otherwise.

Return type:

bool

headers: dict[str, str]
queue: Queue
async report_gathering(reports: list[components.containers.ReportProtocol], session: ClientSession) → None[source]

Collects asynchronous responses from the servers.

Parameters:

  • reports (list[ReportProt]) – Collection of ReportProtocol objects.

  • session (aiohttp.ClientSession) – HTTP client session object to handle request in transaction.

timeout: int
verbose: bool
class components.connectors.SfdcConnector(queue: Queue, *, verbose: bool = False, timeout: int = 900, headers: dict[str, str] = {'Content-Type': 'application/csv', 'X-PrettyPrint': '1'})[source]

Bases: object

Concrete class representing Connector object for SFDC

Parameters:

  • queue (Queue) – Shared queue object.

  • verbose – CLI parameter used as switch between progress bar and logging to stdout on INFO level. Defaults to False.

  • timeout (int) – Request’s timeout value in seconds. Defaults to 900.

  • headers (dict[str, str]) – Headers required to establish the connection. Defaults to {‘Content-Type’: ‘application/csv’, ‘X-PrettyPrint’: ‘1’}.

  • export_params (str) – Default parameters required by SFDC. Defaults to ‘?export=csv&enc=UTF-8&isdtp=p1’.

check_connection() → bool[source]

Checks the connection with given domain.

Returns:

Flag, True if connection was successful, False wasn’t.

Return type:

bool

async handle_requests(reports: list[components.containers.ReportProtocol]) → None[source]

Creates session and process asynchronous tasks.

Parameters:

reports (list[ReportProtocol]) – Collection of ReportProtocol instances.

components.containers module

class components.containers.ReportProtocol(*args, **kwargs)[source]

Bases: Protocol

Protocol class for report object.

Parameters:

  • type (str) – Report type, allowed options [‘SFDC’], type drives connector and report objects selection

  • name (str) – Report name, propagated to report file name

  • id (str) – Report id, identification number of the report in SFDC

  • path (PathLike) – Report path, save location for the report in form of Path object

  • export_params (str) – Default parameters required by SFDC. Defaults to ‘?export=csv&enc=UTF-8&isdtp=p1’.

  • downloaded (bool) – Flag indicating whether the reports has been succesfully downloaded or not

  • valid (bool) – Flag indicating whether the response has been succesfully retrieved or not

  • created_date (datetime) – Report save completition date

  • pull_date (timedelta) – Report response completition date

  • processing_time – The time it took to process the report in seconds

  • attempt_count (int) – Number of attempts to process the report

  • size (float) – Size of saved report file in Mb

  • response (str) – Container for request response

  • content (DataFrame) – Pandas DataFrame based on response

attempt_count: int
content: DataFrame
created_date: datetime
downloaded: bool
export_params: str
id: str
name: str
path: PathLike
processing_time: timedelta
pull_date: datetime
response: str
size: float
type: str
valid: bool
class components.containers.ReportsContainer(reports_params_list: list[dict[str, Any]], summary_report_path: PathLike)[source]

Bases: object

Concrete class representing ReportContainer object.

create_reports() → list[components.containers.ReportProtocol][source]

Orchestrating method to handle report objects factory

Returns:

Collection of Reports

Return type:

list[SfdcReport]

create_summary_report() → None[source]

Creates summary report which consist of all important details regarding reports. Report is generated once all the reports are completed.

class components.containers.ReportsContainerProtocol(*args, **kwargs)[source]

Bases: Protocol

Protocol class for report container object.

Parameters:

  • report_params_list (list[dict[str, Any]]) – Collection of dicts with parameters for object crafting.

  • summary_report_path (PathLike) – Path to save location of summary report.

create_reports() → list[components.containers.ReportProtocol][source]

Orchestrating method to handle report objects factory

Returns:

Collection of Reports

Return type:

list[ReportProtocol]

create_summary_report() → None[source]

Creates summary report which consist of all important details regarding Report objects. Summary report is generated once all the reports are completed.

class components.containers.SfdcReport(type: str, name: str, id: str, path: ~os.PathLike, export_params: str = '?export=csv&enc=UTF-8&isdtp=p1', downloaded: bool = False, valid: bool = False, created_date: ~datetime.datetime = datetime.datetime(2023, 2, 24, 11, 53, 24, 539658), pull_date: ~datetime.datetime = datetime.datetime(2023, 2, 24, 11, 53, 24, 539658), processing_time: ~datetime.timedelta = datetime.timedelta(0), attempt_count: int = 0, size: float = 0.0, response: str = '', content: ~pandas.core.frame.DataFrame = <factory>)[source]

Bases: object

Concrete class representing Report object from SFDC.

Parameters:

  • type (str) – Report type, allowed options [‘SFDC’], type drives connector and report objects selection

  • name (str) – Report name, propagated to report file name

  • id (str) – Report id, identification number of the report in SFDC

  • path (PathLike) – Report path, save location for the report in form of Path object

  • export_params (str) – Default parameters required by SFDC. Defaults to ‘?export=csv&enc=UTF-8&isdtp=p1’.

  • downloaded (bool) – Flag indicating whether the reports has been succesfully downloaded or not. Defaults to False.

  • valid (bool) – Flag indicating whether the response has been succesfully retrieved or not. Defaults to False.

  • created_date (datetime) – Report save completition date. Defaults to current datetime.

  • pull_date (timedelta) – Report response completition date. Defaults to current datetime.

  • processing_time – The time it took to process the report in seconds. Defaults to 0 microseconds.

  • attempt_count (int) – Number of attempts to process the report. Defaults to 0 .

  • size (float) – Size of saved report file in Mb. Defaults to 0.0 .

  • response (str) – Container for request response. Defaults to empty string.

  • content (DataFrame) – Pandas DataFrame based on response. Defaults to empty Pandas DataFrame.

attempt_count: int
content: DataFrame
created_date: datetime
downloaded: bool
export_params: str
id: str
name: str
path: PathLike
processing_time: timedelta
pull_date: datetime
response: str
size: float
type: str
valid: bool

components.exceptions module

exception components.exceptions.EnvFileNotPresent(message: str = '.env file not present in main directory')[source]

Bases: Exception

Exception raised if the .env file is not present in main directory.

…

message

explanation of the error

Type:

str

exception components.exceptions.OutdatedSIDError(message: str = 'Your SID is outdate, please provide recent SID')[source]

Bases: Exception

Exception raised for errors in the SID value.

…

message

explanation of the error

Type:

str

components.handlers module

class components.handlers.Worker(queue: Queue)[source]

Bases: Thread

Concrete class representing Worker object.

process_report(report: ReportProtocol) → None[source]

Orchiestrates entire process of downloading the report.

Parameters:

report (ReportProtocol) – Instance of the ReportProtocol object.

run() → NoReturn[source]

begins to listen to the queue. Starts processing once will get item from the queue. Sends signal to the queue once task is done.

Returns:

Function never returns.

Return type:

NoReturn

class components.handlers.WorkerFactory(queue: Queue, *, threads: int = 1)[source]

Bases: object

Concrete class representing WorkerFactory object.

static active_workers() → int[source]

Returns number of currently active workers.

Returns:

Number of workers.

Return type:

int

create_workers() → None[source]

Deploys given number of workers.

class components.handlers.WorkerFactoryProtocol(*args, **kwargs)[source]

Bases: Protocol

Protocol class for worker factory objects.

Parameters:

  • queue (Queue) – Shared, thread-safe queue.

  • threads (int) – Number of threads, equal to number of Workers to be deployed.

static active_workers() → int[source]

Counts active works in current time.

Returns:

Number of active workers.

Return type:

int

create_workers() → None[source]

Creates workers on independent threads

queue: Queue
threads: int
class components.handlers.WorkerProtocol(*args, **kwargs)[source]

Bases: Protocol

Protocol class for worker factory objects.

Parameters:

queue (Queue) – Shared, thread-safe queue.

report_processing(report: ReportProtocol) → None[source]

Orchiestrates the report processing.

Parameters:

report (ReportProtocol) – Instance of the ReportProtocol object.

run() → NoReturn[source]

Starts listner process on sepearet thread, awaits objects in the queue.

Returns:

Method never returns.

Return type:

NoReturn

components.loggers module

components.loggers.logger_configurer(cli_stdout_loglevel: str, cli_file_loglevel: str, verbose: bool) → None[source]

Configures logger settings for file and stdout handlers.

Parameters:

  • cli_stdout_loglevel (str) – LogLevel for stdout logger handler based on CLI option. Defaults to ERROR.

  • cli_file_loglevel – LogLevel for file logger handler based on CLI option. Defaults to INFO.

  • verbose (str) – Flag toggling LogLevel for stdout logger handler, if True sets to ERROR, else INFO.

Module contents