Philipp/aki_prj23_transparenzregister
1
0
Fork 0
mirror of https://github.com/fhswf/aki_prj23_transparenzregister.git synced 2025-06-22 15:33:55 +02:00
Code Issues Packages Projects Releases Wiki Activity

Ingest schedule (#391)

Browse Source 
Includes a new "app" running the ingestion jobs (aka fetch_news and
find_missing_companies + enrich_company_financials) on a schedule.

This also fixes an issue with the previous schedule implementation by
persisting the schedule in a file that survives new deployment and
continues where it left off.
This commit is contained in:
Tristan Nolde
2023-12-04 19:05:52 +01:00
committed by GitHub
parent 011e169383
commit 92d14b5824
10 changed files with 280 additions and 82 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
pyproject.toml
View File

@ -143,7 +143,9 @@ pytest-repeat = "^0.9.1"
copy-sql = "aki_prj23_transparenzregister.utils.sql.copy_sql:copy_db_cli"
data-processing = "aki_prj23_transparenzregister.utils.data_processing:cli"
data-transformation = "aki_prj23_transparenzregister.utils.data_transfer:transfer_data_cli"
fetch-news-schedule = "aki_prj23_transparenzregister.apps.fetch_news:fetch_news_cli"
fetch-news-schedule = "aki_prj23_transparenzregister.apps.fetch_news:cli"
find-missing-companies = "aki_prj23_transparenzregister.apps.find_missing_companies:cli"
ingest = "aki_prj23_transparenzregister.apps.ingest:cli"
reset-sql = "aki_prj23_transparenzregister.utils.sql.reset_sql:cli"
webserver = "aki_prj23_transparenzregister.ui.app:main"