TrisNol
77f08cd901
Merge branch 'main' into feature/additional-stammdaten
2023-09-23 10:32:09 +02:00
TrisNol
d6223b4192
refactor(data-extraction): Improve variable naming and exception handling
2023-09-23 10:21:26 +02:00
TrisNol
4e25be5466
test(data-extraction): Introduct load.py test and scrape test
2023-09-23 10:07:15 +02:00
TrisNol
d7f167a868
ignore types mypy
2023-09-21 18:08:20 +02:00
TrisNol
e6af96ea6d
test(data-extraction): Host temporary_dir in local env
2023-09-21 17:25:41 +02:00
TrisNol
535c31fc9f
test(data-extraction): Change use of TemporaryDirection
2023-09-21 17:16:25 +02:00
TrisNol
56c2ed55ec
test(data-extraction): Delay file creation in test_rename_latest_file to avoid same timestamps
2023-09-21 16:54:23 +02:00
KM-R
487b2f42d1
update data based on selected company ( #122 )
...
Added UI elements to select a company and update shown data depending on chosen company
---------
Co-authored-by: Philipp Horstenkamp <philipp@horstenkamp.de>
2023-09-19 23:45:10 +02:00
TrisNol
7269e7e6a4
test(data-extraction): Cover transform.py
2023-09-18 20:23:19 +02:00
TrisNol
febcd59e39
test(data-extraction): Include first unit tests
2023-09-17 19:20:28 +02:00
56b6280264
Transfer financials from staging to sql ( #129 )
2023-09-17 13:45:08 +02:00
fea31e543b
Bugfix update for transfer of company data from mongo to sql ( #121 )
...
Fixed the following errors:
- Typo in readme.md
- Mongo delivers the last_update of an company as a string not as date.
Added:
- Entrypoint description in readme.md
2023-09-13 19:14:36 +02:00
ba44b082b8
Data transfer script ( #114 )
...
Transfers data betwenn two sql instances.
Limited in data volume. Should be good enough for now.
---------
Co-authored-by: Tim <tim.ronneburg@outlook.de>
2023-09-11 21:10:36 +02:00
d64f53eca9
Script for the transfer of data from Mongo to SQL ( #80 )
2023-09-11 20:21:35 +02:00
Tristan Nolde
2cd8def200
Feat/fetch financials ( #79 )
2023-09-09 17:28:35 +02:00
TrisNol
b91ecdfe42
test(data-ingestion): Fix myPy complaints regarding tests
2023-09-09 09:57:43 +02:00
TrisNol
2eaa1c5a65
test(utils): Cover added functiosnn in CompanyMongoService
2023-09-08 19:09:27 +02:00
2233b12468
Add sql lite session ( #71 )
...
I added an sql lite session generator.
Changes
The function `def get_session() -> Session`
has changed to
`def get_session(connect_to: ConfigProvider | str) -> Session`
If a JsonFileConfig Is given the postgress conection is checked.
If a string is given that starts with `sqlite:///` an sql db is created.
The use should otherwise be the same.
2023-09-07 18:41:10 +02:00
TrisNol
0a374a10f6
test(data-ingestion): Increase coverage on Bundesanzeiger wrapper
2023-09-06 18:11:47 +02:00
TrisNol
f4a4cbc4f2
test(data-ingestion): Test Bundesanzeiger wrapper
2023-09-06 17:48:00 +02:00
TrisNol
00a5e9ec25
Merge branch 'main' into feat/fetch-financials
2023-09-06 17:16:48 +02:00
06e59b8061
Changes to the database ( #67 )
...
In this PR are pruposed changed that come up when i started the data
migration.
Plese review them an approve and disaprove.
2023-08-24 17:23:19 +02:00
TrisNol
1e15656028
refactor: Pull Auditor extraction into Bundesanzeiger utils
2023-08-18 16:21:52 +02:00
e4a57c9136
Chore/rework workflow ( #52 )
...
* Reworked the lint action
* Removed the file change requirement
* Repaired mypy
* Repaired pip-audit
2023-08-17 22:05:12 +02:00
KM-R
1e7e55e649
Visualization first running dashboard ( #51 )
...
* added session maker
* Update prebuild psycopg-build2
* added table dash
* Update company_stats_dash
* Repaired a test.
* update connector_test
---------
Co-authored-by: Tim <tim.ronneburg@outlook.de>
Co-authored-by: Philipp Horstenkamp <philipp@horstenkamp.de>
2023-08-15 21:30:58 +02:00
TrisNol
775b360ff7
test: Fix unit test of connector.py
2023-08-11 16:50:22 +02:00
TrisNol
46cdaa625f
test: Add Unit test to connector.py
2023-08-11 16:45:05 +02:00
TrisNol
b2b3ab2f04
test: Add test for entities file
2023-08-11 16:35:57 +02:00
TrisNol
4e7f6bca1d
test: Add missing unit tests
2023-08-11 16:29:08 +02:00
TrisNol
d565770b99
checkpoint(db): Refactor mongo utils, extract postgres entities from Juptyer
2023-08-11 15:12:18 +02:00
TrisNol
b05088c966
test: utils.mongo
2023-07-16 15:53:01 +02:00
TrisNol
4b5f63ee62
test: Adding unit tests for news_mongo_service
2023-07-16 15:33:54 +02:00
TrisNol
b788ee3659
test: Adding unit tests
2023-07-16 11:25:21 +02:00
