37 Commits

Author SHA1 Message Date
TrisNol
ac6ca3547b test: Add unit test for news api wrapper 2023-11-11 14:30:00 +01:00
TrisNol
9edf5b1dce test: Increase coverage for multi-column headers 2023-11-11 11:03:36 +01:00
TrisNol
fecf42d75a test: Unit test new KPI extraction 2023-11-11 11:01:17 +01:00
TrisNol
f7ec3eaf24 test: Increase test coverage and refactor v3 2023-11-05 12:55:47 +01:00
TrisNol
e8d1a37cff test: Extend unit tests 2023-11-04 14:19:41 +01:00
TrisNol
61f94fa3b9 test: Unit tests 2023-11-04 11:24:36 +01:00
TrisNol
d6b07431e7 test: Adapt existing unit tests to refactored imports 2023-11-04 11:24:36 +01:00
TrisNol
600039207d test(data-extraction): Adapt unit tests to new behaviour 2023-10-17 18:16:44 +02:00
TrisNol
f1474feaf8 refactor: Adapt to extended unit tests 2023-10-15 13:21:41 +02:00
Tristan Nolde
fd47487367
Update tests/utils/data_extraction/unternehmensregister/transform_test.py
Co-authored-by: Philipp Horstenkamp <philipp@horstenkamp.de>
2023-10-15 13:07:34 +02:00
TrisNol
8db04177be feat(data-extraction): Extract c/o relation from street in company relation 2023-10-15 13:06:32 +02:00
TrisNol
eba5235dff refactor: Implement PR feedback 2023-10-15 12:05:25 +02:00
Tristan Nolde
39c13ac74a
Update tests/utils/data_extraction/unternehmensregister/transform_test.py
Co-authored-by: Philipp Horstenkamp <philipp@horstenkamp.de>
2023-10-15 11:51:11 +02:00
TrisNol
b972acee7a fix(data-extraction): Parse date from Gesellschaftsvertrag entry 2023-10-14 18:22:41 +02:00
f8c111d7e2
Resolve mismatch between staging and prod db data for financials (#211)
SQL Creation is now done dynamicly by the definition of the enumeration
type.
2023-10-14 17:16:14 +02:00
TrisNol
84d0139531 fix(data-extraction): Handle malformed date_of_birth fields 2023-10-07 17:01:34 +02:00
TrisNol
2a446a9937 checkpoint: Remove quotes from company names in relations 2023-10-03 14:33:46 +02:00
TrisNol
49498ad7c0 checkpoint: Remove quotes from company name 2023-10-03 14:33:45 +02:00
Tristan Nolde
7e9cff046a
fix(data-extraction): Parse house-number from street field if possibl… (#179) 2023-10-03 14:26:21 +02:00
01b4ce00c1
Spellchecking with PyCharm (#133)
Co-authored-by: KM-R <129882581+KM-R@users.noreply.github.com>
2023-10-02 20:47:42 +02:00
TrisNol
ab26a7a01e fix(data-extraction): Parse house-number from street field if possible, write Straße in full 2023-10-01 21:24:17 +02:00
TrisNol
2050b49fde fix(data-extraction): Resolve issue in different Bundesanzeiger formats 2023-09-25 18:37:39 +02:00
TrisNol
1e23a8d5a3 refactor(data-extraction): Move date_to_iso function to string_tools 2023-09-23 10:51:54 +02:00
TrisNol
77f08cd901 Merge branch 'main' into feature/additional-stammdaten 2023-09-23 10:32:09 +02:00
TrisNol
d6223b4192 refactor(data-extraction): Improve variable naming and exception handling 2023-09-23 10:21:26 +02:00
TrisNol
4e25be5466 test(data-extraction): Introduct load.py test and scrape test 2023-09-23 10:07:15 +02:00
TrisNol
d7f167a868 ignore types mypy 2023-09-21 18:08:20 +02:00
TrisNol
e6af96ea6d test(data-extraction): Host temporary_dir in local env 2023-09-21 17:25:41 +02:00
TrisNol
535c31fc9f test(data-extraction): Change use of TemporaryDirection 2023-09-21 17:16:25 +02:00
TrisNol
56c2ed55ec test(data-extraction): Delay file creation in test_rename_latest_file to avoid same timestamps 2023-09-21 16:54:23 +02:00
TrisNol
7269e7e6a4 test(data-extraction): Cover transform.py 2023-09-18 20:23:19 +02:00
TrisNol
febcd59e39 test(data-extraction): Include first unit tests 2023-09-17 19:20:28 +02:00
ba44b082b8
Data transfer script (#114)
Transfers data betwenn two sql instances.
Limited in data volume. Should be good enough for now.

---------

Co-authored-by: Tim <tim.ronneburg@outlook.de>
2023-09-11 21:10:36 +02:00
d64f53eca9
Script for the transfer of data from Mongo to SQL (#80) 2023-09-11 20:21:35 +02:00
TrisNol
0a374a10f6 test(data-ingestion): Increase coverage on Bundesanzeiger wrapper 2023-09-06 18:11:47 +02:00
TrisNol
f4a4cbc4f2 test(data-ingestion): Test Bundesanzeiger wrapper 2023-09-06 17:48:00 +02:00
TrisNol
1e15656028 refactor: Pull Auditor extraction into Bundesanzeiger utils 2023-08-18 16:21:52 +02:00