TrisNol
535c31fc9f
test(data-extraction): Change use of TemporaryDirection
2023-09-21 17:16:25 +02:00
TrisNol
56c2ed55ec
test(data-extraction): Delay file creation in test_rename_latest_file to avoid same timestamps
2023-09-21 16:54:23 +02:00
KM-R
487b2f42d1
update data based on selected company ( #122 )
...
Added UI elements to select a company and update shown data depending on chosen company
---------
Co-authored-by: Philipp Horstenkamp <philipp@horstenkamp.de>
2023-09-19 23:45:10 +02:00
TrisNol
7269e7e6a4
test(data-extraction): Cover transform.py
2023-09-18 20:23:19 +02:00
TrisNol
92ea72249c
refactor(data-extraction): Bind company type to an enum
2023-09-17 19:43:22 +02:00
TrisNol
d54c1cd17d
test(data-extraction): Adapt to extended enum
2023-09-17 19:25:30 +02:00
TrisNol
febcd59e39
test(data-extraction): Include first unit tests
2023-09-17 19:20:28 +02:00
80f077ee7a
Version ups ( #117 )
...
Rutine version ups for
- python-depdencies
- pre-commits
- piplines
2023-09-17 13:50:53 +02:00
06fef5361d
Lightend black in the pipline building ( #128 )
...
Pipline should now run now 20 -40s faster.
2023-09-17 13:47:20 +02:00
56b6280264
Transfer financials from staging to sql ( #129 )
2023-09-17 13:45:08 +02:00
TrisNol
bfe50ac76d
checkpoint(data-ingestion): Move Unternehmensregister code to .py
2023-09-15 17:22:54 +02:00
TrisNol
8be192e1de
checkpoint(data-ingestion): Include type in company relations, fix issue in capital for KGs
2023-09-15 15:39:42 +02:00
TrisNol
0c7216e105
checkpoint
2023-09-14 18:17:02 +02:00
TrisNol
413b43c615
checkpoint(data-ingestion): Unify date format in data
2023-09-14 16:47:11 +02:00
fea31e543b
Bugfix update for transfer of company data from mongo to sql ( #121 )
...
Fixed the following errors:
- Typo in readme.md
- Mongo delivers the last_update of an company as a string not as date.
Added:
- Entrypoint description in readme.md
2023-09-13 19:14:36 +02:00
507647d164
Added custom 404 error code page in the sphinx docs ( #115 )
2023-09-13 19:01:09 +02:00
TrisNol
cf92cb61cc
checkpoint(data-ingestion): Extract founding_date and other stats
2023-09-12 19:07:23 +02:00
ba44b082b8
Data transfer script ( #114 )
...
Transfers data betwenn two sql instances.
Limited in data volume. Should be good enough for now.
---------
Co-authored-by: Tim <tim.ronneburg@outlook.de>
2023-09-11 21:10:36 +02:00
d64f53eca9
Script for the transfer of data from Mongo to SQL ( #80 )
2023-09-11 20:21:35 +02:00
2ea99c8c17
Removed unneded lines ☺ ( #113 )
2023-09-11 19:49:08 +02:00
cee1ef901a
Updatest vor everything ( #106 )
...
- direct dependencies
- other depdenencies
- pre-commit
2023-09-09 16:47:48 +00:00
Tristan Nolde
0cca5f429e
feat(config): Read secrets from .env file and environemnt variables ( #109 )
...
Config variables used to connect to our two databased can now also be
ingested from the local environment variables. Such variables can also
be passed in using a `.env` file as described in the README.md
2023-09-09 18:41:56 +02:00
23a193a4fb
Added a small test ☺ ( #108 )
2023-09-09 18:35:50 +02:00
TrisNol
2c8805e12f
checkpoint: Implement PR feedback
2023-09-09 18:22:20 +02:00
TrisNol
330eb466e3
test(config): Fix test with os.environ mocking
2023-09-09 18:13:59 +02:00
TrisNol
0f6ea8a1ec
feat(config): Read secrets from .env file and environemnt variables
2023-09-09 17:59:30 +02:00
Tristan Nolde
2cd8def200
Feat/fetch financials ( #79 )
2023-09-09 17:28:35 +02:00
TrisNol
b91ecdfe42
test(data-ingestion): Fix myPy complaints regarding tests
2023-09-09 09:57:43 +02:00
TrisNol
2eaa1c5a65
test(utils): Cover added functiosnn in CompanyMongoService
2023-09-08 19:09:27 +02:00
TrisNol
4944372ebc
test(date-ingestion): Cover enrich_company_financials
2023-09-08 18:56:10 +02:00
2233b12468
Add sql lite session ( #71 )
...
I added an sql lite session generator.
Changes
The function `def get_session() -> Session`
has changed to
`def get_session(connect_to: ConfigProvider | str) -> Session`
If a JsonFileConfig Is given the postgress conection is checked.
If a string is given that starts with `sqlite:///` an sql db is created.
The use should otherwise be the same.
2023-09-07 18:41:10 +02:00
TrisNol
cfb4459a16
checkpoint
2023-09-07 17:46:17 +02:00
ecf94d4b61
Added an import smoke test.
2023-09-06 19:08:44 +02:00
TrisNol
0a374a10f6
test(data-ingestion): Increase coverage on Bundesanzeiger wrapper
2023-09-06 18:11:47 +02:00
TrisNol
f4a4cbc4f2
test(data-ingestion): Test Bundesanzeiger wrapper
2023-09-06 17:48:00 +02:00
TrisNol
d34bb1f3be
checkpoint(data-ingestion): Replace print statements with loguru
2023-09-06 17:19:19 +02:00
TrisNol
00a5e9ec25
Merge branch 'main' into feat/fetch-financials
2023-09-06 17:16:48 +02:00
d04255a7a8
Old date corrected in meeting notes fixed ( #76 )
...
The date in the meeting nots was from the meeting two weeks before.
2023-09-04 19:49:17 +02:00
17e89da0d8
Repo struktur dokumentieren ( #75 )
2023-09-04 19:48:14 +02:00
f20823a613
Version updates for pre-commit/poetry/linter ( #74 )
...
Just a rutine update to keep ruff up to date and sync.
2023-09-03 17:56:13 +02:00
KM-R
80faf57c77
54 get data from multiple tables postgres ( #72 )
...
Create first dash board including header, basic company information and
finance plot
---------
Co-authored-by: Philipp Horstenkamp <philipp@horstenkamp.de>
2023-09-03 16:18:19 +02:00
011184b898
Small fixes to the documentation. ( #70 )
2023-08-31 16:57:50 +02:00
69039706ac
Version updates ( #69 )
...
Version ups for poetry and pre-commit to keep them in sync with the
pipline.
2023-08-31 16:57:09 +02:00
TrisNol
b970416641
checkpoint(data-ingestion): Add method to mongo service to fetch companies with yearly results
2023-08-29 18:50:33 +02:00
TrisNol
7a2bc5db2b
multi-thread company financial ingestion
2023-08-29 18:38:25 +02:00
TrisNol
ac07bd8e8e
improve financial fetch, bind financial KPIs to enum
2023-08-29 17:10:42 +02:00
06e59b8061
Changes to the database ( #67 )
...
In this PR are pruposed changed that come up when i started the data
migration.
Plese review them an approve and disaprove.
2023-08-24 17:23:19 +02:00
KM-R
ba30ba2cc5
Viz: Company stats ( #64 )
...
First connection to Postgres database and creating dashboard with
dropdown and table
---------
Co-authored-by: Tim <tim.ronneburg@outlook.de>
Co-authored-by: Philipp Horstenkamp <philipp@horstenkamp.de>
Co-authored-by: Sebastian <94404394+SeZett@users.noreply.github.com>
2023-08-21 21:30:19 +02:00
82979b1981
Expanded a bit on the installation of the project.
2023-08-20 13:03:06 +02:00
TrisNol
d1c09d51a2
checkpoint: Update Mongo Company entries with yearly_results
2023-08-19 18:06:33 +02:00
