94 Commits

Author SHA1 Message Date
ea9c777217
Enabeling data processing in docker container. (#195) 2023-10-06 18:36:29 +02:00
ea89a64a80
Linter update to ruff==0.0.292 (#191) 2023-10-04 20:04:59 +02:00
d2d4a436f8
Add a cli interface to choose a configuration (#163)
- [x] add a cli to the webserver to take env variables into account 
- [x] add a cli to the data processing that takes enviromental variable
as a valid source into account
- [x] rework the cli for the reset sql command
- [x] rework the cli for the copying of sql data from one db to another
2023-10-02 20:31:42 +02:00
2abe12f027
Add a function to convert DM to EUR (#168)
The function is ment to transform the captial dict into a format that
can be added as a kwarg (**norm_capital(capital_dict) to the company
entities.
This PR only contains the function itself.
2023-10-02 19:46:17 +02:00
05472cc16a
Added longitude/latitude and positional accuracy to the company data (#180) 2023-10-02 17:18:04 +02:00
4d99601fc6
Optional docs Feature: Changelog (#155)
Added the ability to generate a changelog of the main branch in the
Documenation.
2023-09-28 19:05:05 +02:00
552e1bdf96
Added some metadata to the package (#144)
Filled out the mainter section.
Added a new description.
Added the path to the repo to the package.
Added the path to the documentation.

Co-authored-by: KM-R <129882581+KM-R@users.noreply.github.com>
Co-authored-by: Tristan Nolde <tristan.nolde@yahoo.de>
2023-09-28 17:17:01 +02:00
KM-R
9566276047
Create multi page layout (#147)
Created two pages (home and company), page reloads after company
selection in dropdown or clicking the home button.
2023-09-26 18:38:40 +02:00
54dd574fb8
Added the ability to include drawio diagrams into the sphinx docs. (#107) 2023-09-25 20:37:59 +02:00
091e67de79
build first set of docker container in pipline and place them in the the github registry (#142)
- added a Dockerfile for the thre containers
- added a workflow step to build and placing the container in the
registry
- added a docker-compose.yaml to use the build images
- added a docker compose to build the images locally and a script for
prebuild steps
2023-09-24 16:32:52 +00:00
Tristan Nolde
5c8d20f4c2
Feature/additional stammdaten (#132)
Feature/additional stammdaten
2023-09-24 15:31:17 +02:00
820fb3e52b
Repaired the SQL copy and reduced the log volume a bit (#141)
- Added a cli interface to the SQL copy
- Repaired the SQL copy function
- Added the SQL copy function to the scripts
- Reduced the logging verbosity
2023-09-24 15:11:49 +02:00
TrisNol
81b8752abf Merge branch 'main' into feature/additional-stammdaten 2023-09-24 13:49:02 +02:00
d9ca808efa
Mypy does not validate if type: ignore comments are needed. (#137) 2023-09-23 12:44:27 +02:00
c5a383e903
Pipline / dependency updates (#145) 2023-09-23 10:49:14 +02:00
TrisNol
77f08cd901 Merge branch 'main' into feature/additional-stammdaten 2023-09-23 10:32:09 +02:00
KM-R
487b2f42d1
update data based on selected company (#122)
Added UI elements to select a company and update shown data depending on chosen company



---------

Co-authored-by: Philipp Horstenkamp <philipp@horstenkamp.de>
2023-09-19 23:45:10 +02:00
TrisNol
febcd59e39 test(data-extraction): Include first unit tests 2023-09-17 19:20:28 +02:00
80f077ee7a
Version ups (#117)
Rutine version ups for
- python-depdencies
- pre-commits
- piplines
2023-09-17 13:50:53 +02:00
507647d164
Added custom 404 error code page in the sphinx docs (#115) 2023-09-13 19:01:09 +02:00
TrisNol
cf92cb61cc checkpoint(data-ingestion): Extract founding_date and other stats 2023-09-12 19:07:23 +02:00
d64f53eca9
Script for the transfer of data from Mongo to SQL (#80) 2023-09-11 20:21:35 +02:00
cee1ef901a
Updatest vor everything (#106)
- direct dependencies
- other depdenencies
- pre-commit
2023-09-09 16:47:48 +00:00
TrisNol
2c8805e12f checkpoint: Implement PR feedback 2023-09-09 18:22:20 +02:00
TrisNol
0f6ea8a1ec feat(config): Read secrets from .env file and environemnt variables 2023-09-09 17:59:30 +02:00
TrisNol
0a374a10f6 test(data-ingestion): Increase coverage on Bundesanzeiger wrapper 2023-09-06 18:11:47 +02:00
TrisNol
00a5e9ec25 Merge branch 'main' into feat/fetch-financials 2023-09-06 17:16:48 +02:00
KM-R
80faf57c77
54 get data from multiple tables postgres (#72)
Create first dash board including header, basic company information and
finance plot

---------

Co-authored-by: Philipp Horstenkamp <philipp@horstenkamp.de>
2023-09-03 16:18:19 +02:00
TrisNol
ac07bd8e8e improve financial fetch, bind financial KPIs to enum 2023-08-29 17:10:42 +02:00
TrisNol
f64e0dd96e fix: Install deutschland via HTTPS link 2023-08-18 14:27:32 +02:00
TrisNol
309755383e Install deutschland package 2023-08-18 14:15:05 +02:00
e4a57c9136
Chore/rework workflow (#52)
* Reworked the lint action
* Removed the file change requirement
* Repaired mypy
* Repaired pip-audit
2023-08-17 22:05:12 +02:00
faca0a0af6
Repaired a mypy option 2023-08-16 22:10:52 +02:00
9c237402be
Excluded ipynb files from json pretty formatter + poetry update (#50)
* Excluded ipynb files from json pretty formatter +  poetry update
* Reworked pip audit
* Relocated dependencies
* Reworked the tests a bit.
* Poetry update
* Excluded lint and test workflows against the target branch
2023-08-16 20:25:15 +02:00
KM-R
1e7e55e649
Visualization first running dashboard (#51)
* added session maker

* Update prebuild psycopg-build2

* added table dash

* Update company_stats_dash

* Repaired a test.

* update connector_test

---------

Co-authored-by: Tim <tim.ronneburg@outlook.de>
Co-authored-by: Philipp Horstenkamp <philipp@horstenkamp.de>
2023-08-15 21:30:58 +02:00
Tim
ea7c955533 fixed sqlAlchemy and db connection
Co-authored-by: Tristan Nolde <TrisNol@users.noreply.github.com>
2023-08-14 20:59:30 +02:00
67c94cfb7e
Changes sql-alchemy to version 1.4.49 2023-08-13 15:00:29 +02:00
TrisNol
d565770b99 checkpoint(db): Refactor mongo utils, extract postgres entities from Juptyer 2023-08-11 15:12:18 +02:00
TrisNol
1e44197199 add types-tqdm 2023-07-20 16:46:53 +02:00
TrisNol
b788ee3659 test: Adding unit tests 2023-07-16 11:25:21 +02:00
1c621f46a7
Added selenium to poetry 2023-07-11 01:59:11 +02:00
TrisNol
4c95550dbf feat(data-extraction): MongoWrapper, DataClasses and services for News and Company data 2023-07-10 18:58:31 +02:00
a9304201af
(chore): Initilised devops tools (#29)
* Added a first action

* Repaired a typo

* Repaired a typo2

* Repaired a typo2

* Added flake8 action

* Repaired a typo in the flake8 action.

* Added a first bandit action

* Added a first batch

* Added a first batch

* Added a first batch

* Added a first batch

* Added a first batch

* Added the flake8-prebuild as a need to flake8

* Added the flake8-prebuild as a need to flake8

* Added the flake8-prebuild as a need to flake8

* Added the docker socket to the volume.

* Added the flake8-prebuild as a need to flake8

* Removed latest part from container.

* Removed latest part from container.

* Removed latest part from container.

* Reworked flake8

* Reworked flake8

* Reworked flake8

* Reworked flake8

* Reworked flake8

* Reworked flake8

* Reworked flake8

* Reworked flake8

* Reworked flake8

* Reworked flake8

* Reworked flake8

* Reworked flake8

* Reworked flake8 poetry

* Reworked flake8 poetry

* Changed to 64bit

* Some edits to the runner

* Added python setup

* Added python -m to python docker image.

* Added python -m to python docker image.

* Added python -m to python docker image.

* Added python -m to python docker image.

* Added python -m to python docker image.

* Added python -m to python docker image.

* Added ra run linter

* Added ra run linter

* Added ra run linter

* Added ra run linter

* Removed redundant version

* Removed redundant version

* Added isort

* Added isort

* Added isort

* Added poetry install

* Added poetry install

* Added flake8 as lint.

* Added flake8 as lint.

* Added flake8 as lint.

* Added flake8 as lint.

* Added flake8 as lint.

* Added flake8 as lint.

* Added flake8 as lint.

* Uses nodejs and python image

* Added flake8 as lint.

* Added flake8 as lint.

* Added flake8 as lint.

* Added flake8 as lint.

* Removed selfhosted runner

* Removed self hosted runner

* Removed self hosted runner

* Removed self hosted runner

* Added black and flake8 tests

* Removed self hosted runner

* Removed self hosted runner

* Removed unneded actions

* Added a mypy error.

* Removed poetry call before boetry setup

* Removed poetry call before poetry setup

* Added a test to understand the poetry action better

* Added a test to understand the poetry action better

* Added a test to understand the poetry action better

* Added a test to understand the poetry action better

* Added a test to understand the poetry action better

* Added a test to understand the poetry action better

* Added the snook poetry builder

* Reworked the repo a bit

* Removed unneeded poetry installation

* Added the isort action

* Added isort test

* Added ruff

* Added full ruff configuration

* Added full ruff configuration2

* Added full ruff configuration2

* Removed duplicat configurations

* Removed some redundant pre-commit hooks

* Removed unneeded actions.

* Removed unneeded actions.

* Repaired ruff

* Added tests.

* Removed

* Removed

* Removed a missing file

* Removed a missing file

* Removed a missing file

* Removed a missing file

* Removed a missing file

* Added reports as artifacts

* Added reports as artifacts

* Added reports as artifacts

* Removed the unneded poetry test

* Added a license checker.

* Added a license checker.

* Removed some unneeded configuration.

* Removed the import reformatted.

* Added doc generation.

* Added doc generation.

* Added license summary.

* Add

* Add lint

* Switched pip-licenses to poetry.

* Switched pip-licenses to poetry.

* Switched pip-licenses to poetry.

* Remove some more packages.

* Remove some more packages.

* Added a make file

* Added a make file

* Added a make file

* Added a make file

* Added a make file

* Added a make file

* Added a make file

* Added a make file

* Added a make file

* Added a make file

* Added a make file

* Added a make file

* Added a make file

* Added a make file

* Added a make file

* Added version codes to the main package

* Changed the format of the md files

* Presentation first draft

* Version up and added extensions

* Version up and added extensions

* Version up and added extensions

* Removed the venv path from docbuild

* Actions version up

* Actions version up

* Actions version up

* Actions version up

* Actions version up

* Actions version up

* Experiements with sphinx

* Experiments with sphinx

* Experiments with sphinx

* Experiments with sphinx

* Experiments with sphinx

* Experiments with sphinx

* Experiments with sphinx

* Experiments with sphinx

* First draft of the sphinx documentation.

* Added the protocol to the time series.

* Added the protocol to the time series.

* First draft ot a first build pipline

* Added mermaid version support

* Added documentations pull and branch request requirements.

* Added documentations pull and branch request requirements.

* Added documentations pull and branch request requirements.

* Added documentations pull and branch request requirements.

* Tests should now be passing

* Tests should now be passing

* Tests should now be passing

* Tests should now be passing

* Tests should now be passing

* Tests should now be passing

* Tests should now be passing

* Tests should now be passing

* Add safety

* Add safety

* Add safety

* Added the action on pull_request_target

* Added the action on pull_request_target

* Added the action on pull_request_target

* Added a pytest coverage report

* Added a pytest coverage report

* Added a pytest coverage report

* Added a pytest coverage report

* Added a pytest coverage report

* Added a build step

* Added a build step

* Added a build step

* Added a build step

* Changed the lint action to work only on python changes.

* Changed the lint action to work only on python changes.

* Changed the lint action to work only on python changes.

* Added the ability to compile a html report

* Added the ability to compile a html report

* Added the ability to compile a html report

* Added the ability to compile a html report

* Added the ability to compile a html report

* Added the ability to compile a html report

* Added the ability to compile a html report

* Added the ability to compile a html report

* Added the ability to compile a html report

* Added the ability to compile a html report

* Added the ability to compile a html report

* Added the ability to compile a html report

* Added the ability to compile a html report

* Added the ability to compile a html report

* Added the ability to compile a html report

* Coverage

* Finished test and build workflow

* Finished test and build workflow

* Finished test and build workflow

* Finished test and build workflow

* Finished test and build workflow

* Finished test and build workflow

* Finished test and build workflow

* Finished test and build workflow

* Finished test and build workflow

* Finished test and build workflow

* Finished test and build workflow

* Finished test and build workflow

* Finished test and build workflow

* Finished test and build workflow

* Finished test and build workflow

* Finished test and build workflow

* Finished test and build workflow

* Finished test and build workflow

* Finished test and build workflow

* Finished test and build workflow

* Repaired a bug.

* Repaired a bug.

* Repaired a bug.

* Repaired a bug.

* Repaired a bug.

* Added a github branch.ref

* Removed a poetry install

* Docbuild now excludes templates

* Added the seminarpräsentation to the documentation build

* Added the seminarpräsentation to the documentation build

* Added the seminarpräsentation to the documentation build

* dded a few images

* Changed the pre-commit image

* Changed the pre-commit image

* Presentation done

* Never executing jupyter for sphinx

* Never executing jupyter for sphinx

* Never executing jupyter for sphinx

* Never executing jupyter for sphinx

* Never executing jupyter for sphinx
2023-06-23 18:47:04 +02:00
5a52f61c1c
Added a first trail with FinBert. 2023-04-30 23:23:02 +02:00