34 Commits

Author SHA1 Message Date
Sebastian
ccce24d85e
Delete MongoDB 2023-07-31 20:01:55 +02:00
Sebastian
1c891a5b58
Create directory 'MongoDB' for experiments 2023-07-31 19:56:59 +02:00
TrisNol
ed681d7c47 refactor: Implement linter feedback 2023-07-11 14:20:16 +02:00
TrisNol
4c95550dbf feat(data-extraction): MongoWrapper, DataClasses and services for News and Company data 2023-07-10 18:58:31 +02:00
TrisNol
4c65d37816 Merge main into feature/data-extraction 2023-07-10 17:15:43 +02:00
TrisNol
e44385ce3a style: Refactoring imports, adapting MongoConnector to different connection_strings 2023-06-30 20:36:03 +02:00
TrisNol
3cd8860312 adding distric court location to export 2023-06-27 19:49:23 +02:00
TrisNol
421b1e8c87 Bundesanzeiger preparation, Handeslblatt RSS feed export 2023-06-27 19:17:54 +02:00
TrisNol
37fb1b1da3 multi-process scraping, transforming unternehmensregister output 2023-06-25 15:58:53 +02:00
a9304201af
(chore): Initilised devops tools (#29)
* Added a first action

* Repaired a typo

* Repaired a typo2

* Repaired a typo2

* Added flake8 action

* Repaired a typo in the flake8 action.

* Added a first bandit action

* Added a first batch

* Added a first batch

* Added a first batch

* Added a first batch

* Added a first batch

* Added the flake8-prebuild as a need to flake8

* Added the flake8-prebuild as a need to flake8

* Added the flake8-prebuild as a need to flake8

* Added the docker socket to the volume.

* Added the flake8-prebuild as a need to flake8

* Removed latest part from container.

* Removed latest part from container.

* Removed latest part from container.

* Reworked flake8

* Reworked flake8

* Reworked flake8

* Reworked flake8

* Reworked flake8

* Reworked flake8

* Reworked flake8

* Reworked flake8

* Reworked flake8

* Reworked flake8

* Reworked flake8

* Reworked flake8

* Reworked flake8 poetry

* Reworked flake8 poetry

* Changed to 64bit

* Some edits to the runner

* Added python setup

* Added python -m to python docker image.

* Added python -m to python docker image.

* Added python -m to python docker image.

* Added python -m to python docker image.

* Added python -m to python docker image.

* Added python -m to python docker image.

* Added ra run linter

* Added ra run linter

* Added ra run linter

* Added ra run linter

* Removed redundant version

* Removed redundant version

* Added isort

* Added isort

* Added isort

* Added poetry install

* Added poetry install

* Added flake8 as lint.

* Added flake8 as lint.

* Added flake8 as lint.

* Added flake8 as lint.

* Added flake8 as lint.

* Added flake8 as lint.

* Added flake8 as lint.

* Uses nodejs and python image

* Added flake8 as lint.

* Added flake8 as lint.

* Added flake8 as lint.

* Added flake8 as lint.

* Removed selfhosted runner

* Removed self hosted runner

* Removed self hosted runner

* Removed self hosted runner

* Added black and flake8 tests

* Removed self hosted runner

* Removed self hosted runner

* Removed unneded actions

* Added a mypy error.

* Removed poetry call before boetry setup

* Removed poetry call before poetry setup

* Added a test to understand the poetry action better

* Added a test to understand the poetry action better

* Added a test to understand the poetry action better

* Added a test to understand the poetry action better

* Added a test to understand the poetry action better

* Added a test to understand the poetry action better

* Added the snook poetry builder

* Reworked the repo a bit

* Removed unneeded poetry installation

* Added the isort action

* Added isort test

* Added ruff

* Added full ruff configuration

* Added full ruff configuration2

* Added full ruff configuration2

* Removed duplicat configurations

* Removed some redundant pre-commit hooks

* Removed unneeded actions.

* Removed unneeded actions.

* Repaired ruff

* Added tests.

* Removed

* Removed

* Removed a missing file

* Removed a missing file

* Removed a missing file

* Removed a missing file

* Removed a missing file

* Added reports as artifacts

* Added reports as artifacts

* Added reports as artifacts

* Removed the unneded poetry test

* Added a license checker.

* Added a license checker.

* Removed some unneeded configuration.

* Removed the import reformatted.

* Added doc generation.

* Added doc generation.

* Added license summary.

* Add

* Add lint

* Switched pip-licenses to poetry.

* Switched pip-licenses to poetry.

* Switched pip-licenses to poetry.

* Remove some more packages.

* Remove some more packages.

* Added a make file

* Added a make file

* Added a make file

* Added a make file

* Added a make file

* Added a make file

* Added a make file

* Added a make file

* Added a make file

* Added a make file

* Added a make file

* Added a make file

* Added a make file

* Added a make file

* Added a make file

* Added version codes to the main package

* Changed the format of the md files

* Presentation first draft

* Version up and added extensions

* Version up and added extensions

* Version up and added extensions

* Removed the venv path from docbuild

* Actions version up

* Actions version up

* Actions version up

* Actions version up

* Actions version up

* Actions version up

* Experiements with sphinx

* Experiments with sphinx

* Experiments with sphinx

* Experiments with sphinx

* Experiments with sphinx

* Experiments with sphinx

* Experiments with sphinx

* Experiments with sphinx

* First draft of the sphinx documentation.

* Added the protocol to the time series.

* Added the protocol to the time series.

* First draft ot a first build pipline

* Added mermaid version support

* Added documentations pull and branch request requirements.

* Added documentations pull and branch request requirements.

* Added documentations pull and branch request requirements.

* Added documentations pull and branch request requirements.

* Tests should now be passing

* Tests should now be passing

* Tests should now be passing

* Tests should now be passing

* Tests should now be passing

* Tests should now be passing

* Tests should now be passing

* Tests should now be passing

* Add safety

* Add safety

* Add safety

* Added the action on pull_request_target

* Added the action on pull_request_target

* Added the action on pull_request_target

* Added a pytest coverage report

* Added a pytest coverage report

* Added a pytest coverage report

* Added a pytest coverage report

* Added a pytest coverage report

* Added a build step

* Added a build step

* Added a build step

* Added a build step

* Changed the lint action to work only on python changes.

* Changed the lint action to work only on python changes.

* Changed the lint action to work only on python changes.

* Added the ability to compile a html report

* Added the ability to compile a html report

* Added the ability to compile a html report

* Added the ability to compile a html report

* Added the ability to compile a html report

* Added the ability to compile a html report

* Added the ability to compile a html report

* Added the ability to compile a html report

* Added the ability to compile a html report

* Added the ability to compile a html report

* Added the ability to compile a html report

* Added the ability to compile a html report

* Added the ability to compile a html report

* Added the ability to compile a html report

* Added the ability to compile a html report

* Coverage

* Finished test and build workflow

* Finished test and build workflow

* Finished test and build workflow

* Finished test and build workflow

* Finished test and build workflow

* Finished test and build workflow

* Finished test and build workflow

* Finished test and build workflow

* Finished test and build workflow

* Finished test and build workflow

* Finished test and build workflow

* Finished test and build workflow

* Finished test and build workflow

* Finished test and build workflow

* Finished test and build workflow

* Finished test and build workflow

* Finished test and build workflow

* Finished test and build workflow

* Finished test and build workflow

* Finished test and build workflow

* Repaired a bug.

* Repaired a bug.

* Repaired a bug.

* Repaired a bug.

* Repaired a bug.

* Added a github branch.ref

* Removed a poetry install

* Docbuild now excludes templates

* Added the seminarpräsentation to the documentation build

* Added the seminarpräsentation to the documentation build

* Added the seminarpräsentation to the documentation build

* dded a few images

* Changed the pre-commit image

* Changed the pre-commit image

* Presentation done

* Never executing jupyter for sphinx

* Never executing jupyter for sphinx

* Never executing jupyter for sphinx

* Never executing jupyter for sphinx

* Never executing jupyter for sphinx
2023-06-23 18:47:04 +02:00
TrisNol
c9c7b0cf7a code cleanup, presentation on data extraction 2023-06-19 18:02:34 +02:00
TrisNol
6e31bc62bd mongodb wrapper for managing News objects 2023-06-16 18:50:19 +02:00
TrisNol
5b96bb7e3e adding company ID as well as compatible dataclasses 2023-06-16 18:00:11 +02:00
TrisNol
d3d8adabad dockerized mongodb as staging DB 2023-06-15 20:24:39 +02:00
TrisNol
3e737fbac5 first news article data extraction from tagesschau api 2023-06-15 18:04:23 +02:00
TrisNol
058c16b3ff Bulk process Unternehmensregister .xmls 2023-06-11 13:11:44 +02:00
TrisNol
1010b43a5f Extract first stakeholder informationen from Unternehmensregister export 2023-06-09 14:23:56 +02:00
TrisNol
e2ad2d475a Traverse all pages 2023-06-09 13:51:36 +02:00
TrisNol
d69368318f Download Unternehmensregister export via Selenium 2023-06-09 13:01:46 +02:00
SeZett
ba46532e0a Change on timeseries Notebook: filepath 2023-05-11 15:56:14 +02:00
RonnyFlex
6d509ee6ed News research + Abstract verflechtungsanalyse V1 2023-05-08 21:01:33 +02:00
SeZett
783891557b added ideas to DB scheme 2023-05-04 11:20:37 +02:00
KM-R
375d96b0ed
Example for sentiment analysis using VADER
To compare another sentiment analysis library to FINBert, used the same sample texts
2023-05-03 00:06:09 +02:00
f1e1a05fe8
Removed empty cell. 2023-05-01 13:21:58 +02:00
ccfb1e7899
Added a translation prior to the use of FinBert. 2023-05-01 13:19:52 +02:00
d3b14e78ea
Merge remote-tracking branch 'github/main' into main 2023-05-01 12:21:37 +02:00
29347ce7fb
Added a text in research central. 2023-05-01 12:20:56 +02:00
TrisNol
0b4d955d26 feat(data extraction): Scraping data from Bundesanzeiger and parsing the results 2023-05-01 11:33:50 +02:00
141070ea16
Added another example 2023-04-30 23:32:13 +02:00
5a52f61c1c
Added a first trail with FinBert. 2023-04-30 23:23:02 +02:00
SeZett
cac8610eda Ideas regarding timeseries 2023-04-20 14:53:53 +02:00
TrisNol
a79c5b6560 extended bundesanzeiger API test 2023-04-12 10:58:39 +02:00
TrisNol
1762f41cb1 api tests: Adding previous test results for different apis 2023-04-07 11:51:55 +02:00
262476bd87
Added a frist structure 2023-04-06 19:00:15 +02:00