Created a dataprocessing pipline that enhances the raw mined data with
Organsiation extractions and sentiment analysis prio to moving the data
to the sql db.
The transfer of matched data is done afterword.
---------
Co-authored-by: SeZett <zeleny.sebastian@fh-swf.de>
The git hash looks currently like this:
`Build from GiT Revision: ('d3189f78c47d812f447739f97876ced01227e1fd',)`
After this change it should look like this:
`Build from GiT Revision: d3189f78c47d812f447739f97876ced01227e1fd`
I think the current blueish plotly default theme bits itself quit beadly
with @KM-R s great work.
I propose to set the deault plotly template to either:
- plotly_white
- ggplot2
- seaborn
Simply change the constant i have set will change this.
Heren you can find the catalog of themes:
https://plotly.com/python/templates/#disable-default-theming
I would not suggest to do more than changing the default layout.
This page was added since it is sometimes difficult to say which version
was deployed on an server. This should allow an easy lookup on the
server and make it comparable with what is expected.
Added that the maintance action allso runs on changes of itself.
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: philipp-horstenkamp <philipp-horstenkamp@users.noreply.github.com>
The building and pushing of artifacts takes to long. With those changes
build experimental images aren't pushed anymore.
This saves build time and disc space.
- [x] Add a new table
- [x] Add a field to the table that can register if the company was
already queried
- [x] Add a field to the table that counts how many times a relation
partner was missing
- [x] Add a function that restets the counter
Also:
- Reworked the get_company function to use the location dict as kwargs
This adds the additional company data as proposed to the sql db.
- [x] @TrisNol Is everything included or did I miss a feature. Relations
are in another issue.
- [x] @KM-R New DB features for the Dashbord for your review.
- [x] add a cli to the webserver to take env variables into account
- [x] add a cli to the data processing that takes enviromental variable
as a valid source into account
- [x] rework the cli for the reset sql command
- [x] rework the cli for the copying of sql data from one db to another
The function is ment to transform the captial dict into a format that
can be added as a kwarg (**norm_capital(capital_dict) to the company
entities.
This PR only contains the function itself.
Since I committed to main again I added an pre-commit hook that prevents
that. Since pushing to main is disallowed this should prevent someone to
be stuck with a mess.
Filled out the mainter section.
Added a new description.
Added the path to the repo to the package.
Added the path to the documentation.
Co-authored-by: KM-R <129882581+KM-R@users.noreply.github.com>
Co-authored-by: Tristan Nolde <tristan.nolde@yahoo.de>
To learn the use of the Walrus opperator this hook integrates it in your
code if it finds a reason for it.
This branch is a suggestion of mine to force the use of the walrus
operaor as an autofix to keep it in mind when programming.
This should not increase the workload for us. It just lets us confront a
new programming tool. It also isn't a rull to be follwed. Just a hook
that looks at your code with an autofix.
- added a Dockerfile for the thre containers
- added a workflow step to build and placing the container in the
registry
- added a docker-compose.yaml to use the build images
- added a docker compose to build the images locally and a script for
prebuild steps
Fixed the following errors:
- Typo in readme.md
- Mongo delivers the last_update of an company as a string not as date.
Added:
- Entrypoint description in readme.md
Transfers data betwenn two sql instances.
Limited in data volume. Should be good enough for now.
---------
Co-authored-by: Tim <tim.ronneburg@outlook.de>
I added an sql lite session generator.
Changes
The function `def get_session() -> Session`
has changed to
`def get_session(connect_to: ConfigProvider | str) -> Session`
If a JsonFileConfig Is given the postgress conection is checked.
If a string is given that starts with `sqlite:///` an sql db is created.
The use should otherwise be the same.
Philipp
