Philipp/aki_prj23_transparenzregister
1
0
Fork 0
mirror of https://github.com/fhswf/aki_prj23_transparenzregister.git synced 2026-02-13 22:17:38 +01:00
Code Issues Packages Projects Releases Wiki Activity
402 Commits 7 Branches 0 Tags
d79bc0f26771563243a475f75cab9eee5ceb1f30
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
PhilippandGitHub d79bc0f267 Added poetry install to the pre-commit hooks. (#332) 
Limiting most pre-commit hooks to the pre-commit stage.
 2023-11-06 21:58:33 +01:00
PhilippandGitHub 4323522b96 Added poetry install to the pre-commit hooks. (#329) 2023-11-06 20:02:29 +01:00
PhilippandGitHub 2fddc9149a Reduced the number of times the dependabot auto merge is triggered. (#318) 2023-11-03 21:25:55 +01:00
PhilippandGitHub ad36c68993 Moved the AI tests into the AI folder. (#315) 2023-11-03 13:45:24 +01:00
PhilippandGitHub a638949644 Added a types package and upgraded poetry. (#313) 2023-11-02 22:13:32 +01:00
PhilippandGitHub d40746c69b Used black on jupyter. (#312) 2023-11-02 20:52:46 +00:00
PhilippandGitHub 8d9981d967 Moved AI files in the AI module. (#308) 2023-11-02 20:30:04 +01:00
PhilippandGitHub 389810c9bc Update and rename dependabot-approve.yaml to dependabot-auto-merge.yaml 2023-11-01 18:21:56 +01:00
PhilippandGitHub 3c2394d2d5 Sets auto merge for PRs opened by dependabot to true. (#293) 
Sets auto merge for PRs opened by dependabot to true.
Review is still needed.
 2023-10-30 19:20:54 +01:00
PhilippandGitHub 4362463f3c Added types-six to the linting packages and unlocked all the linter versions. (#286) 
Added the from mypy demanded types-six package as a lint dependency.
This should allow for a faster execution.
 2023-10-29 20:40:57 +01:00
PhilippandGitHub f72d606d18 Added base-path support in URL generating features (#288) 
Add the basepath dash url to the path generation for dynamicly generated
links.
 2023-10-29 20:40:40 +01:00
PhilippandGitHub 6526549bdd Reworked the about page for a nicer version of the git hash (#283) 
The git hash looks currently like this:

`Build from GiT Revision: ('d3189f78c47d812f447739f97876ced01227e1fd',)`

After this change it should look like this:
`Build from GiT Revision: d3189f78c47d812f447739f97876ced01227e1fd`
 2023-10-29 12:56:08 +01:00
PhilippandGitHub fca53c5e0c Dash security-fix against DoS. (#284) 
Dash has currently a security vulnerability.
This updated should fix this.
Dependabot is currently not allowed to upgrade main-version.
 2023-10-29 12:55:28 +01:00
PhilippandGitHub 0e388eb927 Using actor to identify dependabot and skipp some actions. (#273) 2023-10-26 19:29:25 +02:00
PhilippandGitHub e7a778e4cb Changed the plotly default theme (#269) 
I think the current blueish plotly default theme bits itself quit beadly
with @KM-R s great work.
I propose to set the deault plotly template to either:

- plotly_white
- ggplot2
- seaborn

Simply change the constant i have set will change this.
Heren you can find the catalog of themes:
https://plotly.com/python/templates/#disable-default-theming

I would not suggest to do more than changing the default layout.
 2023-10-26 19:22:44 +02:00
PhilippandGitHub 7953ba9291 Mixed typo fixes (#270) 2023-10-26 19:06:45 +02:00
PhilippandGitHub ab97d77d2d Made pip-audit pass on every build. (#272) 
Since dependabot is activated this can be reduced as an reporting tool.
 2023-10-26 16:50:24 +00:00
PhilippandGitHub 4c56431bff Update dependabot.yml 2023-10-26 18:19:20 +02:00
PhilippandGitHub 896136dcee Added an about page (#251) 
This page was added since it is sometimes difficult to say which version
was deployed on an server. This should allow an easy lookup on the
server and make it comparable with what is expected.
 2023-10-26 17:32:17 +02:00
PhilippandGitHub 6d77867eb5 Using the python action and the inbuild poetry cache (#261) 
I have found out that the python setup action supports the use of the
poetry cache which increases the build speed.
 2023-10-24 21:49:09 +02:00
PhilippandGitHub 1eb972b7ff Adds the transfer of sentiments into the sql db (#253) 
Transfers the sentimenes from the mongodb int the sql db.
 2023-10-24 17:50:40 +02:00
PhilippGitHubSchilder-Reiskephilipp-horstenkamp
 		b250b67343 Auto maintance push (#258) 
Added that the maintance action allso runs on changes of itself.

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: philipp-horstenkamp <philipp-horstenkamp@users.noreply.github.com>
 2023-10-23 17:50:51 +00:00
PhilippandGitHub 77a0faa257 Reduced build time (#260) 
The building and pushing of artifacts takes to long. With those changes
build experimental images aren't pushed anymore.
This saves build time and disc space.
 2023-10-23 19:43:53 +02:00
PhilippandGitHub f0d763cc5b Fixing the pre-commit upgrade (#256) 
The action allso executes over all files. This is ok when using
pre-conmit.ci. We only execute file fixes once.
 2023-10-23 17:15:39 +02:00
PhilippandGitHub ad7aa3e53b Filled Docker Labels and ENVs to identify the git revision it was build from (#250) 
Added labels and env vars to identify the git revion a container was
build from to have a more transparent build flow of untaged images.
 2023-10-21 18:13:03 +02:00
PhilippandGitHub 88f0673470 Slimmed down the env vars access and added logging in case the password protection was disabled. (#252) 
Slimmed down the env vars access and added logging in case the password
protection was disabled.
 2023-10-21 17:57:56 +02:00
PhilippandGitHub 55ebb4c17d Typo fixes (#249) 2023-10-21 10:58:54 +02:00
PhilippandGitHub 640faeccec Adding auth back in. (#248) 2023-10-20 14:56:54 +00:00
PhilippandGitHub 86daa9a781 Renamed a few jobs (#246) 2023-10-19 19:43:55 +02:00
PhilippandGitHub 2d80298aa2 Dependabot: Ignoring all major updates. (#241) 2023-10-19 19:24:58 +02:00
PhilippandGitHub 36a0bab6ff Add relations from finanical reports to SQL (#216) 2023-10-19 19:21:33 +02:00
PhilippandGitHub a358ebe9c1 Dependabot/coverage (#240) 2023-10-19 18:47:32 +02:00
PhilippandGitHub b92be60789 Automaticlly updating pre-commits (#220) 2023-10-19 18:44:13 +02:00
PhilippandGitHub 0441a3482b Create dependabot.yml (#227) 2023-10-19 18:05:36 +02:00
PhilippandGitHub fdba24ef11 Updates including security-updates (#226) 2023-10-19 17:59:15 +02:00
PhilippandGitHub c0d42a22d7 General updates & linting (#218) 2023-10-14 17:43:45 +02:00
PhilippandGitHub 6365e252b9 Added location to person (#185) 2023-10-14 15:27:19 +00:00
PhilippandGitHub 411f1053dd Set home title (#217) 2023-10-14 17:16:54 +02:00
PhilippandGitHub f8c111d7e2 Resolve mismatch between staging and prod db data for financials (#211) 
SQL Creation is now done dynamicly by the definition of the enumeration
type.
 2023-10-14 17:16:14 +02:00
PhilippandGitHub c8d3c7395b Update docker-compose.yml (#210) 2023-10-12 18:09:52 +02:00
PhilippandGitHub 68346aff15 Removed error suppression from sql data transformation. (#205) 2023-10-09 17:58:20 +02:00
PhilippandGitHub 63325e7faa Add constraints to the SQL entities (#186) 2023-10-06 18:48:58 +02:00
PhilippandGitHub ea9c777217 Enabeling data processing in docker container. (#195) 2023-10-06 18:36:29 +02:00
PhilippandGitHub b1ca268a62 SQL fixes after new mongo ingest (#199) 2023-10-06 18:22:19 +02:00
PhilippandGitHub 8bb27f5195 Set the DASH_URL_BASE_PATHNAME in the docker container to be used as a default (#192) 
Set the DASH_URL_BASE_PATHNAME in the docker container to be used as a
prefix.

Added a Dash Title
 2023-10-06 18:13:25 +02:00
Philipp ab96c5a01a Switched two docker container off. 2023-10-05 22:16:45 +02:00
PhilippandGitHub 09c36960e3 Add an list of missing relation partners to be searched (#171) 
- [x] Add a new table
- [x] Add a field to the table that can register if the company was
already queried
- [x] Add a field to the table that counts how many times a relation
partner was missing
- [x] Add a function that restets the counter

Also:
- Reworked the get_company function to use the location dict as kwargs
 2023-10-05 19:57:30 +02:00
PhilippandGitHub c6f2c7467c Rework the transfer of company data to fit the new data in the mongodb (#188) 
This adds the additional company data as proposed to the sql db.

- [x] @TrisNol Is everything included or did I miss a feature. Relations
are in another issue.
- [x] @KM-R New DB features for the Dashbord for your review.
 2023-10-05 19:47:46 +02:00
PhilippandGitHub ea89a64a80 Linter update to ruff==0.0.292 (#191) 2023-10-04 20:04:59 +02:00
PhilippandGitHub 41f2c9f995 Executing black over all jupyter notebook (#190) 
Reverting black for the jupyter notebooks gets old. Can we just run
black over all of them?
 2023-10-04 20:03:47 +02:00
PhilippandGitHub 030ad00c7d Testing speedup with in memory SQLite (#189) 
If no SQLite File is written and deleted testing is MUCH faster.
 2023-10-04 19:36:57 +02:00
PhilippGitHubKM-R
 		01b4ce00c1 Spellchecking with PyCharm (#133) 
Co-authored-by: KM-R <129882581+KM-R@users.noreply.github.com>
 2023-10-02 20:47:42 +02:00
PhilippandGitHub d2d4a436f8 Add a cli interface to choose a configuration (#163) 
- [x] add a cli to the webserver to take env variables into account 
- [x] add a cli to the data processing that takes enviromental variable
as a valid source into account
- [x] rework the cli for the reset sql command
- [x] rework the cli for the copying of sql data from one db to another
 2023-10-02 20:31:42 +02:00
PhilippandGitHub 2abe12f027 Add a function to convert DM to EUR (#168) 
The function is ment to transform the captial dict into a format that
can be added as a kwarg (**norm_capital(capital_dict) to the company
entities.
This PR only contains the function itself.
 2023-10-02 19:46:17 +02:00
PhilippandGitHub 05472cc16a Added longitude/latitude and positional accuracy to the company data (#180) 2023-10-02 17:18:04 +02:00
PhilippandGitHub c96462532b Added a pre-commit hook that disallows committing to main. (#181) 
Since I committed to main again I added an pre-commit hook that prevents
that. Since pushing to main is disallowed this should prevent someone to
be stuck with a mess.
 2023-10-01 21:40:39 +02:00
PhilippandGitHub b46a52e019 Added a weekly counter similar to all other weeklies to weekly 10. (#165) 2023-10-01 15:08:45 +02:00
PhilippandGitHub ebf30da778 Removed some superfluous tools from the docker image (#157) 2023-09-28 19:06:58 +02:00
PhilippandGitHub 4d99601fc6 Optional docs Feature: Changelog (#155) 
Added the ability to generate a changelog of the main branch in the
Documenation.
 2023-09-28 19:05:05 +02:00
PhilippGitHubKM-RTristan Nolde
 		552e1bdf96 Added some metadata to the package (#144) 
Filled out the mainter section.
Added a new description.
Added the path to the repo to the package.
Added the path to the documentation.

Co-authored-by: KM-R <129882581+KM-R@users.noreply.github.com>
Co-authored-by: Tristan Nolde <tristan.nolde@yahoo.de>
 2023-09-28 17:17:01 +02:00
PhilippandGitHub 7d6bb8b6f5 Optional: Add auto-walrus (#158) 
To learn the use of the Walrus opperator this hook integrates it in your
code if it finds a reason for it.
This branch is a suggestion of mine to force the use of the walrus
operaor as an autofix to keep it in mind when programming.
This should not increase the workload for us. It just lets us confront a
new programming tool. It also isn't a rull to be follwed. Just a hook
that looks at your code with an autofix.
 2023-09-28 17:13:42 +02:00
Philipp f54c140d98 Update after rebasing 2023-09-25 20:39:13 +02:00
Philipp 54dd574fb8 Added the ability to include drawio diagrams into the sphinx docs. (#107) 2023-09-25 20:37:59 +02:00
PhilippandGitHub 0fdfcc938a Added 2 Seminararbeiten to sphinx (#154) 
Added 2 Seminararbeiten to sphinx
 2023-09-25 20:30:43 +02:00
PhilippandGitHub 9cc0654f56 Delete untagged images (#151) 
This change to the pipline deletes untagged / overwritten tags once a
week to keep the memory consumption in the GH registry reasonable.
 2023-09-25 20:28:49 +02:00
PhilippandGitHub 5e716801c0 Change usage of black to the official black action (#152) 2023-09-25 20:26:02 +02:00
PhilippandGitHub 5bbdf046d2 Ruff fix after merge (#149) 2023-09-24 18:38:05 +02:00
PhilippandGitHub 091e67de79 build first set of docker container in pipline and place them in the the github registry (#142) 
- added a Dockerfile for the thre containers
- added a workflow step to build and placing the container in the
registry
- added a docker-compose.yaml to use the build images
- added a docker compose to build the images locally and a script for
prebuild steps
 2023-09-24 16:32:52 +00:00
PhilippandGitHub 820fb3e52b Repaired the SQL copy and reduced the log volume a bit (#141) 
- Added a cli interface to the SQL copy
- Repaired the SQL copy function
- Added the SQL copy function to the scripts
- Reduced the logging verbosity
 2023-09-24 15:11:49 +02:00
PhilippandGitHub d9ca808efa Mypy does not validate if type: ignore comments are needed. (#137) 2023-09-23 12:44:27 +02:00
PhilippandGitHub c5a383e903 Pipline / dependency updates (#145) 2023-09-23 10:49:14 +02:00
PhilippandGitHub 286636dfa4 Reduced the execution of tests and lint actions on pull requests (#143) 
This change will change the pipline to limit duplicat execution of tests
and lint actions in pull requests to save computing time.
 2023-09-23 09:22:14 +02:00
PhilippandGitHub 495cd90d85 Reduced required code coverage to 70% (#140) 2023-09-21 18:09:16 +02:00
PhilippandGitHub 80f077ee7a Version ups (#117) 
Rutine version ups for
- python-depdencies
- pre-commits
- piplines
 2023-09-17 13:50:53 +02:00
PhilippandGitHub 06fef5361d Lightend black in the pipline building (#128) 
Pipline should now run now 20 -40s faster.
 2023-09-17 13:47:20 +02:00
PhilippandGitHub 56b6280264 Transfer financials from staging to sql (#129) 2023-09-17 13:45:08 +02:00
PhilippandGitHub fea31e543b Bugfix update for transfer of company data from mongo to sql (#121) 
Fixed the following errors:

- Typo in readme.md
- Mongo delivers the last_update of an company as a string not as date.

Added:
- Entrypoint description in readme.md
 2023-09-13 19:14:36 +02:00
PhilippandGitHub 507647d164 Added custom 404 error code page in the sphinx docs (#115) 2023-09-13 19:01:09 +02:00
PhilippGitHubTim
 		ba44b082b8 Data transfer script (#114) 
Transfers data betwenn two sql instances.
Limited in data volume. Should be good enough for now.

---------

Co-authored-by: Tim <tim.ronneburg@outlook.de>
 2023-09-11 21:10:36 +02:00
PhilippandGitHub d64f53eca9 Script for the transfer of data from Mongo to SQL (#80) 2023-09-11 20:21:35 +02:00
PhilippandGitHub 2ea99c8c17 Removed unneded lines ☺ (#113) 2023-09-11 19:49:08 +02:00
PhilippandGitHub cee1ef901a Updatest vor everything (#106) 
- direct dependencies
- other depdenencies
- pre-commit
 2023-09-09 16:47:48 +00:00
PhilippandGitHub 23a193a4fb Added a small test ☺ (#108) 2023-09-09 18:35:50 +02:00
PhilippandGitHub 2233b12468 Add sql lite session (#71) 
I added an sql lite session generator.

Changes
The function `def get_session() -> Session` 
has changed to 
`def get_session(connect_to: ConfigProvider | str) -> Session`
If a JsonFileConfig Is given the postgress conection is checked.
If a string is given that starts with `sqlite:///` an sql db is created.
The use should otherwise be the same.
 2023-09-07 18:41:10 +02:00
Philipp ecf94d4b61 Added an import smoke test. 2023-09-06 19:08:44 +02:00
PhilippandGitHub d04255a7a8 Old date corrected in meeting notes fixed (#76) 
The date in the meeting nots was from the meeting two weeks before.
 2023-09-04 19:49:17 +02:00
PhilippandGitHub 17e89da0d8 Repo struktur dokumentieren (#75) 2023-09-04 19:48:14 +02:00
PhilippandGitHub f20823a613 Version updates for pre-commit/poetry/linter (#74) 
Just a rutine update to keep ruff up to date and sync.
 2023-09-03 17:56:13 +02:00
PhilippandGitHub 011184b898 Small fixes to the documentation. (#70) 2023-08-31 16:57:50 +02:00
PhilippandGitHub 69039706ac Version updates (#69) 
Version ups for poetry and pre-commit to keep them in sync with the
pipline.
 2023-08-31 16:57:09 +02:00
PhilippandGitHub 06e59b8061 Changes to the database (#67) 
In this PR are pruposed changed that come up when i started the data
migration.
Plese review them an approve and disaprove.
 2023-08-24 17:23:19 +02:00
Philipp 82979b1981 Expanded a bit on the installation of the project. 2023-08-20 13:03:06 +02:00
PhilippandGitHub e4a57c9136 Chore/rework workflow (#52) 
* Reworked the lint action
* Removed the file change requirement
* Repaired mypy
* Repaired pip-audit
 2023-08-17 22:05:12 +02:00
Philipp faca0a0af6 Repaired a mypy option 2023-08-16 22:10:52 +02:00
PhilippandGitHub 9c237402be Excluded ipynb files from json pretty formatter + poetry update (#50) 
* Excluded ipynb files from json pretty formatter +  poetry update
* Reworked pip audit
* Relocated dependencies
* Reworked the tests a bit.
* Poetry update
* Excluded lint and test workflows against the target branch
 2023-08-16 20:25:15 +02:00
Philipp 67c94cfb7e Changes sql-alchemy to version 1.4.49 2023-08-13 15:00:29 +02:00
PhilippandGitHub 50bf7811ef Div. dev ops updates (#43) 
* Pipline rework to limit mypy and balck to src and tests
* Poetry update
* pre-commit update
 2023-08-10 19:20:12 +02:00
PhilippandGitHub f9b1f1e5e4 Ruff update and lint version definition (#36) 2023-07-11 14:23:52 +02:00
PhilippandGitHub aea4b450c8 Pre commit update (#35) 2023-07-11 14:23:15 +02:00
Philipp 1c621f46a7 Added selenium to poetry 2023-07-11 01:59:11 +02:00