Philipp Horstenkamp daffcadd11
Seminararbeit DevOps & CI/CD (#486)
Co-authored-by: Sebastian <94404394+SeZett@users.noreply.github.com>
Co-authored-by: Tristan Nolde <tristan.nolde@yahoo.de>
2024-01-01 14:26:05 +01:00

2.2 KiB

aki_prj23_transparenzregister

Contributions

See the CONTRIBUTING.md about how code should be formatted and what kind of rules we set ourselves.

Available entrypoints

The project has currently the following entrypoint available:

  • data-transformation > Transfers all the data from the mongodb into the sql db to make it available as production data.
  • data-processing > Processes the data using NLP methods and transfers matched data into the SQL table ready for use.
  • reset-sql > Resets all sql tables in the connected db.
  • copy-sql > Copys the content of a db to another db.
  • webserver > Starts the webserver showing the analysis results.

DB Connection settings

To connect to the SQL db see sql/connector.py To connect to the Mongo db see [connect]

Create a secrets.json in the root of this repo with the following structure (values to be replaces by desired config):

The sqlite db is alternative to the postgres section.

{
  "sqlite": "path-to-sqlite.db",
  "postgres": {               
    "username": "username",      
    "password": "password",
    "host": "localhost",
    "database": "db-name",
    "port": 5432
  },
  "mongo": {
    "username": "username",
    "password": "password",
    "host": "localhost",
    "database": "transparenzregister",
    "port": 27017
  }
}

Alternatively, the secrets can be provided as environment variables. One option to do so is to add a .env file with the following layout:

PYTHON_POSTGRES_USERNAME=postgres
PYTHON_POSTGRES_PASSWORD=postgres
PYTHON_POSTGRES_HOST=postgres
PYTHON_POSTGRES_DATABASE=postgres
PYTHON_POSTGRES_PORT=5432

PYTHON_MONGO_USERNAME=username
PYTHON_MONGO_HOST=mongodb
PYTHON_MONGO_PASSWORD=password
PYTHON_MONGO_PORT=27017
PYTHON_MONGO_DATABASE=transparenzregister

# An overwrite path to an sqlite db
PYTHON_SQLITE_PATH=PathToSQLite3.db

PYTHON_DASH_LOGIN_USERNAME=some-login-to-webgui
PYTHON_DASH_LOGIN_PW=some-pw-to-login-to-webgui

# Every x hours
PYTHON_INGEST_SCHEDULE=12

CR=ghcr.io/fhswf/aki_prj23_transparenzregister
TAG=latest

HTTP_PORT=80

The prefix PYTHON_ can be customized by setting a different prefix when constructing the ConfigProvider.