Added a section on startup in the README.md (#507)

Reworked the index.md
- removed commented out text

Minor restructuring of the README.md

---------

Co-authored-by: Tristan Nolde <tristan.nolde@yahoo.de>
This commit is contained in:
2024-01-03 18:31:01 +01:00
committed by GitHub
parent 6e0247c58c
commit f645e2ec4b
5 changed files with 85 additions and 64 deletions

116
README.md
View File

@ -1,10 +1,10 @@
# aki_prj23_transparenzregister
# README.md of the aki_prj23_transparenzregister
## Contributions
See the [CONTRIBUTING.md](CONTRIBUTING.md) about how code should be formatted and what kind of rules we set ourselves.
## Available entrypoints
## Defined entrypoints
The project has currently the following entrypoint available:
@ -13,15 +13,82 @@ The project has currently the following entrypoint available:
- **reset-sql** > Resets all sql tables in the connected db.
- **copy-sql** > Copys the content of a db to another db.
- **webserver** > Starts the webserver showing the analysis results.
- **find-missing-companies** >
- **ingest** >
## DB Connection settings
All entrypoints support the `-h` argument and show a short help.
## Applikation startup
To connect to the SQL db see [sql/connector.py](./src/aki_prj23_transparenzregister/utils/sql/connector.py)
To connect to the Mongo db see [connect]
### Central Build
Create a `secrets.json` in the root of this repo with the following structure (values to be replaces by desired config):
The packages / container built by GitHub are accessible for users that are logged into the GitHub Container Registry (GHCR) with a Personal Access token via Docker.
Run `docker login ghcr.io` to start that process. [The complete docs on logging in can be found here.](https://docs.github.com/de/packages/working-with-a-github-packages-registry/working-with-the-container-registry)
The application can than be simply started with `docker compose up --pull`.
Please note that some configuration with a `.env` is necessary.
### Local Build
The application can be locally build by starting the `rebuild-and-start.bat`, if `poetry` and `docker-compose` is installed.
This will build a current `*.whl` and build the Docker container locally.
The configuration that start like this is the `local-docker-compose.yaml`.
Please note that some configuration with a `.env` is necessary.
## Application Settings
### Docker configuration / Environmental-Variables
The current design of this application suggests that it is started inside a `docker-compose` configuration.
For `docker-compose` this is commonly done by providing a `.env` file in the root folder.
To use the environmental configuration start an application part with the `ENV` argument (`webserver ENV`).
```dotenv
# Defines the container registry used. Default: "ghcr.io/fhswf/aki_prj23_transparenzregister"
CR=ghcr.io/fhswf/aki_prj23_transparenzregister
# main is the tag the main branch is taged with and is currently in use
TAG=latest
# Configures the access port for the webserver.
# Default: "80" (local only)
HTTP_PORT: 8888
# configures where the application root is based. Default: "/"
DASH_URL_BASE_PATHNAME=/transparenzregister/
# Enables basic auth for the application.
# Diabled when one is empty. Default: Disabled
PYTHON_DASH_LOGIN_USERNAME=some-login-to-webgui
PYTHON_DASH_LOGIN_PW=some-pw-to-login-to-webgui
# How often data should be ingested in houres, Default: "4"
PYTHON_INGEST_SCHEDULE=12
# Acces to the mongo db
PYTHON_MONGO_USERNAME=username
PYTHON_MONGO_HOST=mongodb
PYTHON_MONGO_PASSWORD=password
PYTHON_MONGO_PORT=27017
PYTHON_MONGO_DATABASE=transparenzregister
# Acces to the postress sql db
PYTHON_POSTGRES_USERNAME=username
PYTHON_POSTGRES_PASSWORD=password
PYTHON_POSTGRES_HOST=postgres-host
PYTHON_POSTGRES_DATABASE=db-name
PYTHON_POSTGRES_PORT=5432
# An overwrite path to an sqlite db, overwrites the POSTGRES section
PYTHON_SQLITE_PATH=PathToSQLite3.db
```
### Local execution / config file
Create a `*.json` in the root of this repo with the following structure
(values to be replaces by desired config):
Please note that an `sqlite` entry overwrites the `postgres` entry.
To use the `*.json` use the path to it as an argument when using an entrypoint (`webserver secrets.json`).
The sqlite db is alternative to the postgres section.
```json
{
"sqlite": "path-to-sqlite.db",
@ -42,35 +109,8 @@ The sqlite db is alternative to the postgres section.
}
```
Alternatively, the secrets can be provided as environment variables. One option to do so is to add a `.env` file with
the following layout:
### sqlite vs. postgres
```dotenv
PYTHON_POSTGRES_USERNAME=postgres
PYTHON_POSTGRES_PASSWORD=postgres
PYTHON_POSTGRES_HOST=postgres
PYTHON_POSTGRES_DATABASE=postgres
PYTHON_POSTGRES_PORT=5432
PYTHON_MONGO_USERNAME=username
PYTHON_MONGO_HOST=mongodb
PYTHON_MONGO_PASSWORD=password
PYTHON_MONGO_PORT=27017
PYTHON_MONGO_DATABASE=transparenzregister
# An overwrite path to an sqlite db
PYTHON_SQLITE_PATH=PathToSQLite3.db
PYTHON_DASH_LOGIN_USERNAME=some-login-to-webgui
PYTHON_DASH_LOGIN_PW=some-pw-to-login-to-webgui
# Every x hours
PYTHON_INGEST_SCHEDULE=12
CR=ghcr.io/fhswf/aki_prj23_transparenzregister
TAG=latest
HTTP_PORT=80
```
The prefix `PYTHON_` can be customized by setting a different `prefix` when constructing the ConfigProvider.
We support both sqlite and postgres because a local db is filled in about 10% of the time the remote db needs to be completed.
Even tough we use the `sqlite` for testing the connection can't manage multithreading or multiprocessing.
This clashes with the webserver. For production mode use the `postgres`-db.