Added a section on startup in the README.md (#507)

Reworked the index.md
- removed commented out text

Minor restructuring of the README.md

---------

Co-authored-by: Tristan Nolde <tristan.nolde@yahoo.de>

CONTRIBUTING.md

@@ -1,4 +1,4 @@
-# Contribution guidelines
+# CONTRIBUTING.md: Contribution guidelines
 
 ## Dev Setup
 

README.md

@@ -1,10 +1,10 @@
-# aki_prj23_transparenzregister
+# README.md of the aki_prj23_transparenzregister 
 
 ## Contributions
 
 See the [CONTRIBUTING.md](CONTRIBUTING.md) about how code should be formatted and what kind of rules we set ourselves.
 
-## Available entrypoints
+## Defined entrypoints
 
 The project has currently the following entrypoint available:
 
@@ -13,15 +13,82 @@ The project has currently the following entrypoint available:
 - **reset-sql** > Resets all sql tables in the connected db.
 - **copy-sql** > Copys the content of a db to another db.
 - **webserver** > Starts the webserver showing the analysis results.
+- **find-missing-companies** >
+- **ingest** >
 
-## DB Connection settings
+All entrypoints support the `-h` argument and show a short help.
+  
+## Applikation startup
 
-To connect to the SQL db see [sql/connector.py](./src/aki_prj23_transparenzregister/utils/sql/connector.py)
+### Central Build
-To connect to the Mongo db see [connect]
 
-Create a `secrets.json` in the root of this repo with the following structure (values to be replaces by desired config):
+The packages / container built by GitHub are accessible for users that are logged into the GitHub Container Registry (GHCR) with a Personal Access token via Docker.
+Run `docker login ghcr.io` to start that process. [The complete docs on logging in can be found here.](https://docs.github.com/de/packages/working-with-a-github-packages-registry/working-with-the-container-registry) 
+The application can than be simply started with `docker compose up --pull`.
+Please note that some configuration with a `.env` is necessary.
+
+### Local Build
+
+The application can be locally build by starting the `rebuild-and-start.bat`, if `poetry` and `docker-compose` is installed.
+This will build a current `*.whl` and build the Docker container locally.
+The configuration that start like this is the `local-docker-compose.yaml`.
+Please note that some configuration with a `.env` is necessary.
+
+## Application Settings
+
+### Docker configuration / Environmental-Variables
+The current design of this application suggests that it is started inside a `docker-compose` configuration.
+For `docker-compose` this is commonly done by providing a `.env` file in the root folder.
+
+To use the environmental configuration start an application part with the `ENV` argument (`webserver ENV`). 
+
+```dotenv
+# Defines the container registry used. Default: "ghcr.io/fhswf/aki_prj23_transparenzregister"
+CR=ghcr.io/fhswf/aki_prj23_transparenzregister
+
+# main is the tag the main branch is taged with and is currently in use
+TAG=latest
+
+# Configures the access port for the webserver. 
+# Default: "80" (local only)
+HTTP_PORT: 8888
+
+# configures where the application root is based. Default: "/"
+DASH_URL_BASE_PATHNAME=/transparenzregister/
+
+# Enables basic auth for the application. 
+# Diabled when one is empty. Default: Disabled
+PYTHON_DASH_LOGIN_USERNAME=some-login-to-webgui
+PYTHON_DASH_LOGIN_PW=some-pw-to-login-to-webgui
+
+# How often data should be ingested in houres, Default: "4"
+PYTHON_INGEST_SCHEDULE=12
+
+# Acces to the mongo db
+PYTHON_MONGO_USERNAME=username
+PYTHON_MONGO_HOST=mongodb
+PYTHON_MONGO_PASSWORD=password
+PYTHON_MONGO_PORT=27017
+PYTHON_MONGO_DATABASE=transparenzregister
+
+# Acces to the postress sql db
+PYTHON_POSTGRES_USERNAME=username
+PYTHON_POSTGRES_PASSWORD=password
+PYTHON_POSTGRES_HOST=postgres-host
+PYTHON_POSTGRES_DATABASE=db-name
+PYTHON_POSTGRES_PORT=5432
+
+# An overwrite path to an sqlite db, overwrites the POSTGRES section
+PYTHON_SQLITE_PATH=PathToSQLite3.db
+```
+
+### Local execution / config file
+
+Create a `*.json` in the root of this repo with the following structure 
+    (values to be replaces by desired config):
+Please note that an `sqlite` entry overwrites the `postgres` entry.
+To use the `*.json` use the path to it as an argument when using an entrypoint (`webserver secrets.json`).
 
-The sqlite db is alternative to the postgres section.
 ```json
 {
   "sqlite": "path-to-sqlite.db",
@@ -42,35 +109,8 @@ The sqlite db is alternative to the postgres section.
 }
 ```
 
-Alternatively, the secrets can be provided as environment variables. One option to do so is to add a `.env` file with
+### sqlite vs. postgres
-the following layout:
 
-```dotenv
+We support both sqlite and postgres because a local db is filled in about 10% of the time the remote db needs to be completed.
-PYTHON_POSTGRES_USERNAME=postgres
+Even tough we use the `sqlite` for testing the connection can't manage multithreading or multiprocessing.
-PYTHON_POSTGRES_PASSWORD=postgres
+This clashes with the webserver. For production mode use the `postgres`-db.
-PYTHON_POSTGRES_HOST=postgres
-PYTHON_POSTGRES_DATABASE=postgres
-PYTHON_POSTGRES_PORT=5432
-
-PYTHON_MONGO_USERNAME=username
-PYTHON_MONGO_HOST=mongodb
-PYTHON_MONGO_PASSWORD=password
-PYTHON_MONGO_PORT=27017
-PYTHON_MONGO_DATABASE=transparenzregister
-
-# An overwrite path to an sqlite db
-PYTHON_SQLITE_PATH=PathToSQLite3.db
-
-PYTHON_DASH_LOGIN_USERNAME=some-login-to-webgui
-PYTHON_DASH_LOGIN_PW=some-pw-to-login-to-webgui
-
-# Every x hours
-PYTHON_INGEST_SCHEDULE=12
-
-CR=ghcr.io/fhswf/aki_prj23_transparenzregister
-TAG=latest
-
-HTTP_PORT=80
-```
-
-The prefix `PYTHON_` can be customized by setting a different `prefix` when constructing the ConfigProvider.

docker-compose.yml

@@ -67,7 +67,7 @@ services:
     depends_on:
     - postgres
     ports:
-    - ${HTTP_PORT}:${HTTP_PORT}
+    - ${HTTP_PORT:-error}:${HTTP_PORT:-error}
     environment:
       PYTHON_POSTGRES_USERNAME: ${PYTHON_POSTGRES_USERNAME:-postgres}
       PYTHON_POSTGRES_PASSWORD: ${PYTHON_POSTGRES_PASSWORD:?error}
@@ -76,6 +76,6 @@ services:
       PYTHON_POSTGRES_PORT: ${PYTHON_POSTGRES_PORT:-5432}
       PYTHON_DASH_LOGIN_PW: ${PYTHON_DASH_LOGIN_PW}
       PYTHON_DASH_LOGIN_USERNAME: ${PYTHON_DASH_LOGIN_USERNAME}
-      DASH_URL_BASE_PATHNAME: /transparenzregister/
+      DASH_URL_BASE_PATHNAME: ${DASH_URL_BASE_PATHNAME:-/transparenzregister/}
-      PORT: ${HTTP_PORT}
+      PORT: ${HTTP_PORT:-error}
       HOST: 0.0.0.0

documentations/index.rst

@@ -7,26 +7,6 @@ Das Projekt startete im Sommersemester 2023 und dauerte ein Jahr.
 Die Dokumentation ist zwar auf Deutsch, wir steigen aber ein mit der `README.md` und der `CONRIBUTING.md`.
 Diese sind, um Industriestandards zu entsprechen, auf Englisch gehalten.
 
-.. On the 9th September 2023 the following current development state has been determined:
-
-.. .. drawio-figure:: project_management/Arch.-Planning.drawio
-   :format: png
-   :page-index: 1
-
-.. The target deployment architecture should look as follows:
-
-.. .. drawio-figure:: project_management/Arch.-Planning.drawio
-   :format: png
-   :page-index: 2
-
-.. The timeline to reach this goal is:
-
-.. .. drawio-figure:: project_management/Arch.-Planning.drawio
-   :format: png
-   :page-index: 3
-
-..    An example diagram
-
 .. include:: ../README.md
     :parser: myst_parser.sphinx_
 .. include:: ../CONTRIBUTING.md

local-docker-compose.yml

@@ -94,7 +94,7 @@ services:
     depends_on:
     - postgres
     ports:
-    - 8888:8888
+    - ${HTTP_PORT:-8888}:${HTTP_PORT:-8888}
     environment:
       PYTHON_POSTGRES_USERNAME: ${PYTHON_POSTGRES_USERNAME:-postgres}
       PYTHON_POSTGRES_PASSWORD: ${PYTHON_POSTGRES_PASSWORD:?error}
@@ -103,7 +103,8 @@ services:
       PYTHON_POSTGRES_PORT: ${PYTHON_POSTGRES_PORT:-5432}
       PYTHON_DASH_LOGIN_PW: ${PYTHON_DASH_LOGIN_PW}
       PYTHON_DASH_LOGIN_USERNAME: ${PYTHON_DASH_LOGIN_USERNAME}
-      PORT: 8888
+      PORT: ${HTTP_PORT:-8888}
+
       HOST: 0.0.0.0
 
 volumes: