{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Deep Otello AI\n",
    "\n",
    "The game reversi is a very good game to apply deep learning methods to.\n",
    "\n",
    "Othello also known as reversi is a board game first published in 1883 by eiter Lewis Waterman or John W. Mollet in England (each one was denouncing the other as fraud).\n",
    "It is a strickt turn based zero-sum game with a clear Markov chain and now hidden states like in card games with an unknown distribution of cards or unknown player allegiance.\n",
    "There is like for the game go only one set of stones with two colors which is much easier to abstract than chess with its 6 unique pieces.\n",
    "The game has a symmetrical game board wich allows to play with rotating the state around an axis to allow for a breaking of sequences or interesting ANN architectures, quadruple the data generation by simulation or interesting test cases where a symetry in turns should be observable if the AI reaches an \"objective\" policy."
   ]
  },
  {
   "cell_type": "markdown",
   "source": [
    "\n",
    "## Content\n",
    "\n",
    "* [The game rules](#the-game-rules) A short overview over the rules of the game.\n",
    "* [Some common Otello strategies](#some-common-otello-strategies) introduces some easy approaches to a classic Otello AI and defines some behavioral expectations.\n",
    "* [Initial design decisions](#initial-design-decisions) an explanation about some initial design decision and assumptions\n",
    "* [Imports and dependencies](#imports-and-dependencies) explains what libraries where used"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "\n",
    "## The game rules\n",
    "\n",
    "Othello is played on a board with 8 x 8 fields for two player.\n",
    "The board geometry is equal to a chess game.\n",
    "The game is played with game stones that are black on one siede and white on the other.\n",
    "![Othello game board example](reversi_example.png)\n",
    "The player take turns.\n",
    "A player places a stone with his or her color up on the game board.\n",
    "The player can only place stones when he surrounds a number of stones with the opponents color with the new stone and already placed stones of his color.\n",
    "Those surrounded stones can either be horizontally, vertically and/or diagonally be placed.\n",
    "All stones thus surrounded will be flipped to be of the players color.\n",
    "Turns are only possible if the player is also changing the color of the opponents stones. If a player can't act he is skipped.\n",
    "The game ends if both players can't act. The player with the most stones wins.\n",
    "If the score is counted in detail unclaimed fields go to the player with more stones of his or her color on the board.\n",
    "The game begins with four stones places in the center of the game. Each player gets two. They are placed diagonally to each other.\n",
    "\n",
    "\n",
    "<img alt=\"Startaufstellung.png\" src=\"Startaufstellung.png\"/>\n",
    "\n",
    "## Some common Othello strategies\n",
    "\n",
    "As can be easily understood the placement of stones and on the bord is always a careful balance of attack and defence.\n",
    "If the player occupies huge homogenous stretches on the board it can be attacked easier.\n",
    "The boards corners provide safety from wich occupied territory is impossible to loos but since it is only possible to reach the corners if the enemy is forced to allow this or calculates the cost of giving a stable base to the enemy it is difficult to obtain.\n",
    "There are some text on otello computer strategies which implement greedy algorithms for reversi based on a modified score to each field.\n",
    "Those different values are score modifiers for a traditional greedy algorithm.\n",
    "If a players stone has captured such a filed the score reached is multiplied by the modifier.\n",
    "The total score is the score reached by the player subtracted with the score of the enemy.\n",
    "The scores change in the course of the game and converges against one. This gives some indications of what to expect from an Othello AI.\n",
    "\n",
    "<img alt=\"ComputerPossitionScore\" src=\"computer-score.png\"/>\n"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Initial design decisions\n",
    "\n",
    "At the beginning of this project I made some design decisions.\n",
    "The first onw was that I do not want to use a gym library because it limits the data formats accessible.\n",
    "I choose to implement the hole game as entry in a stack in numpy arrays to be able to accommodate interfacing with a neural network easier and to use scipy pattern recognition tools to implement some game mechanics for a fast simulation cycle.\n",
    "I chose to ignore player colors as far as I could instead a player perspective was used. Which allowed to change the perspective with a flipping of the sign. (multiplying with -1).\n",
    "The array format should also allow for data multiplication or the breaking of strikt sequences by flipping the game along one the for axis, (horizontal, vertical, transpose along both diagonals).\n",
    "\n",
    "I wanted to implement different agents as classes that act on those game stacks.\n",
    "\n",
    "Since computation time is critical all computational have results are saved.\n",
    "The analysis of those is then repeated in real time. If a recalculation of such a section is required the save file can be deleted and the code should be executed again."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from multiprocessing import Pool\n",
    "\n",
    "%load_ext blackcellmagic"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Imports and dependencies\n",
    "\n",
    "The following direct dependencies where used for this project:\n",
    "```toml\n",
    "jupyter = \"^1.0.0\"\n",
    "matplotlib = \"^3.6.3\"\n",
    "numpy = \"^1.24.1\"\n",
    "pytest = \"^7.2.1\"\n",
    "python = \"3.10.*\"\n",
    "scipy = \"^1.10.0\"\n",
    "tqdm = \"^4.64.1\"\n",
    "jupyterlab = \"^3.6.1\"\n",
    "torchvision = \"^0.14.1\"\n",
    "torchaudio = \"^0.13.1\"\n",
    "```\n",
    "* `Jupyter` and `jupyterlab` on pycharm was used as a IDE / Ipython was used to implement this code.\n",
    "* `matplotlib` was used for visualisation and statistics.\n",
    "* `numpy` was used for array support and mathematical functions\n",
    "* `tqdm` was used for progress bars\n",
    "* `scipy` contains fast pattern recognition tools for images. It was used to make an initial estimation about where possible turns should be.\n",
    "* `torch` supplied the ANN functionalities."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import itertools\n",
    "import numpy as np\n",
    "import abc\n",
    "from typing import Final\n",
    "from scipy.ndimage import binary_dilation\n",
    "import matplotlib.pyplot as plt\n",
    "from abc import ABC"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Constants\n",
    "\n",
    "Some general constants needed to be defined. Such as board game size and Player and Enemy representations. Also, directional offsets and the initial placement of blocks."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "BOARD_SIZE: Final[int] = 8  # defines the board side length as 8\n",
    "PLAYER: Final[int] = 1  # defines the number symbolising the player as 1\n",
    "ENEMY: Final[int] = -1  # defines the number symbolising the enemy as -1\n",
    "EXAMPLE_STACK_SIZE: Final[int] = 1000  # defines the game stack size for examples\n",
    "IMPOSSIBLE: Final[np.ndarray] = np.array([-1, -1], dtype=int)\n",
    "IMPOSSIBLE.setflags(write=False)\n",
    "SIMULATE_TURNS: Final[int] = 70\n",
    "VERIFY_POLICY: Final[bool] = True"
   ]
  },
  {
   "cell_type": "markdown",
   "source": [
    "The directions array contains all the numerical offsets needed to move along one of the 8 directions in a 2 dimensional grid. This will allow an iteration over the game board.\n",
    "![8-directions.png](8-directions.png \"Offset in 8 directions\")"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": "array([[-1, -1],\n       [-1,  0],\n       [-1,  1],\n       [ 0, -1],\n       [ 0,  1],\n       [ 1, -1],\n       [ 1,  0],\n       [ 1,  1]])"
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DIRECTIONS: Final[np.ndarray] = np.array(\n",
    "    [[i, j] for i in range(-1, 2) for j in range(-1, 2) if j != 0 or i != 0],\n",
    "    dtype=int,\n",
    ")\n",
    "DIRECTIONS.setflags(write=False)\n",
    "DIRECTIONS"
   ]
  },
  {
   "cell_type": "markdown",
   "source": [
    "Another constant needed is the initial start square at the center of the board."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "outputs": [
    {
     "data": {
      "text/plain": "array([[-1,  1],\n       [ 1, -1]])"
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "START_SQUARE: Final[np.ndarray] = np.array(\n",
    "    [[ENEMY, PLAYER], [PLAYER, ENEMY]], dtype=int\n",
    ")\n",
    "START_SQUARE.setflags(write=False)\n",
    "START_SQUARE"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating new boards\n",
    "\n",
    "The first function implemented and tested is a function to generate the starting environment as a stack of games.\n",
    "As described above I simply placed a 2 by 2 square in the center of an empty stack of boards."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": "array([[ 0,  0,  0,  0,  0,  0,  0,  0],\n       [ 0,  0,  0,  0,  0,  0,  0,  0],\n       [ 0,  0,  0,  0,  0,  0,  0,  0],\n       [ 0,  0,  0, -1,  1,  0,  0,  0],\n       [ 0,  0,  0,  1, -1,  0,  0,  0],\n       [ 0,  0,  0,  0,  0,  0,  0,  0],\n       [ 0,  0,  0,  0,  0,  0,  0,  0],\n       [ 0,  0,  0,  0,  0,  0,  0,  0]])"
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def get_new_games(number_of_games: int) -> np.ndarray:\n",
    "    \"\"\"Generates a stack of initialised game boards.\n",
    "\n",
    "    Args:\n",
    "        number_of_games: The size of the board stack.\n",
    "\n",
    "    Returns: The generates stack of games as a stack n x 8 x 8.\n",
    "\n",
    "    \"\"\"\n",
    "    empty = np.zeros([number_of_games, BOARD_SIZE, BOARD_SIZE], dtype=int)\n",
    "    empty[:, 3:5, 3:5] = START_SQUARE\n",
    "    return empty\n",
    "\n",
    "\n",
    "get_new_games(1)[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_number_of_games = 3\n",
    "assert get_new_games(test_number_of_games).shape == (\n",
    "    test_number_of_games,\n",
    "    BOARD_SIZE,\n",
    "    BOARD_SIZE,\n",
    ")\n",
    "np.testing.assert_equal(\n",
    "    get_new_games(test_number_of_games).sum(axis=1),\n",
    "    np.zeros(\n",
    "        [\n",
    "            test_number_of_games,\n",
    "            8,\n",
    "        ]\n",
    "    ),\n",
    ")\n",
    "np.testing.assert_equal(\n",
    "    get_new_games(test_number_of_games).sum(axis=2),\n",
    "    np.zeros(\n",
    "        [\n",
    "            test_number_of_games,\n",
    "            8,\n",
    "        ]\n",
    "    ),\n",
    ")\n",
    "assert np.all(get_new_games(test_number_of_games)[:, 3:4, 3:4] != 0)\n",
    "del test_number_of_games"
   ]
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Visualisation tools\n",
    "\n",
    "In this section a visualisation help was implemented for debugging of the game and a proper display of the results.\n",
    "For this visualisation ChatGPT was used as a prompted code generator that was later reviewed and refactored by hand to integrate seamlessly into the project as a whole.\n",
    "White stones represent the player, black stones the enemy. A single plot can be used as a subplot when the `ax` argument is used."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": "<Figure size 300x300 with 1 Axes>",
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAASIAAAEiCAYAAABdvt+2AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/P9b71AAAACXBIWXMAAA9hAAAPYQGoP6dpAAAdq0lEQVR4nO3de3BU5f0G8OckG1dIsivEYFizQEIsMcHwA0RLMyhBQQKh0HFocUIBRcslAuq0lWBtaRUWx7ZDBQwUwqVDuNkR6jByEeRSUS5BodyChktZSCKMhV0S6prsnt8fx90mkE32bPacN8l5PjNndHfP2e/7kvDw7tnzvkeSZVkGEZFAUaIbQETEICIi4RhERCQcg4iIhGMQEZFwDCIiEo5BRETCMYiISDiT3gV9Ph8qKioQHx8PSZL0Lk9EOpFlGTdv3oTNZkNUVNNjHt2DqKKiAna7Xe+yRCSI0+lEcnJyk/voHkTx8fGB/+/YVd/at6oAyAAkoGOScWqLrs++i6ktuv6tSuW/9f/OB6N7EPk/jnXsCoyv0Ld2STJQcwWItQH5l41TW3R99t2YfV9rU8IolFMwPFlNRMIxiIhIOAYREQnHICIi4RhERCQcg4iIhGMQEZFwDCIiEk51EO3fvx+jRo2CzWaDJEnYsmWLBs0iIiNRHUQ1NTXo06cPlixZokV7iMiAVE/xyM3NRW5urhZtISKD0nyumcfjgcfjCTx2u91alySiNkbzk9UOhwNWqzWwcQkQIrqd5kFUWFgIl8sV2JxOp9YliaiN0fyjmdlshtls1roMEbVhvI6IiIRTPSKqrq5GeXl54PGFCxdw7NgxdO7cGd26dYto44jIGFQHUWlpKXJycgKPX3nlFQDAxIkTsXr16og1jIiMQ3UQDR48GLIsa9EWIjIoniMiIuEYREQkHIOIiIRjEBGRcAwiIhKOQUREwjGIiEg4BhERCSfJOl+d6Ha7YbVaAUm5H7eeblUCsg+QooCOXY1TW3R99t2Yfa+pACADLpcLFoulyX3FBRERGUIoQaT5MiBBcURkmPrsuzH77h8RhUJYEHVMAvIv61uzJBmouaL8QIxUW3R99t2YfV9rU4IwFDxZTUTCMYiISDgGEREJxyAiIuEYREQkHIOIiIRjEBGRcAwiIhJOVRA5HA4MGDAA8fHx6NKlC8aMGYOzZ89q1TYiMghVQbRv3z4UFBTg4MGD+Oijj1BbW4thw4ahpqZGq/YRkQGomuKxffv2Bo9Xr16NLl264OjRo3jsscci2jAiMo4WzTVzuVwAgM6dOwfdx+PxwOPxBB673e6WlCSidijsk9U+nw8vvfQSsrOz0bt376D7ORwOWK3WwGa328MtSUTtVNhBVFBQgJMnT2LDhg1N7ldYWAiXyxXYnE5nuCWJqJ0K66PZiy++iK1bt2L//v1ITk5ucl+z2Qyz2RxW44jIGFQFkSzLmDFjBjZv3oy9e/ciJSVFq3YRkYGoCqKCggKsW7cO//jHPxAfH4+qqioAgNVqRYcOHTRpIBG1f6rOERUVFcHlcmHw4MHo2rVrYNu4caNW7SMiA1D90YyIKNI414yIhGMQEZFwDCIiEo5BRETCMYiISDgGEREJxyAiIuEYREQknCTrfJWi2+2G1WoFJCDWpmdl5T7csg+QopR7gRultuj67Lsx+15TAUBW1i2zWCxN7isuiIjIEEIJohat0NgiHBEZpj77bsy++0dEoRAWRB2TgPzL+tYsSQZqrig/ECPVFl2ffTdm39falCAMBU9WE5FwDCIiEo5BRETCMYiISDgGEREJxyAiIuEYREQknOrF87OysmCxWGCxWDBw4EBs27ZNq7YRkUGoCqLk5GQsWLAAR48eRWlpKYYMGYLRo0fj1KlTWrWPiAxA1ZXVo0aNavB43rx5KCoqwsGDB5GZmRnRhhGRcYQ9xcPr9eK9995DTU0NBg4cGMk2EZHBqA6iEydOYODAgfj2228RFxeHzZs3IyMjI+j+Ho8HHo8n8NjtdofXUiJqt1R/a9arVy8cO3YMhw4dwrRp0zBx4kScPn066P4OhwNWqzWw2e32FjWYiNof1UF01113IS0tDf3794fD4UCfPn3wl7/8Jej+hYWFcLlcgc3pdLaowUTU/rR4GRCfz9fgo9ftzGYzzGZzS8sQUTumKogKCwuRm5uLbt264ebNm1i3bh327t2LHTt2aNU+IjIAVUF09epVTJgwAZWVlbBarcjKysKOHTswdOhQrdpHRAagKoiKi4u1agcRGRjnmhGRcAwiIhKOQUREwjGIiEg4BhERCccgIiLhGEREJByDiIiEk2RZDvHu1JHhdrthtVoBCYi16VmZ90Bn39l3PdVUAJABl8sFi8XS5L7igoiIDCGUIGrx7PuwcURkmPrsuzH77h8RhUJYEHVMAvIv61uzJBmouaL8QIxUW3R99t2YfV9rU4IwFOJGRNRmmBGLRKTBBDPq4ME1lMODGl1q11YDrnLA5wGizIA1DYiJ06U06YhBRI3qigfxGKaiN0YgEamQ6n3BKsOHaziPk/gQ+7EUlTgT0drXTwOnlwLODwH3eTQc3kuAJRWwjwAypgKdgi+XTm0Ig4gaSEAP5GMZMjEMXtQiGjF37CMhCl2QhscxDUMwE6ewEyWYgm9wsUW13ReAf04BrnwESCZArmtkJxlwnwNOFwGnFgH3DwUGLQMsKS0qTYLxOiIKyMZkzMVppCMHABoNofr8r6cjB3NxCtmYHHbtshXAexlAxR7lcaMhVI//9Yo9ynFlK8IuTa0Ag4gAALmYgwlYgRjc3WwA3S4aMYhBB0zACuRijuran88D9r8AeL9tPoBuJ9cpx+1/QXkfapsYRIRsTMYYKH+LJUhhvYf/uDGYh2w8F/JxZSuA0t+EVfIOpb8ByriIaJvEIDK4BPTAOCyCHOoFH82QIWMcFiEBPZrd130BODAjImUDDryovC+1LQwig8vHMkTDFPZI6HYSJEQjBvlY1uy+/5wC+FR+FGuOr055X2pbWhRECxYsgCRJeOmllyLUHNJTVzyITAxTfU6oOdGIQSaGIQnpQfe5flr5dkztOaHmyHXK+16P7BUFpLGwg+jIkSNYtmwZsrKyItke0tFjmAovajV5by9q8TimBX399FLlK3otSCbl631qO8IKourqauTn52P58uXo1KlTpNtEOumNEREfDflFIwa9kRv0deeHkR8N+cl1gHObNu9N2ggriAoKCjBy5Eg8+eSTkW4P6cSMOCQiVdMaiegJM2LveP67m99fMa0h9zllegi1DaoHxxs2bMDnn3+OI0eOhLS/x+OBx+MJPHa73WpLkgYS0bPBtA0tSIhCItIAHG/wvPscQp6VHTZZmaN27/9pXIciQtVvotPpxKxZs1BSUoK77747pGMcDgesVmtgs9vtYTWUIssEs7A6Pk8jO2pArzrUcqqC6OjRo7h69Sr69esHk8kEk8mEffv24Z133oHJZILX673jmMLCQrhcrsDmdDoj1ngKXx30+VvaWJ0ofTJQtzrUcqo+mj3xxBM4ceJEg+eeffZZpKen49VXX0V0dPQdx5jNZpjN/I1oba6hHDJ8mn48U2bpl9/xvDUNgARtP55J39ehNkFVEMXHx6N3794NnouNjUVCQsIdz1Pr5kENruE8ukC7v63XcK7RdYti4pSlPNznNCsNS0+uW9SW8MpqAzuJDzW9jugkgn+Hbh+h7XVE9uBXDlAr1OJfhb1790agGSTCfizFEMzU5L2jEYN9CH5VYcZUZT0hLch1QEbwaympFeKIyMAqcQansDPioyIvanEKO1GFsqD7dMpQFjWL9KhIMinv2+nByL4vaYtBZHAlmAIvaiM6+96LWpSg+Zmng5YBUREOoiiT8r7UtjCIDO4bXMQGzIzo7PsNmBHSsrGWFCA7wh/Pshdz2di2iEFEOIBibMFrABD2yMh/3BbMwQGsDPm49OeBh98Mq+QdBswD0sNfrZYE4uL5BADYhvlw42uMwyJEw6RqMqwXtfCiFhswQ1UI+fV7Deh4n7JImq9O3WRYyaR8HMtezBBqyzgiooADKMZcZKAMygr2zZ3E9r9ehj2Yi8ywQsgv/Xlg7GnApqzb3+xJbP/rthzlOIZQ28YRETXwDS7iHTxV775muXdMkFWumD6Hk9iGfShq8tsxNSwpwMid9e5rtq2RCbKScrGiPVf5ip7fjrUPDCJqVCXOYCNmYSNm6X6n104ZQPY7yv/zTq/GIMmyrPWCDA243W5YrVZAAmJtelZW7sMt+wApSrkXuFFqi67Pvhuz7zUVUJZjcblgsVia3FdcEBGRIYQSROI+mnFEZJj67Lsx++4fEYVCWBB1TALyL+tbsyQZqLmi/ECMVFt0ffbdmH1fa1OCMBQ8WU3NEnnCWO8T5SQGg4gaFfgK/cPvF7q//Sv0VGUpj4ypyrdckfS/SwdGIBGpjVw6cB4n8SH2YykqwRuYtQcMImrAfUG5U+qVj5SLBhu9yllWru85XaQs5XH/UGWiaUvneCWgB/KxDJkYBi9qG726W0IUuiANj2MahmAmTmEnSjAlpLlt1HrxymoKKFsBvJcBVCgXVjc71cL/esUe5biyFeHXzsZkzMVppEO5tLq5KSb+19ORg7k4hWzw0uq2jEFEAIDP5wH7XwC836q/8aFcpxy3/wXlfdTKxRxMwArE4G7VN3yMRgxi0AETsAK5mKO+OLUKDCJC2Qqg9DeRea/S3wBlxaHvn43JGAMlvcJdisR/3BjMQzaeC+s9SCwGkcG5Lyiz3iPpwIvK+zYnAT0wDosiuijbOCxCAnpE5P1IPwwig/vnFGXpjUjy1Snv25x8LEM0TBFdlC0aMcgHl2hsa1QF0dy5cyFJUoMtPT1dq7aRxq6fVr4dU3tOqDlynfK+15v4Zr0rHkQmhqk+J9ScaMQgE8OQBP5etiWqR0SZmZmorKwMbJ988okW7SIdnF6q7S19Tge/iQcew1RNb2X0OHgbj7ZE9a+hyWRCUlKSFm0hnTk/jPxoyE+uU9YTCqY3RkR8NOQXjRj0Ri42YpYm70+Rp3pE9NVXX8FmsyE1NRX5+fm4dOmSFu0ijX138/srpjXkPqdMD7mdGXFIRKqmtRPRE2bEalqDIkdVED366KNYvXo1tm/fjqKiIly4cAGDBg3CzZs3gx7j8XjgdrsbbCTeHSsfakFW5qjd7vYVH7UgIQqJGt5OmyJL1Uez3Nz/3cc3KysLjz76KLp3745NmzZh8uTGr2x1OBz4/e9/37JWUsT5POLqmGDWpbZedajlWvTP0j333IMf/OAHKC9v5J+97xUWFsLlcgU2p9PZkpIUIVE6/R1trE4d9ElBvepQy7UoiKqrq3Hu3Dl07Rp8xSWz2QyLxdJgI/GsaUCELt8JTvq+zm2uoRwyfJqWVmbpB/8HkloXVUH0y1/+Evv27cPFixfx6aef4ic/+Qmio6PxzDPPaNU+0khMnLKUh5YsPRtft8iDGlyDtmfKr+Ec1y1qQ1QF0eXLl/HMM8+gV69e+OlPf4qEhAQcPHgQiYmJWrWPNGQfoe11RPbc4K+fxIeaXkd0Ek1cO0Ctjqpfww0bNmjVDhIgY6qynpAW5DrlvmPB7MdSDMFMTWpHIwb70MTVlNTqcK6ZgXXKUBY1i/SoSDIp79vUzQ8rcQansDPioyIvanEKOyN200fSB4PI4AYtU+4dH0lRJuV9m1OCKfCiNqKz772oRQlCmHFLrQqDyOAsKUB2hD+eZS8ObdnYb3ARGzAzorPvN2AGl41tgxhEhPTngYffjMx7DZgHpKtYtfUAirEFrwFA2CMj/3FbMAcHsDKs9yCxuHg+AQD6vQZ0vE9ZJM1Xp24yrGRSPo5lL1YXQn7bMB9ufI1xWIRomFRNhvWiFl7UYgNmMITaMI6IKCD9eWDsacCmrF/f7Els/+u2HOW4cELI7wCKMRcZKIOycn9zJ7H9r5dhD+YikyHUxnFERA1YUoCRO+vd12xbIxNkJeViRXuu8hV9U9+OqfENLuIdPFXvvma5d0yQVa6YPoeT2IZ9KOK3Y+0Eg4ga1SkDyH5H+X+97/RaiTPYiFnYiFm806tBSLIsa70YRANutxtWqxWQgFibnpWV+3DLPkCKUu4FbpTaouuz78bse00FlKVgXK5m55iKCyIiMoRQgkjcRzOOiAxTn303Zt/9I6JQCAuijklA/mV9a5YkAzVXlB+IkWqLrs++G7Pva21KEIaCX98TkXAMIiISjkFERMIxiIhIOAYREQnHICIi4RhERCQcg4iIhFMdRFeuXMH48eORkJCADh064KGHHkJpaakWbSMig1B1ZfX169eRnZ2NnJwcbNu2DYmJifjqq6/QqVMnrdpHRAagKojeeust2O12rFq1KvBcSkoIixMTETVB1UezDz74AA8//DDGjh2LLl26oG/fvli+fHmTx3g8Hrjd7gYbEVF9qoLo/PnzKCoqwgMPPIAdO3Zg2rRpmDlzJtasWRP0GIfDAavVGtjsdnuLG01E7YuqIPL5fOjXrx/mz5+Pvn374he/+AVeeOEFLF26NOgxhYWFcLlcgc3pdLa40UTUvqgKoq5duyIjI6PBcw8++CAuXboU9Biz2QyLxdJgIyKqT1UQZWdn4+zZsw2e+/LLL9G9e/eINoqIjEVVEL388ss4ePAg5s+fj/Lycqxbtw5//etfUVBQoFX7iMgAVAXRgAEDsHnzZqxfvx69e/fGG2+8gYULFyI/P1+r9hGRAaheKjYvLw95eXlatIWIDIpzzYhIOAYREQnHICIi4RhERCQcg4iIhGMQEZFwDCIiEo5BRETCSbIsy3oWdLvdsFqtgATE2vSsrNyHW/YBUpRyL3Cj1BZdn303Zt9rKgDIgMvlanayu7ggIiJDCCWIVE/xiBiOiAxTn303Zt/9I6JQCAuijklA/mV9a5YkAzVXlB+IkWqLrs++G7Pva21KEIaCJ6uJSDgGEREJxyAiIuEYREQkHIOIiIRjEBGRcAwiIhKOQUREwqkKoh49ekCSpDs23k6IiFpC1ZXVR44cgdfrDTw+efIkhg4dirFjx0a8YURkHKqCKDExscHjBQsWoGfPnnj88ccj2igiMpaw55p99913WLt2LV555RVIkhR0P4/HA4/HE3jsdrvDLUlE7VTYJ6u3bNmCGzduYNKkSU3u53A4YLVaA5vdbg+3JBG1U2EHUXFxMXJzc2GzNb2WR2FhIVwuV2BzOp3hliSidiqsj2b//ve/sWvXLrz//vvN7ms2m2E2m8MpQ0QGEdaIaNWqVejSpQtGjhwZ6fYQkQGpDiKfz4dVq1Zh4sSJMJnELfBIRO2H6iDatWsXLl26hOeee06L9hCRAake0gwbNgw6r7dPRO0c55oRkXAMIiISjkFERMIxiIhIOAYREQnHICIi4RhERCScJOt8UZDb7YbVagUkILbp+bIRx3ugs+/su35qKgDIgMvlgsViaXJfcUFERIYQShCJmyzGEZFh6rPvxuy7f0QUCmFB1DEJyL+sb82SZKDmivIDMVJt0fXZd2P2fa1NCcJQ8GQ1EQnHICIi4RhERCQcg4iIhGMQEZFwDCIiEo5BRETCMYiISDhVQeT1evH6668jJSUFHTp0QM+ePfHGG29wDWsiahFVV1a/9dZbKCoqwpo1a5CZmYnS0lI8++yzsFqtmDlzplZtJKJ2TlUQffrppxg9enTgxoo9evTA+vXrcfjwYU0aR0TGoOqj2Y9+9CPs3r0bX375JQDg+PHj+OSTT5Cbm6tJ44jIGFSNiGbPng2324309HRER0fD6/Vi3rx5yM/PD3qMx+OBx+MJPHa73eG3lojaJVUjok2bNqGkpATr1q3D559/jjVr1uCPf/wj1qxZE/QYh8MBq9Ua2Ox2e4sbTUTti6og+tWvfoXZs2dj3LhxeOihh/Dzn/8cL7/8MhwOR9BjCgsL4XK5ApvT6Wxxo4mofVH10ezWrVuIimqYXdHR0fD5fEGPMZvNMJvN4bWOiAxBVRCNGjUK8+bNQ7du3ZCZmYkvvvgCf/7zn/Hcc89p1T4iMgBVQbRo0SK8/vrrmD59Oq5evQqbzYYpU6bgt7/9rVbtIyIDUBVE8fHxWLhwIRYuXKhRc4jIiDjXjIiEYxARkXAMIiISjkFERMIxiIhIOAYREQnHICIi4RhERCScJOu8zqvL5cI999wDQLkft55uVQGQAUhAxyTj1BZdn30XU1t0ff9972/cuAGr1drkvroH0eXLl7kUCJGBOJ1OJCcnN7mP7kHk8/lQUVGB+Ph4SJKk6li32w273Q6n0wmLxaJRC1tnffbdeLVF129pbVmWcfPmTdhstjtW7bidqrlmkRAVFdVsOjbHYrEI+aVoDfXZd+PVFl2/JbWb+0jmx5PVRCQcg4iIhGtTQWQ2m/G73/1O2IqPIuuz78arLbq+nrV1P1lNRHS7NjUiIqL2iUFERMIxiIhIOAYREQnXpoLos88+Q3R0NEaOHKlbzUmTJkGSpMCWkJCA4cOH41//+pdubaiqqsKMGTOQmpoKs9kMu92OUaNGYffu3ZrWrd/3mJgY3HfffRg6dChWrlzZ5L3stKhffxs+fLjmtZuqX15ernntqqoqzJo1C2lpabj77rtx3333ITs7G0VFRbh165ZmdSdNmoQxY8bc8fzevXshSRJu3LihSd02FUTFxcWYMWMG9u/fj4qKCt3qDh8+HJWVlaisrMTu3bthMpmQl5enS+2LFy+if//++Pjjj/H222/jxIkT2L59O3JyclBQUKB5fX/fL168iG3btiEnJwezZs1CXl4e6urqdKtff1u/fr3mdZuqn5KSomnN8+fPo2/fvti5cyfmz5+PL774Ap999hl+/etfY+vWrdi1a5em9UXQfYpHuKqrq7Fx40aUlpaiqqoKq1evxpw5c3SpbTabkZSkTF1OSkrC7NmzMWjQIFy7dg2JiYma1p4+fTokScLhw4cRGxsbeD4zM1OXG1vW7/v999+Pfv364Yc//CGeeOIJrF69Gs8//7xu9UUQUX/69OkwmUwoLS1t8DNPTU3F6NGj0R6vuGkzI6JNmzYhPT0dvXr1wvjx47Fy5UohP5Dq6mqsXbsWaWlpSEhI0LTWf/7zH2zfvh0FBQUNfiH9/Mup6G3IkCHo06cP3n//fSH127NvvvkGO3fuDPozB6B6snhb0GaCqLi4GOPHjwegDJddLhf27dunS+2tW7ciLi4OcXFxiI+PxwcffICNGzc2O6O4pcrLyyHLMtLT0zWtE4709HRcvHhR8zr1/+z92/z58zWvG6z+2LFjNa3n/5n36tWrwfP33ntvoA2vvvqqpm1o7M88NzdX05pt4qPZ2bNncfjwYWzevBkAYDKZ8LOf/QzFxcUYPHiw5vVzcnJQVFQEALh+/Treffdd5Obm4vDhw+jevbtmdVvzEFyWZV3+Za7/Z+/XuXNnzesGqx9slKK1w4cPw+fzIT8/Hx6PR9Najf2ZHzp0KDAQ0EKbCKLi4mLU1dXBZrMFnpNlGWazGYsXLw55qYFwxcbGIi0tLfB4xYoVsFqtWL58Od58803N6j7wwAOQJAllZWWa1QjXmTNnND9pC9z5Z683veunpaVBkiScPXu2wfOpqakAgA4dOmjehsb6fPnyZU1rtvqPZnV1dfjb3/6GP/3pTzh27FhgO378OGw2m67foPhJkoSoqCj897//1bRO586d8dRTT2HJkiWoqam543Wtvkptzscff4wTJ07g6aefFlK/PUtISMDQoUOxePHiRn/m7VWrHxFt3boV169fx+TJk+8Y+Tz99NMoLi7G1KlTNW2Dx+NBVVUVAOWj2eLFi1FdXY1Ro0ZpWhcAlixZguzsbDzyyCP4wx/+gKysLNTV1eGjjz5CUVERzpw5o2l9f9+9Xi++/vprbN++HQ6HA3l5eZgwYYKmtevXr89kMuHee+/VvLYo7777LrKzs/Hwww9j7ty5yMrKQlRUFI4cOYKysjL0799fdBMjT27l8vLy5BEjRjT62qFDh2QA8vHjxzWrP3HiRBnK8uMyADk+Pl4eMGCA/Pe//12zmrerqKiQCwoK5O7du8t33XWXfP/998s//vGP5T179mhat37fTSaTnJiYKD/55JPyypUrZa/Xq2nt2+vX33r16qV5bX/90aNH61LrdhUVFfKLL74op6SkyDExMXJcXJz8yCOPyG+//bZcU1OjWd1gfd6zZ48MQL5+/bomdbkMCBEJ1+rPERFR+8cgIiLhGEREJByDiIiEYxARkXAMIiISjkFERMIxiIhIOAYREQnHICIi4RhERCQcg4iIhPt/kWo4zMTZT44AAAAASUVORK5CYII=\n"
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def plot_othello_board(board, ax=None) -> None:\n",
    "    \"\"\"Plots a single otello board.\n",
    "\n",
    "    If a matplot axis object is given the board will be plotted into that axis. If not an axis object will be generated.\n",
    "    The image generated will be shown directly.\n",
    "\n",
    "    Args:\n",
    "        board: The bord that should be plotted. Only a single games is allowed. A numpy array of the form 8x8 is expected.\n",
    "        ax: If needed a matplotlib axis object can be defined that is used to place the board as a sublot into a bigger context.\n",
    "    \"\"\"\n",
    "    assert board.shape == (8, 8)\n",
    "    plot_all = False\n",
    "    if ax is None:\n",
    "        fig_size = 3\n",
    "        plot_all = True\n",
    "        fig, ax = plt.subplots(figsize=(fig_size, fig_size))\n",
    "\n",
    "    ax.set_facecolor(\"#66FF00\")\n",
    "    for x_pos, y_pos in itertools.product(range(BOARD_SIZE), range(BOARD_SIZE)):\n",
    "        if board[x_pos, y_pos] == -1:\n",
    "            color = \"white\"\n",
    "        elif board[x_pos, y_pos] == 1:\n",
    "            color = \"black\"\n",
    "        else:\n",
    "            continue\n",
    "        ax.scatter(y_pos, x_pos, s=300 if plot_all else 150, c=color)\n",
    "    for x_pos in range(-1, 8):\n",
    "        ax.axhline(x_pos + 0.5, color=\"black\", lw=2)\n",
    "        ax.axvline(x_pos + 0.5, color=\"black\", lw=2)\n",
    "    ax.set_xlim(-0.5, 7.5)\n",
    "    ax.set_ylim(7.5, -0.5)\n",
    "    ax.set_xticks(np.arange(8))\n",
    "    ax.set_xticklabels(list(\"ABCDEFGH\"))\n",
    "    ax.set_yticks(np.arange(8))\n",
    "    ax.set_yticklabels(list(\"12345678\"))\n",
    "    if plot_all:\n",
    "        plt.tight_layout()\n",
    "        plt.show()\n",
    "\n",
    "\n",
    "plot_othello_board(get_new_games(1)[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_othello_boards(boards: np.ndarray) -> None:\n",
    "    \"\"\"Plots multiple boards into subplots.\n",
    "\n",
    "    The plots are shown directly.\n",
    "\n",
    "    Args:\n",
    "        boards: Plots the boards given into subplots. The maximum number of boards accepted is 70.\n",
    "    \"\"\"\n",
    "    assert len(boards.shape) == 3\n",
    "    assert boards.shape[1:] == (BOARD_SIZE, BOARD_SIZE)\n",
    "    assert boards.shape[0] < 70\n",
    "\n",
    "    plots_per_row = 4\n",
    "    rows = int(np.ceil(boards.shape[0] / plots_per_row))\n",
    "    fig, axs = plt.subplots(rows, plots_per_row, figsize=(12, 3 * rows))\n",
    "    for game_index, ax in enumerate(axs.flatten()):\n",
    "        if game_index >= boards.shape[0]:\n",
    "            fig.delaxes(ax)\n",
    "        else:\n",
    "            plot_othello_board(boards[game_index], ax)\n",
    "    plt.tight_layout()\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Find possible actions to take\n",
    "\n",
    "The frist step in the implementation of an AI like this is to get an overview over the possible actions that can be taken in a situation.\n",
    "Here was the design choice taken to first find fields that are empty and have at least one neighbouring enemy stone.\n",
    "This was implemented with element wise check for a stone and a binary dilation marking all fields neighboring an enemy stone.\n",
    "For that the `SURROUNDING` mask was used. Both aries are then element wise combined using and.\n",
    "The resulting array contains all filed where a turn could potentially be made. Those are then check in detail.\n",
    "The previous element wise operations on the numpy array increase the spead for this operation dramatically.\n",
    "\n",
    "The check for a possible turn is done in detail by following each direction step by step as long as there are enemy stones in that direction.\n",
    "If the board end is reached or en empty filed before reaching a field occupied by the player that direction does not surround enemy stones.\n",
    "If one direction surrounds enemy stone a turn is possible.\n",
    "This detailed step is implemented as a recursion and need to go at leas one step to return True."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": "array([[[1, 1, 1],\n        [1, 0, 1],\n        [1, 1, 1]]])"
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "SURROUNDING: Final = np.array(\n",
    "    [[[1, 1, 1], [1, 0, 1], [1, 1, 1]]]\n",
    ")  # defines the binary dilation mask to check if a field is next to an enemy stones\n",
    "SURROUNDING"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "9.31 ms ± 1.67 ms per loop (mean ± std. dev. of 7 runs, 100 loops each)\n",
      "831 ms ± 25.2 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)\n"
     ]
    },
    {
     "data": {
      "text/plain": "array([[[False, False, False, False, False, False, False, False],\n        [False, False, False, False, False, False, False, False],\n        [False, False, False,  True, False, False, False, False],\n        [False, False,  True, False, False, False, False, False],\n        [False, False, False, False, False,  True, False, False],\n        [False, False, False, False,  True, False, False, False],\n        [False, False, False, False, False, False, False, False],\n        [False, False, False, False, False, False, False, False]]])"
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def _recursive_steps(board, rec_direction, rec_position, step_one=True) -> bool:\n",
    "    \"\"\"Check if a player can place a stone on the board specified in the direction specified and direction specified.\n",
    "\n",
    "    Args:\n",
    "        board: The board that should be checked for a playable action.\n",
    "        rec_direction: The direction that should be checked.\n",
    "        rec_position: The position that should be checked.\n",
    "        step_one: Defines if the call of this function is the firs or not. Should be kept to the default value for proper functionality.\n",
    "\n",
    "    Returns:\n",
    "        True if a turn is possible for possition and direction on the board defined.\n",
    "    \"\"\"\n",
    "    rec_position = rec_position + rec_direction\n",
    "    if np.any((rec_position >= BOARD_SIZE) | (rec_position < 0)):\n",
    "        return False\n",
    "    next_field = board[tuple(rec_position.tolist())]\n",
    "    if next_field == 0:\n",
    "        return False\n",
    "    if next_field == -1:\n",
    "        return _recursive_steps(board, rec_direction, rec_position, step_one=False)\n",
    "    if next_field == 1:\n",
    "        return not step_one\n",
    "\n",
    "\n",
    "def get_possible_turns(boards: np.ndarray) -> np.ndarray:\n",
    "    \"\"\"Analyses a stack of boards.\n",
    "\n",
    "    Args:\n",
    "        boards: A stack of boards to check.\n",
    "\n",
    "    Returns:\n",
    "        A stack of game boards containing boolean values showing where turns are possible for the player.\n",
    "    \"\"\"\n",
    "    assert len(boards.shape) == 3, \"The number fo input dimensions does not fit.\"\n",
    "    assert boards.shape[1:] == (\n",
    "        BOARD_SIZE,\n",
    "        BOARD_SIZE,\n",
    "    ), \"The input dimensions do not fit.\"\n",
    "\n",
    "    _poss_turns = boards == 0  # checks where fields are empty.\n",
    "    _poss_turns &= binary_dilation(\n",
    "        boards == -1, SURROUNDING\n",
    "    )  # checks where fields are next to an enemy filed an empty\n",
    "    for game, idx, idy in itertools.product(\n",
    "        range(boards.shape[0]), range(BOARD_SIZE), range(BOARD_SIZE)\n",
    "    ):\n",
    "        position = idx, idy\n",
    "        if _poss_turns[game, idx, idy]:\n",
    "            _poss_turns[game, idx, idy] = any(\n",
    "                _recursive_steps(boards[game, :, :], direction, position)\n",
    "                for direction in DIRECTIONS\n",
    "            )\n",
    "    return _poss_turns\n",
    "\n",
    "\n",
    "# some simple testing to ensure the function works after simple changes\n",
    "# this testing is complete, its more of a smoke-test\n",
    "test_array = get_new_games(3)\n",
    "expected_result = np.zeros_like(test_array, dtype=bool)\n",
    "expected_result[:, 4, 5] = expected_result[:, 2, 3] = True\n",
    "expected_result[:, 5, 4] = expected_result[:, 3, 2] = True\n",
    "np.testing.assert_equal(get_possible_turns(test_array), expected_result)\n",
    "\n",
    "\n",
    "%timeit get_possible_turns(get_new_games(10))  # checks turn possibility evaluation time for 10 initial games\n",
    "%timeit get_possible_turns(get_new_games(EXAMPLE_STACK_SIZE))  # check turn possibility evaluation time for EXAMPLE_STACK_SIZE initial games\n",
    "\n",
    "# shows a singe game\n",
    "get_possible_turns(get_new_games(3))[:1]"
   ]
  },
  {
   "cell_type": "markdown",
   "source": [
    "Besides the ability to generate an array of possible turns there needs to be a functions that check if a given turn is possible.\n",
    "On is needed for the action space validation. The other is for validating a players turn."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "def move_possible(board: np.ndarray, move: np.ndarray) -> bool:\n",
    "    \"\"\"Checks if a turn is possible.\n",
    "\n",
    "    Checks if a turn is possible. If no turn is possible to input array [-1, -1] is expected.\n",
    "\n",
    "    Args:\n",
    "        board: A board where it should be checkt if a turn is possible.\n",
    "        move: The move that should be taken. Expected is the index of the filed where a stone should be placed [x, y]. If no placement is possible [-1, -1] is expected as an input.\n",
    "\n",
    "    Returns:\n",
    "        True if the move is possible\n",
    "    \"\"\"\n",
    "    if np.all(move == -1):\n",
    "        return not np.any(get_possible_turns(np.reshape(board, (1, 8, 8))))\n",
    "    return any(\n",
    "        _recursive_steps(board[:, :], direction, move) for direction in DIRECTIONS\n",
    "    )\n",
    "\n",
    "\n",
    "# Some testing for this function and the underlying recursive functions that are called.\n",
    "assert move_possible(get_new_games(1)[0], np.array([2, 3])) is True\n",
    "assert move_possible(get_new_games(1)[0], np.array([3, 2])) is True\n",
    "assert move_possible(get_new_games(1)[0], np.array([2, 2])) is False\n",
    "assert move_possible(np.zeros((8, 8)), np.array([3, 2])) is False\n",
    "assert move_possible(np.ones((8, 8)) * 1, np.array([-1, -1])) is True\n",
    "assert move_possible(np.ones((8, 8)) * -1, np.array([-1, -1])) is True\n",
    "assert move_possible(np.ones((8, 8)) * 0, np.array([-1, -1])) is True"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "def moves_possible(boards: np.ndarray, moves: np.ndarray) -> np.ndarray:\n",
    "    \"\"\"Checks if a stack of moves can be executed on a stack of boards.\n",
    "\n",
    "    Args:\n",
    "        boards: A board where the next stone should be placed.\n",
    "        moves: A stack stones to be placed. Each move is formatted as an array in the form of [x, y] if no turn is possible the value [-1, -1] is expected.\n",
    "\n",
    "    Returns:\n",
    "        An array marking for each and every game and move in the stack if the move can be executed.\n",
    "    \"\"\"\n",
    "    arr_moves_possible = np.zeros(boards.shape[0], dtype=bool)\n",
    "    for game in range(boards.shape[0]):\n",
    "        if np.all(\n",
    "            moves[game] == -1\n",
    "        ):  # can be all or any. All should be faster since most times neither value will be -1.\n",
    "            arr_moves_possible[game] = not np.any(\n",
    "                get_possible_turns(np.reshape(boards[game], (1, 8, 8)))\n",
    "            )\n",
    "        else:\n",
    "            arr_moves_possible[game] = any(\n",
    "                _recursive_steps(boards[game, :, :], direction, moves[game])\n",
    "                for direction in DIRECTIONS\n",
    "            )\n",
    "    return arr_moves_possible\n",
    "\n",
    "\n",
    "np.testing.assert_array_equal(\n",
    "    moves_possible(np.ones((3, 8, 8)) * 1, np.array([[-1, -1]] * 3)),\n",
    "    np.array([True] * 3),\n",
    ")\n",
    "\n",
    "np.testing.assert_array_equal(\n",
    "    moves_possible(get_new_games(3), np.array([[2, 3], [3, 2], [3, 2]])),\n",
    "    np.array([True] * 3),\n",
    ")\n",
    "np.testing.assert_array_equal(\n",
    "    moves_possible(get_new_games(3), np.array([[2, 2], [1, 1], [0, 0]])),\n",
    "    np.array([False] * 3),\n",
    ")\n",
    "np.testing.assert_array_equal(\n",
    "    moves_possible(np.ones((3, 8, 8)) * -1, np.array([[-1, -1]] * 3)),\n",
    "    np.array([True] * 3),\n",
    ")\n",
    "np.testing.assert_array_equal(\n",
    "    moves_possible(np.zeros((3, 8, 8)), np.array([[-1, -1]] * 3)),\n",
    "    np.array([True] * 3),\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Reword functions\n",
    "\n",
    "For any kind of reinforcement learning is a reword function needed.\n",
    "For otello this would be the final score, the information who won or changes to the score.\n",
    "A combination of those three would also be possible.\n",
    "It is probably not be possible to weight the current score to high in a reword function since that would be to close to a classic greedy algorithm.\n",
    "But some direct influence would increase the learning speed.\n",
    "In the next section are all three reword functions implemented to be combined and weight later on as needed."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "172 µs ± 7.68 µs per loop (mean ± std. dev. of 7 runs, 10,000 loops each)\n",
      "29.9 µs ± 1.08 µs per loop (mean ± std. dev. of 7 runs, 10,000 loops each)\n",
      "31.6 µs ± 1.01 µs per loop (mean ± std. dev. of 7 runs, 10,000 loops each)\n"
     ]
    }
   ],
   "source": [
    "def final_boards_evaluation(boards: np.ndarray) -> np.ndarray:\n",
    "    \"\"\"Evaluates the board at the end of the game.\n",
    "\n",
    "    All unused fields are added to the score of the player that has more stones with his color up.\n",
    "    This score only applies to the end of the game.\n",
    "    Normally the score is represented by the number of stones each player has.\n",
    "    In this case the score was combined by building the difference.\n",
    "\n",
    "    Args:\n",
    "        boards: A stack of game bords ot the end of the game.\n",
    "\n",
    "    Returns:\n",
    "        the combined score for both player.\n",
    "    \"\"\"\n",
    "    score1, score2 = np.sum(boards == 1, axis=(1, 2)), np.sum(boards == -1, axis=(1, 2))\n",
    "    player_1_won = score1 > score2\n",
    "    player_2_won = score1 < score2\n",
    "    score1_final = 64 - score2[player_1_won]\n",
    "    score2_final = 64 - score1[player_2_won]\n",
    "    score1[player_1_won] = score1_final\n",
    "    score2[player_2_won] = score2_final\n",
    "    return score1 - score2\n",
    "\n",
    "\n",
    "def evaluate_boards(boards: np.ndarray) -> np.ndarray:\n",
    "    \"\"\"Counts the stones each player has on the board.\n",
    "\n",
    "    Args:\n",
    "        boards: A stack of boards for evaluation.\n",
    "\n",
    "    Returns:\n",
    "        the combined score for both player.\n",
    "    \"\"\"\n",
    "    return np.sum(boards, axis=(1, 2))\n",
    "\n",
    "\n",
    "def evaluate_who_won(boards: np.ndarray) -> np.ndarray:\n",
    "    \"\"\"Checks who won or is winning a game.\n",
    "\n",
    "    Args:\n",
    "        boards: A stack of boards for evaluation.\n",
    "\n",
    "    Returns:\n",
    "        The information who won for both player. 1 meaning the player won, -1 means the opponent lost. 0 represents a patt.\n",
    "    \"\"\"\n",
    "    return np.sign(np.sum(boards, axis=(1, 2)))\n",
    "\n",
    "\n",
    "_boards = get_new_games(EXAMPLE_STACK_SIZE)\n",
    "%timeit final_boards_evaluation(_boards)\n",
    "%timeit evaluate_boards(_boards)\n",
    "%timeit evaluate_who_won(_boards)"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Execute a chosen action\n",
    "\n",
    "After an evaluation what turns are possible there needs to be a function that executes a turn.\n",
    "This next sections does that."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "class InvalidTurn(ValueError):\n",
    "    \"\"\"\n",
    "    This error is thrown if a given turn is not valid.\n",
    "    \"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "89.4 ms ± 3.1 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)\n"
     ]
    },
    {
     "data": {
      "text/plain": "<Figure size 300x300 with 1 Axes>",
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAASIAAAEiCAYAAABdvt+2AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/P9b71AAAACXBIWXMAAA9hAAAPYQGoP6dpAAAdqElEQVR4nO3de3BU5f0/8PdJNi4QsivEYFiyQEIsMcHw5aalGZCoIIFQ7Di0OqGCAgUJF3XaSqy2tAqLY9uhCgYK4dIh3OyIdRi5y6WiXAJCwRA0CGUhpDg27JJQV5I9vz8Ou78EctmzOec8m5z3a+aMbHbP+TwPG9885/YcSZZlGUREAkWJbgAREYOIiIRjEBGRcAwiIhKOQUREwjGIiEg4BhERCccgIiLhLEYX9Pv9qKioQFxcHCRJMro8ERlElmVcv34dDocDUVHNj3kMD6KKigo4nU6jyxKRIG63G0lJSc1+xvAgiouLC/65U3dja9+oBCADkIBOieapLbo++y6mtuj6N64o/63//3xTDA+iwO5Yp+7AxApjaxcnATWXgVgHkHfJPLVF12ffzdn3dQ4ljEI5BMOD1UQkHIOIiIRjEBGRcAwiIhKOQUREwjGIiEg4BhERCccgIiLhVAfRgQMHMG7cODgcDkiShA8++ECHZhGRmagOopqaGvTv3x9Lly7Voz1EZEKqb/HIyclBTk6OHm0hIpPS/V4zn88Hn88XfO31evUuSURtjO4Hq10uF+x2e3DhFCBEdDvdg6igoAAejye4uN1uvUsSURuj+66Z1WqF1WrVuwwRtWG8joiIhFM9IqqurkZ5eXnw9fnz53HixAl07doVPXv21LRxRGQOqoOopKQE2dnZwdcvvfQSAGDSpElYs2aNZg0jIvNQHUQjRoyALMt6tIWITIrHiIhIOAYREQnHICIi4RhERCQcg4iIhGMQEZFwDCIiEo5BRETCSbLBVyd6vV7Y7XZAUp7HbaQbVwDZD0hRQKfu5qktuj77bs6+11QAkAGPxwObzdbsZ8UFERGZQihBpPs0IE3iiMg09dl3c/Y9MCIKhbAg6pQI5F0ytmZxElBzWflCzFRbdH323Zx9X+dQgjAUPFhNRMIxiIhIOAYREQnHICIi4RhERCQcg4iIhGMQEZFwDCIiEk5VELlcLgwZMgRxcXHo1q0bnnjiCZw9e1avthGRSagKov379yM/Px+HDh3Crl27cPPmTYwaNQo1NTV6tY+ITEDVLR7bt29v8HrNmjXo1q0bjh07huHDh2vaMCIyj1bda+bxeAAAXbt2bfIzPp8PPp8v+Nrr9bamJBG1Q2EfrPb7/XjhhReQlZWFfv36Nfk5l8sFu90eXJxOZ7gliaidCjuI8vPzcfr0aWzcuLHZzxUUFMDj8QQXt9sdbkkiaqfC2jWbNWsWtm7digMHDiApKanZz1qtVlit1rAaR0TmoCqIZFnG7NmzsWXLFuzbtw/Jycl6tYuITERVEOXn52P9+vX4xz/+gbi4OFRWVgIA7HY7OnbsqEsDiaj9U3WMqLCwEB6PByNGjED37t2Dy6ZNm/RqHxGZgOpdMyIirfFeMyISjkFERMIxiIhIOAYREQnHICIi4RhERCQcg4iIhGMQEZFwkmzwVYperxd2ux2QgFiHkZWV53DLfkCKUp4Fbpbaouuz7+bse00FAFmZt8xmszX7WXFBRESmEEoQtWqGxlbhiMg09dl3c/Y9MCIKhbAg6pQI5F0ytmZxElBzWflCzFS7tfVvVgOecsDvA6KsgD0ViOlsTG0t8HsXU3+dQwnCUIgbEVFEqyoFSpcB7o8A79do+C+bBNhSAOcYIH0G0CVdVCupvWAQUQPe88A/pwOXdwGSBZBrG/mQDHjPAaWFwBfvAD1GAsOWAzbOk0dh4ul7CipbCbyXDlTsVV43GkL1BN6v2KusV7ZS3/ZR+8UgIgDA8QXAgWlA3XctB9Dt5FplvQPTlO0QqcUgIpStBEpe1WZbJa8CZUXabIvMg0Fkct7zwMHZ2m7z4Cxlu0ShYhCZ3D+nA36Vu2It8dcq2yUKlerJ8zMzM2Gz2WCz2TB06FBs27ZNr7aRzqpKlbNjao8JtUSuVbZbdUbb7VL7pSqIkpKSsGjRIhw7dgwlJSV45JFHMH78eHzxxRd6tY90VLpMOUWvB8minN4nCoWqX8Nx48Y1eL1gwQIUFhbi0KFDyMjI0LRhpD/3R9qPhgLkWsDNwTKFKOx/D+vq6vDee++hpqYGQ4cO1bJNZIDvr9+6YlpH3nPK7SFELVEdRKdOncLQoUPx3XffoXPnztiyZQvS05u+xt/n88Hn8wVfe73e8FpKmvKeQ8g3JIZNVu5RI2qJ6rNmffv2xYkTJ3D48GE8//zzmDRpEkpLS5v8vMvlgt1uDy5Op7NVDSZt+H0tf6Yt1aG2TXUQ3XXXXUhNTcWgQYPgcrnQv39//OUvf2ny8wUFBfB4PMHF7Xa3qsGkjShr+6pDbVurz5n4/f4Gu163s1qtsFr52xhp7KkAJOi7eybdqkPUAlVBVFBQgJycHPTs2RPXr1/H+vXrsW/fPuzYsUOv9pFOYjorU3l4z+lXw9ZH3bxFZF6qgujq1at45plncOXKFdjtdmRmZmLHjh0YOXKkXu0jHTnHKNf66HEKX7IAzhztt0vtk6ogKiri3YztSfoMZT4hPci1QPrz+myb2h/ea2ZiXdKVSc20vrpasijb7XK/ttul9otBZHLDlgNRGgdRlEXZLlGoGEQmZ0sGsjTePctawmljSR0GESFtKjD4DW22NWQBkDZFm22ReTCICAAw8DfA8BVAdAf1x4wki7Le8JXAgFf0aR+1bwwiCkqbCkwoBRzZyuuWAinwviNbWY8jIQoXHydEDdiSgbE76z3XbFsjN8hKysWKzhzlFD3PjlFrMYioUV3Sgay3lT+39kmvRC2RZFnWezKIBrxeL+x2OyABsQ4jK/MZ6Ow7+26kmgooU8F4PLDZbM1+VlwQEZEphBJE4nbNOCIyTX323Zx9D4yIQiEsiDolAnmXjK1ZnATUXFa+EDPVFl2ffTdn39c5lCAMBQ9WU4tEHqzmgXJzYBBRo4Kn7z+6Ncn+7afvU5RpRNJnKGfY2kttEoNBRA14zytPab28S7lgsdG5imTl2qLSQmUakR4jlZtcW3t/mcjaJBavrKagspXAe+lAxV7ldUsTpgXer9irrFe2sm3WJvEYRAQAOL4AODANqPtO/YyNcq2y3oFpynbaUm2KDAwiQtlKoORVbbZV8ipQpmIiT5G1KXIwiEzOex44OFvbbR6cpWw3kmtTZGEQmdw/pwN+jSfP99cq243k2hRZWhVEixYtgiRJeOGFFzRqDhmpqlQ5Q6X1UzzkWmW7VWciszZFnrCD6OjRo1i+fDkyMzO1bA8ZqHSZ9hPnB0gW5RR7JNamyBNWEFVXVyMvLw8rVqxAly5dtG4TGcT9kT7PNAOU7bq3RWZtijxhBVF+fj7Gjh2Lxx57TOv2kEG+v37rqmUdec8pt2hEUm2KTKoHxxs3bsTx48dx9OjRkD7v8/ng8/mCr71er9qSpIM7Zl3Ug6zcJyay9j3/p3Md0oSqEZHb7cbcuXNRXFyMDh06hLSOy+WC3W4PLk6nM6yGkrb8vpY/o1cdkbUpMqkKomPHjuHq1asYOHAgLBYLLBYL9u/fj7fffhsWiwV1dXV3rFNQUACPxxNc3G63Zo2n8EVZxdURWZsik6pds0cffRSnTp1q8LNnn30WaWlpePnllxEdHX3HOlarFVYrfyMijT0VgAR9d5GkW3UiqDZFJlVBFBcXh379+jX4WWxsLOLj4+/4OUW2mM7KdBrec/rVsPVpfO4gkbUpMvHKahNzjtH3Wh5nTmTWpsjT6l+Fffv2adAMEiF9hjKnjx7kWuWZZ5FYmyIPR0Qm1iVdmVhM65GJZFG229yDF0XWpsjDIDK5YcuBKI3DIMqibDeSa1NkYRCZnC0ZyNJ4FylrSWhTt4qsTZGFQURImwoMfkObbQ1ZAKRNaRu1KXIwiAgAMPA3wPAVQHQH9cdtJIuy3vCVwIBX2lZtigwMIgpKmwpMKAUc2crrlkIh8L4jW1mvNaMRkbVJPD5OiBqwJQNjd9Z7tti2Rm5SlZQLBp05ymlyrc5QiaxNYjGIqFFd0oGst5U/G/20VZG1SQxJlmW9J2RowOv1wm63AxIQ6zCysvIcbtkPSFHKs8DNUlt0ffbdnH2vqYAyHYvHA5vN1uxnxQUREZlCKEEkbteMIyLT1Gffzdn3wIgoFMKCqFMikHfJ2JrFSUDNZeULMVNt0fXZd3P2fZ1DCcJQ8GA1tUjkAWMrYpGAVFhgRS18+Abl8KHGmOJkGAYRNSp4Cv2jWxPd334KPUWZyiN9hnKWS0vdcT+GYwb6YQwSkAKp3uVuMvz4Bl/jND7CASzDFfABZu0Bg4ga8J5XnpR6eZdy0WCjj/yRlet7SguVqTx6jFRuNG3tPV7x6I08LEcGRqEONxGNmDs+IyEK3ZCKh/E8HsEcfIGdKMZ0fIsLrStOQvHKagoqWwm8lw5U7FVet/TcscD7FXuV9cpWhl87C1MwH6VIg3JpdWMhVF/g/TRkYz6+QBZ4aXVbxiAiAMDxBcCBaUDdd+offCjXKusdmKZsR60cvIJnsBIx6NBiAN0uGjGIQUc8g5XIAW82a6sYRISylUDJq9psq+RVoKwo9M9nYQqegJJeEqSwagbWewILkIXnwtoGicUgMjnveeDgbG23eXCWst2WxKM3nsI7kDV6nIcMGU/hHcSjtybbI+MwiEzun9MBv8bPoPfXKtttSR6WIxqWsEdCt5MgIRoxyAOnaGxrVAXR/PnzIUlSgyUtLU2vtpHOqkqVs2Nqjwm1RK5VtlvVzJn17rgfGRil+phQS6IRgwyMQiL4e9mWqB4RZWRk4MqVK8Hlk08+0aNdZIDSZfo+0qe0sOn3h2MG6nBTl9p1uImHwcd4tCWqfw0tFgsSExP1aAsZzP2R9qOhALlWmU+oKf0wRvPRUEA0YtAPOdiEubpsn7SnekT01VdfweFwICUlBXl5ebh48aIe7SKdfX/91hXTOvKeU24PuZ0VnZGAFF1rJ6APrIjVtQZpR1UQPfTQQ1izZg22b9+OwsJCnD9/HsOGDcP169ebXMfn88Hr9TZYSLw7Zj7Ug6zco3a7BPRpcNuGHiREIQGputYg7ajaNcvJ+f/P8c3MzMRDDz2EXr16YfPmzZgypfErW10uF37/+9+3rpWkOb9PXB0LrIbUNqoOtV6r/lm6++678YMf/ADl5Y38s3dLQUEBPB5PcHG73a0pSRqJMuj/0cbq1MKYFDSqDrVeq4Kouroa586dQ/fuTc+4ZLVaYbPZGiwknj0V0OjynaZJt+rc5huUQ4Zf19LKXfpN/wNJkUVVEP3yl7/E/v37ceHCBXz66af4yU9+gujoaDz99NN6tY90EtNZmcpDT7Y+jc9b5EMNvoG+R8q/wTnOW9SGqAqiS5cu4emnn0bfvn3x05/+FPHx8Th06BASEhL0ah/pyDlG3+uInDlNv38aH+l6HdFpNHPtAEUcVb+GGzdu1KsdJED6DGU+IT3Itcpzx5pyAMvwCOboUjsaMdiPZq6mpIjDe81MrEu6MqmZ1qMiyaJst7mHH17BGXyBnZqPiupwE19gJypRpul2SV8MIpMbthyI0jiIoizKdltSjOmow01N776vw00UI4Q7bimiMIhMzpYMZGm8e5a1JLRpY7/FBWzEHE3vvt+I2Zw2tg1iEBHSpgKD39BmW0MWAGkqZm09iCJ8gN8AQNgjo8B6H+AVHMSqsLZBYnHyfAIADPwN0OleZZI0f626m2Eli7I7lrVEXQgFbMNCePEfPIV3EA2Lqpth63ATdbiJjZjNEGrDOCKioLSpwIRSwKHMX9/iQezA+45sZb1wQijgIIowH+kogzJzf0sHsQPvl2Ev5iODIdTGcUREDdiSgbE76z3XbFsjN8hKysWKzhzlFH1zZ8fU+BYX8DYer/dcs5w7bpBVrpg+h9PYhv0o5NmxdoJBRI3qkg5kva382egnvV7BGWzCXGzCXD7p1SQkWZb1ngyiAa/XC7vdDkhArMPIyspzuGU/IEUpzwI3S23R9dl3c/a9pgLKVDAeT4v3mIoLIiIyhVCCSNyuGUdEpqnPvpuz74ERUSiEBVGnRCDvkrE1i5OAmsvKF2Km2qLrs+/m7Ps6hxKEoeDpeyISjkFERMIxiIhIOAYREQnHICIi4RhERCQcg4iIhGMQEZFwqoPo8uXLmDhxIuLj49GxY0c88MADKCkp0aNtRGQSqq6srqqqQlZWFrKzs7Ft2zYkJCTgq6++QpcuXfRqHxGZgKogevPNN+F0OrF69ergz5KTQ5icmIioGap2zT788EMMHjwYEyZMQLdu3TBgwACsWLGi2XV8Ph+8Xm+DhYioPlVB9PXXX6OwsBD33XcfduzYgeeffx5z5szB2rVrm1zH5XLBbrcHF6fT2epGE1H7oiqI/H4/Bg4ciIULF2LAgAH4xS9+gWnTpmHZsmVNrlNQUACPxxNc3G53qxtNRO2LqiDq3r070tPTG/zs/vvvx8WLF5tcx2q1wmazNViIiOpTFURZWVk4e/Zsg599+eWX6NWrl6aNIiJzURVEL774Ig4dOoSFCxeivLwc69evx1//+lfk5+fr1T4iMgFVQTRkyBBs2bIFGzZsQL9+/fD6669j8eLFyMvL06t9RGQCqqeKzc3NRW5urh5tISKT4r1mRCQcg4iIhGMQEZFwDCIiEo5BRETCMYiISDgGEREJxyAiIuEkWZZlIwt6vV7Y7XZAAmIdRlZWnsMt+wEpSnkWuFlqi67Pvpuz7zUVAGTA4/G0eLO7uCAiIlMIJYhU3+KhGY6ITFOffTdn3wMjolAIC6JOiUDeJWNrFicBNZeVL8RMtUXXZ9/N2fd1DiUIQ8GD1UQkHIOIiIRjEBGRcAwiIhKOQUREwjGIiEg4BhERCccgIiLhVAVR7969IUnSHQsfJ0REraHqyuqjR4+irq4u+Pr06dMYOXIkJkyYoHnDiMg8VAVRQkJCg9eLFi1Cnz598PDDD2vaKCIyl7DvNfv++++xbt06vPTSS5AkqcnP+Xw++Hy+4Guv1xtuSSJqp8I+WP3BBx/g2rVrmDx5crOfc7lcsNvtwcXpdIZbkojaqbCDqKioCDk5OXA4mp/Lo6CgAB6PJ7i43e5wSxJROxXWrtm///1v7N69G++//36Ln7VarbBareGUISKTCGtEtHr1anTr1g1jx47Vuj1EZEKqg8jv92P16tWYNGkSLBZxEzwSUfuhOoh2796Nixcv4rnnntOjPURkQqqHNKNGjYLB8+0TUTvHe82ISDgGEREJxyAiIuEYREQkHIOIiIRjEBGRcAwiIhJOkg2+KMjr9cJutwMSENv8/bKa4zPQ2Xf23Tg1FQBkwOPxwGazNftZcUFERKYQShCJu1mMIyLT1Gffzdn3wIgoFMKCqFMikHfJ2JrFSUDNZeULMVNt0fXZd3P2fZ1DCcJQ8GA1EQnHICIi4RhERCQcg4iIhGMQEZFwDCIiEo5BRETCMYiISDhVQVRXV4fXXnsNycnJ6NixI/r06YPXX3+dc1gTUauourL6zTffRGFhIdauXYuMjAyUlJTg2Wefhd1ux5w5c/RqIxG1c6qC6NNPP8X48eODD1bs3bs3NmzYgCNHjujSOCIyB1W7Zj/60Y+wZ88efPnllwCAkydP4pNPPkFOTo4ujSMic1A1Ipo3bx68Xi/S0tIQHR2Nuro6LFiwAHl5eU2u4/P54PP5gq+9Xm/4rSWidknViGjz5s0oLi7G+vXrcfz4caxduxZ//OMfsXbt2ibXcblcsNvtwcXpdLa60UTUvqgKol/96leYN28ennrqKTzwwAP4+c9/jhdffBEul6vJdQoKCuDxeIKL2+1udaOJqH1RtWt248YNREU1zK7o6Gj4/f4m17FarbBareG1johMQVUQjRs3DgsWLEDPnj2RkZGBzz//HH/+85/x3HPP6dU+IjIBVUH0zjvv4LXXXsPMmTNx9epVOBwOTJ8+Hb/97W/1ah8RmYCqIIqLi8PixYuxePFinZpDRGbEe82ISDgGEREJxyAiIuEYREQkHIOIiIRjEBGRcAwiIhKOQUREwkmywfO8ejwe3H333QCU53Eb6UYlABmABHRKNE9t0fXZdzG1RdcPPPf+2rVrsNvtzX7W8CC6dOkSpwIhMhG3242kpKRmP2N4EPn9flRUVCAuLg6SJKla1+v1wul0wu12w2az6dTCyKzPvpuvtuj6ra0tyzKuX78Oh8Nxx6wdt1N1r5kWoqKiWkzHlthsNiG/FJFQn303X23R9VtTu6VdsgAerCYi4RhERCRcmwoiq9WK3/3ud8JmfBRZn303X23R9Y2sbfjBaiKi27WpERERtU8MIiISjkFERMIxiIhIuDYVRJ999hmio6MxduxYw2pOnjwZkiQFl/j4eIwePRr/+te/DGtDZWUlZs+ejZSUFFitVjidTowbNw579uzRtW79vsfExODee+/FyJEjsWrVqmafZadH/frL6NGjda/dXP3y8nLda1dWVmLu3LlITU1Fhw4dcO+99yIrKwuFhYW4ceOGbnUnT56MJ5544o6f79u3D5Ik4dq1a7rUbVNBVFRUhNmzZ+PAgQOoqKgwrO7o0aNx5coVXLlyBXv27IHFYkFubq4htS9cuIBBgwbh448/xltvvYVTp05h+/btyM7ORn5+vu71A32/cOECtm3bhuzsbMydOxe5ubmora01rH79ZcOGDbrXba5+cnKyrjW//vprDBgwADt37sTChQvx+eef47PPPsOvf/1rbN26Fbt379a1vgiG3+IRrurqamzatAklJSWorKzEmjVr8MorrxhS22q1IjFRuXU5MTER8+bNw7Bhw/DNN98gISFB19ozZ86EJEk4cuQIYmNjgz/PyMgw5MGW9fveo0cPDBw4ED/84Q/x6KOPYs2aNZg6daph9UUQUX/mzJmwWCwoKSlp8J2npKRg/PjxaI9X3LSZEdHmzZuRlpaGvn37YuLEiVi1apWQL6S6uhrr1q1Damoq4uPjda313//+F9u3b0d+fn6DX8iAwHQqRnvkkUfQv39/vP/++0Lqt2fffvstdu7c2eR3DkD1zeJtQZsJoqKiIkycOBGAMlz2eDzYv3+/IbW3bt2Kzp07o3PnzoiLi8OHH36ITZs2tXhHcWuVl5dDlmWkpaXpWiccaWlpuHDhgu516v/dB5aFCxfqXrep+hMmTNC1XuA779u3b4Of33PPPcE2vPzyy7q2obG/85ycHF1rtolds7Nnz+LIkSPYsmULAMBiseBnP/sZioqKMGLECN3rZ2dno7CwEABQVVWFd999Fzk5OThy5Ah69eqlW91IHoLLsmzIv8z1/+4Dunbtqnvdpuo3NUrR25EjR+D3+5GXlwefz6drrcb+zg8fPhwcCOihTQRRUVERamtr4XA4gj+TZRlWqxVLliwJeaqBcMXGxiI1NTX4euXKlbDb7VixYgXeeOMN3ered999kCQJZWVlutUI15kzZ3Q/aAvc+XdvNKPrp6amQpIknD17tsHPU1JSAAAdO3bUvQ2N9fnSpUu61oz4XbPa2lr87W9/w5/+9CecOHEiuJw8eRIOh8PQMygBkiQhKioK//vf/3St07VrVzz++ONYunQpampq7nhfr1OpLfn4449x6tQpPPnkk0Lqt2fx8fEYOXIklixZ0uh33l5F/Iho69atqKqqwpQpU+4Y+Tz55JMoKirCjBkzdG2Dz+dDZWUlAGXXbMmSJaiursa4ceN0rQsAS5cuRVZWFh588EH84Q9/QGZmJmpra7Fr1y4UFhbizJkzutYP9L2urg7/+c9/sH37drhcLuTm5uKZZ57RtXb9+vVZLBbcc889utcW5d1330VWVhYGDx6M+fPnIzMzE1FRUTh69CjKysowaNAg0U3UnhzhcnNz5TFjxjT63uHDh2UA8smTJ3WrP2nSJBnK9OMyADkuLk4eMmSI/Pe//123mrerqKiQ8/Pz5V69esl33XWX3KNHD/nHP/6xvHfvXl3r1u+7xWKRExIS5Mcee0xetWqVXFdXp2vt2+vXX/r27at77UD98ePHG1LrdhUVFfKsWbPk5ORkOSYmRu7cubP84IMPym+99ZZcU1OjW92m+rx3714ZgFxVVaVLXU4DQkTCRfwxIiJq/xhERCQcg4iIhGMQEZFwDCIiEo5BRETCMYiISDgGEREJxyAiIuEYREQkHIOIiIRjEBGRcP8P3ZHAPKDQyJ0AAAAASUVORK5CYII=\n"
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def do_moves(boards: np.ndarray, moves: np.ndarray) -> np.ndarray:\n",
    "    \"\"\"Executes a single move on a stack o Othello boards.\n",
    "\n",
    "    Args:\n",
    "        boards: A stack of Othello boards where the next stone should be placed.\n",
    "        moves: A stack of stone placement orders for the game. Formatted as coordinates in an array [x, y] of the place where the stone should be placed. Should contain [-1,-1] if no new placement is possible.\n",
    "\n",
    "    Returns:\n",
    "        The new state of the board.\n",
    "    \"\"\"\n",
    "\n",
    "    def _do_directional_move(\n",
    "        board: np.ndarray, rec_move: np.ndarray, rev_direction, step_one=True\n",
    "    ) -> bool:\n",
    "        \"\"\"Changes the color of enemy stones in one direction.\n",
    "\n",
    "        This function works recursive. The argument step_one should always be used in its default value.\n",
    "\n",
    "        Args:\n",
    "            board: A bord on which a stone was placed.\n",
    "            rec_move: The position on the board in x and y where this function is called from. Will be moved by recursive called.\n",
    "            rev_direction: The position where the stone was placed. Inside this recursion it will also be the last step that was checked.\n",
    "            step_one: Set to true if this is the first step in the recursion. False later on.\n",
    "\n",
    "        Returns:\n",
    "            True if a stone could be flipped.\n",
    "            All changes are made on the view of the numpy array and therefore not included in the return value.\n",
    "        \"\"\"\n",
    "        rec_position = rec_move + rev_direction\n",
    "        if np.any((rec_position >= 8) | (rec_position < 0)):\n",
    "            return False\n",
    "        next_field = board[tuple(rec_position.tolist())]\n",
    "        if next_field == 0:\n",
    "            return False\n",
    "        if next_field == 1:\n",
    "            return not step_one\n",
    "        if next_field == -1:\n",
    "            if _do_directional_move(board, rec_position, rev_direction, step_one=False):\n",
    "                board[tuple(rec_position.tolist())] = 1\n",
    "                return True\n",
    "            return False\n",
    "\n",
    "    def _do_move(_board: np.ndarray, move: np.ndarray) -> None:\n",
    "        \"\"\"Executes a turn on a board.\n",
    "\n",
    "        Args:\n",
    "            _board: The game board on wich to place a stone.\n",
    "            move: The coordinates of a stone that should be placed. Should be formatted as an array of the form [x, y]. The value [-1, -1] is expected if no turn is possible.\n",
    "\n",
    "        Returns:\n",
    "            All changes are made on the view of the numpy array.\n",
    "        \"\"\"\n",
    "        if np.all(move == -1):\n",
    "            if not move_possible(_board, move):\n",
    "                raise InvalidTurn(\"An action should be taken. A turn is possible.\")\n",
    "            return\n",
    "\n",
    "        # noinspection PyTypeChecker\n",
    "        if _board[tuple(move.tolist())] != 0:\n",
    "            raise InvalidTurn(\"This turn is not possible.\")\n",
    "\n",
    "        action = False\n",
    "        for direction in DIRECTIONS:\n",
    "            if _do_directional_move(_board, move, direction):\n",
    "                action = True\n",
    "        if not action:\n",
    "            raise InvalidTurn(\"This turn is not possible.\")\n",
    "\n",
    "        # noinspection PyTypeChecker\n",
    "        _board[tuple(move.tolist())] = 1\n",
    "\n",
    "    boards = boards.copy()\n",
    "    for game in range(boards.shape[0]):\n",
    "        _do_move(boards[game], moves[game])\n",
    "    return boards\n",
    "\n",
    "\n",
    "%timeit do_moves(get_new_games(EXAMPLE_STACK_SIZE), np.array([[2, 3]] * EXAMPLE_STACK_SIZE))[0]\n",
    "\n",
    "plot_othello_board(\n",
    "    do_moves(\n",
    "        get_new_games(EXAMPLE_STACK_SIZE), np.array([[2, 3]] * EXAMPLE_STACK_SIZE)\n",
    "    )[0]\n",
    ")"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## An abstract reversi game policy\n",
    "\n",
    "For an easy use of policies an abstract class containing the policy generation / requests an action in an inherited instance of this class.\n",
    "This class filters the policy to only propose valid actions. Inherited instance do not need to care about this. This super class also manges exploration and exploitation with the epsilon value."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "class GamePolicy(ABC):\n",
    "    \"\"\"\n",
    "    A game policy. Proposes where to place a stone next.\n",
    "    \"\"\"\n",
    "\n",
    "    def __init__(self, epsilon: float):\n",
    "        \"\"\"\n",
    "\n",
    "        Args:\n",
    "            epsilon: the epsilon / greedy value. Should be between zero and one. Set the mixture of policy and exploration. One means only the policy is used. Zero means only random policies are used. All mixtures inbetween between are possible.\n",
    "        \"\"\"\n",
    "        if 0 > epsilon > 1:\n",
    "            raise ValueError(\"Epsilon should be between zero and one.\")\n",
    "        self._epsilon: float = epsilon\n",
    "\n",
    "    @property\n",
    "    def epsilon(self):\n",
    "        return self._epsilon\n",
    "\n",
    "    @property\n",
    "    @abc.abstractmethod\n",
    "    def policy_name(self) -> str:\n",
    "        \"\"\"The name of this policy\"\"\"\n",
    "        raise NotImplementedError()\n",
    "\n",
    "    @abc.abstractmethod\n",
    "    def _internal_policy(self, boards: np.ndarray) -> np.ndarray:\n",
    "        \"\"\"The internal policy is an unfiltered policy. It should only be called from inside this function\n",
    "\n",
    "        Args:\n",
    "            boards: A board where a policy should be calculated for.\n",
    "\n",
    "        Returns:\n",
    "            The policy for this board. Should have the same size as the boards array.\n",
    "        \"\"\"\n",
    "        raise NotImplementedError()\n",
    "\n",
    "    def get_policy(self, boards: np.ndarray) -> np.ndarray:\n",
    "        \"\"\"Calculates the policy that should be followed.\n",
    "\n",
    "        Calculates the policy that should be followed.\n",
    "        This function does include the usage of epsilon to configure greediness and exploration.\n",
    "\n",
    "        Args:\n",
    "            boards: A set of boards that show the environment where the policy should be calculated for.\n",
    "\n",
    "        Returns:\n",
    "            A vector of indices. Should be formatted as an array of the form [x, y]. The value [-1, -1] is expected if no turn is possible.\n",
    "        \"\"\"\n",
    "        assert len(boards.shape) == 3\n",
    "        assert boards.shape[1:] == (BOARD_SIZE, BOARD_SIZE)\n",
    "\n",
    "        if self.epsilon <= 0:\n",
    "            policies = np.random.rand(*boards.shape)\n",
    "        else:\n",
    "            policies = self._internal_policy(boards)\n",
    "            if self.epsilon < 1:\n",
    "                policies = policies * self.epsilon + np.random.rand(*boards.shape) * (\n",
    "                    1 - self.epsilon\n",
    "                )\n",
    "\n",
    "        # todo talk to team about backpropagation of score and epsilon for greedy factor\n",
    "\n",
    "        # todo possibly change this function to only validate the purpose turn and not all turns\n",
    "        possible_turns = get_possible_turns(boards)\n",
    "        policies[possible_turns == False] = -1.0\n",
    "        max_indices = [\n",
    "            np.unravel_index(policy.argmax(), policy.shape) for policy in policies\n",
    "        ]\n",
    "        policy_vector = np.array(max_indices)\n",
    "        no_turn_possible_1 = np.all(policy_vector == 0, 1)\n",
    "        zero_pos = policies[:, 0, 0] == -1.0\n",
    "        no_turn_possible = np.all(policy_vector == 0, 1) & (policies[:, 0, 0] == -1.0)\n",
    "\n",
    "        policy_vector[no_turn_possible, :] = IMPOSSIBLE\n",
    "        return policy_vector"
   ]
  },
  {
   "cell_type": "markdown",
   "source": [
    "## A first policy\n",
    "\n",
    "To quantify the quality of a game AI there needs to be some benchmarks.\n",
    "The easiest benchmark is to play against a random player.\n",
    "The easiest player to use as a benchmark is the random player.\n",
    "For this and testing purpose the random policy was implemented."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "outputs": [],
   "source": [
    "class RandomPolicy(GamePolicy):\n",
    "    \"\"\"\n",
    "    A policy playing a random turn by setting epsilon to 0.\n",
    "    \"\"\"\n",
    "\n",
    "    def __init__(self, epsilon: float):\n",
    "        _ = epsilon\n",
    "        super().__init__(epsilon=0)\n",
    "\n",
    "    @property\n",
    "    def policy_name(self) -> str:\n",
    "        return \"random\"\n",
    "\n",
    "    def _internal_policy(self, boards: np.ndarray) -> np.ndarray:\n",
    "        pass\n",
    "\n",
    "\n",
    "rnd_policy = RandomPolicy(1)\n",
    "assert rnd_policy.policy_name == \"random\"\n",
    "assert rnd_policy.epsilon == 0\n",
    "\n",
    "rnd_policy_result = rnd_policy.get_policy(get_new_games(10))\n",
    "assert np.any((5 >= rnd_policy_result) & (rnd_policy_result >= 3))"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Putting the game simulation together\n",
    "Now it's time to bring all together for a proper simulation."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Playing a single turn\n",
    "\n",
    "The next function needed is used to request a policy, verify that the turn is legit and place a stone and turn enemy stones if possible."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.02 s ± 58.8 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)\n",
      "949 ms ± 43.3 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)\n"
     ]
    },
    {
     "data": {
      "text/plain": "<Figure size 1200x600 with 8 Axes>",
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABKUAAAJOCAYAAABm7rQwAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/P9b71AAAACXBIWXMAAA9hAAAPYQGoP6dpAABhL0lEQVR4nO3dfZCddX03/vfZLKxAsisgSGISBKGhAmEUtSgjgoo1kogdb9o68RawvX+txqcCtqYzWq2V6AiMvaumrUVCB/CpU6ziDSoqwQ7yqDzYakxqMYsJxWlxlwRdye75/XHM05KQPWd3r+u7Oa/XzBl3s+fs522uPW9OPnudcxrNZrMZAAAAAKhQT90BAAAAAOg+llIAAAAAVM5SCgAAAIDKWUoBAAAAUDlLKQAAAAAqZykFAAAAQOUspQAAAAConKUUAAAAAJXrrXrg2NhYNm3alDlz5qTRaFQ9HihQs9nMY489lnnz5qWnp75duX4C9qSEjtJPwJ7oJ6BUE+2nypdSmzZtyoIFC6oeC8wAg4ODmT9/fm3z9RPwVOrsKP0EPBX9BJRqX/1U+VJqzpw5Oz4+eG7V05PHH07STNJIDj6q+vkyyFBahrrnJ8njm1v/u2s/1KHufkoKOR5+JmWQYfcMBXSUfpKhlPkyFJZBPyUp5FjIIEMh84vJMMF+qnwptf2UzoPnJm/cVPX05Nr5ydafJofMS5Y/VP18GWQoLUPd85Pkmnmt0qr7lO+6+ykp43jUnaHu+TLIMF4JHaWfZChlvgxlZdBPLSUcCxlkKGV+KRkm2k9e6BwAAACAyllKAQAAAFA5SykAAAAAKmcpBQAAAEDlLKUAAAAAqJylFAAAAACVs5QCAAAAoHKWUgAAAABUru2l1K233pply5Zl3rx5aTQa+eIXvzgNsQDap5+AUuknoFT6CahT20uprVu35pRTTsknPvGJ6cgD0DH9BJRKPwGl0k9AnXrbvcGSJUuyZMmS6cgCMCn6CSiVfgJKpZ+AOnlNKQAAAAAq1/aZUu0aGRnJyMjIjs+Hh4eneyTAhOgnoFT6CSiVfgKm0rSfKbVq1aoMDAzsuCxYsGC6RwJMiH4CSqWfgFLpJ2AqTftSauXKlRkaGtpxGRwcnO6RABOin4BS6SegVPoJmErT/vS9vr6+9PX1TfcYgLbpJ6BU+gkolX4CplLbS6ktW7Zkw4YNOz7/z//8z9x777057LDDsnDhwikNB9AO/QSUSj8BpdJPQJ3aXkrdfffdOeuss3Z8ftFFFyVJzj///KxZs2bKggG0Sz8BpdJPQKn0E1CntpdSZ555ZprN5nRkAZgU/QSUSj8BpdJPQJ2m/YXOAQAAAGA8SykAAAAAKmcpBQAAAEDlLKUAAAAAqJylFAAAAACVs5QCAAAAoHKWUgAAAABUzlIKAAAAgMpZSgEAAABQuUaz2WxWOXB4eDgDAwNJIzlkXpWTWx7fnDTHkkZPcvDc6ufLIENpGeqenyRbNyVpJkNDQ+nv768nROrvp6SM41F3hrrnyyDDeCV0lH6SoZT5MpSVQT+1lHAsZJChlPmlZJhoP9W3lAIYp5ilFMAeFPGPPoA90E9AqfbVT70VZtmdM6VkkKGIDHXPT3Zu0YvhN31d/zMpgwy7Kqqj9FPXZ6h7vgxlZdBPLSUcCxlkKGV+KRkm2k+1LaUOPipZ/lD1c6+dn2z9aevA1DFfBhlKy1D3/CS5Zl6rOEtRVz8lZRyPujPUPV8GGcYrqaP0kwx1z5ehrAz6qaWEYyGDDKXMLyXDRPvJC50DAAAAUDlLKQAAAAAqZykFAAAAQOUspQAAAAConKUUAAAAAJWzlAIAAACgcpZSAAAAAFTOUgoAAACAyrW1lFq1alVe+MIXZs6cOTnyyCPzute9LuvWrZuubAATpp+AkukooFT6CahTW0uptWvXZsWKFbn99tvz9a9/PU888URe9apXZevWrdOVD2BC9BNQMh0FlEo/AXXqbefKN910026fr1mzJkceeWTuueeenHHGGVMaDKAd+gkomY4CSqWfgDpN6jWlhoaGkiSHHXbYlIQBmCr6CSiZjgJKpZ+AKrV1ptSuxsbG8q53vSunn356TjrppL1eb2RkJCMjIzs+Hx4e7nQkwIToJ6BkE+ko/QTUQT8BVev4TKkVK1bk+9//fj772c8+5fVWrVqVgYGBHZcFCxZ0OhJgQvQTULKJdJR+Auqgn4CqdbSUetvb3pYbbrgh3/rWtzJ//vynvO7KlSszNDS04zI4ONhRUICJ0E9AySbaUfoJqJp+AurQ1tP3ms1m3v72t+f666/PLbfckmOOOWaft+nr60tfX1/HAQEmQj8BJWu3o/QTUBX9BNSpraXUihUrct111+Vf/uVfMmfOnDz88MNJkoGBgRx00EHTEhBgIvQTUDIdBZRKPwF1auvpe6tXr87Q0FDOPPPMzJ07d8flc5/73HTlA5gQ/QSUTEcBpdJPQJ3afvoeQIn0E1AyHQWUSj8Bder43fcAAAAAoFOWUgAAAABUzlIKAAAAgMpZSgEAAABQOUspAAAAACpnKQUAAABA5SylAAAAAKicpRQAAAAAlWs0m81mlQOHh4czMDCQNJJD5lU5ueXxzUlzLGn0JAfPrX6+DDKUlqHu+UmydVOSZjI0NJT+/v56QqT+fkrKOB51Z6h7vgwyjFdCR+knGUqZL0NZGfRTSwnHQgYZSplfSoaJ9lN9SymAcYpZSgHsQRH/6APYA/0ElGpf/dRbYZbdOVNKBhmKyFD3/GTnFr0YftPX9T+TMsiwq6I6Sj91fYa658tQVgb91FLCsZBBhlLml5Jhov1U21Lq4KOS5Q9VP/fa+cnWn7YOTB3zZZChtAx1z0+Sa+a1irMUdfVTUsbxqDtD3fNlkGG8kjpKP8lQ93wZysqgn1pKOBYyyFDK/FIyTLSfvNA5AAAAAJWzlAIAAACgcpZSAAAAAFTOUgoAAACAytX37nsAANAltmxM1q1JhtYnTzyWHDAnGTg+WXRBMnth3ekAoB6WUgAAME02rU3uvzzZeEPrrbmTpDmaNGa1Pr7n/cnRS5PFlyRzz6gtJgDUwtP3AABgijWbyX2XJTecmQzemKTZWkY1R3/99e0fN5ONNyZffllredVs1hgaACpmKQUAAFPsgSuSO97d+ri57amvu/3rt1/Suh0AdAtLKQAAmEKb1rYWTJ24/ZJk861TmwcAStXWUmr16tVZvHhx+vv709/fnxe/+MW58cYbpysbwITpJ6BkOqq73H950ujwlVsbva3bQ1X0E1CntpZS8+fPz4c//OHcc889ufvuu/Pyl7885557bv7t3/5tuvIBTIh+Akqmo7rHlo2tFzXf11P29qa5LfnJl5Mtg1ObC/ZGPwF1amsptWzZsrzmNa/J8ccfn9/4jd/Ihz70ocyePTu33377dOUDmBD9BJRMR3WPdWt2vstepxo9ybqrpiQO7JN+AurU4YnFyejoaL7whS9k69atefGLX7zX642MjGRkZGTH58PDw52OBJgQ/QSUbCIdpZ9mrqH1U/N9hjdMzfeBdugnoGpt/x7ngQceyOzZs9PX15c//uM/zvXXX5/nPve5e73+qlWrMjAwsOOyYMGCSQUG2Bv9BJSsnY7STzPXE48lzdHJfY/maPIr/86nQvoJqEvbS6lFixbl3nvvzR133JG3vOUtOf/88/Pv//7ve73+ypUrMzQ0tOMyOOgJ8sD00E9AydrpKP00cx0wJ2nMmtz3aMxKDuyfmjwwEfoJqEvbT9878MADc9xxxyVJTj311Nx1113567/+6/zd3/3dHq/f19eXvr6+yaUEmAD9BJSsnY7STzPXwPFT8336j5ua7wMToZ+AukzyZRiTsbGx3Z5TDFAK/QSUTEftnxZdkDTHJvc9mmPJogunJA50RD8BVWnrTKmVK1dmyZIlWbhwYR577LFcd911ueWWW/LVr351uvIBTIh+Akqmo7rH7IXJwqXJ4I1Jc1v7t2/0Jgtfk8z2Mj1URD8BdWprKfXII4/kTW96UzZv3pyBgYEsXrw4X/3qV3P22WdPVz6ACdFPQMl0VHc55ZJk45c7u21zNFl88dTmgaein4A6tbWUuvLKK6crB8Ck6CegZDqqu8w9IzntsuT2S9q/7Wkfbd0eqqKfgDpN+jWlAACA3Z18UWsxlbSekvdUtn/9tMtatwOAbmEpBQAAU6zRaD0Nb9na1mtEpZE0ZrUuyS4fN1pfX7a2df1Go87UAFCttp6+BwAATNzcM1qXLYPJuquS4Q3Jr4aTA/uT/uNa77LnRc0B6FaWUgAAMM1mL0hOfV/dKQCgLJ6+BwAAAEDlLKUAAAAAqJylFAAAAACVs5QCAAAAoHKNZrPZrHLg8PBwBgYGkkZyyLwqJ7c8vjlpjiWNnuTgudXPl0GG0jLUPT9Jtm5K0kyGhobS399fT4jU309JGcej7gx1z5dBhvFK6Cj9JEMp82UoK4N+ainhWMggQynzS8kw0X6qbykFME4xSymAPSjiH30Ae6CfgFLtq596K8yyO2dKySBDERnqnp/s3KIXw2/6uv5nUgYZdlVUR+mnrs9Q93wZysqgn1pKOBYyyFDK/FIyTLSfaltKHXxUsvyh6udeOz/Z+tPWgaljvgwylJah7vlJcs28VnGWoq5+Sso4HnVnqHu+DDKMV1JH6ScZ6p4vQ1kZ9FNLCcdCBhlKmV9Khon2kxc6BwAAAKByllIAAAAAVM5SCgAAAIDKWUoBAAAAULn63n2PjmzZmKxbkwytT554LDlgTjJwfLLogmT2wu7IUPd8oFyHZkFekgtyZI7P0zInv8xjeSTrc1vW5NEMVpJBRwF7UkI3lJABKI/HT9TJUmqG2LQ2uf/yZOMNrbd1TJLmaNKY1fr4nvcnRy9NFl+SzD1j/8xQ93ygXMfnjJydi3NylqaZsSRJT3oy9uuPl+b9uT9fzs25POvz7WnJoKOAPSmhG0rIAJTH4ydK4Ol7hWs2k/suS244Mxm8MUmzdSdtjv7669s/biYbb0y+/LLWnbrZ3H8y1D0fKNvZuTiXZG1OypL0pCez0ptZ6U1jl4970pOT85pcklvzylw0pfN1FLAnJXRDCRmAMnn8RCkspQr3wBXJHe9ufdzc9tTX3f712y9p3W5/yVD3fKBcr8xF+V+5LEkyKwc85XW3f/28XD6lD6x0FLAnJXRDCRmA8nj8REkspQq2aW3rjteJ2y9JNt868zPUPR8o1/E5I+fl8o5ue14uz/F56aQz6ChgT0rohhIyAOXx+InSTGop9eEPfziNRiPvete7pigOu7r/8qTR4at+NXpbt5/pGeqez8yln/Z/Z+fijOaJjm47miem5Ld9OopO6Kf9XwndUEIGZiYdtX/z+InSdLyUuuuuu/J3f/d3Wbx48VTm4de2bGy92Nu+TmXcm+a25CdfTrZM4s0S6s5Q93xmLv20/zs0C3Jylu7zlPO9mZUDsjivzaGZ33EGHUUn9NP+r4RuKCEDM5OO2r95/ESJOlpKbdmyJcuXL8+nPvWpHHrooVOdibTeDrMxySdXNnqSdVfN3Ax1z2dm0k/d4SW5YMe7xHSqmbG8JBd2fHsdRbv0U3cooRtKyMDMo6P2fx4/UaKOfhxWrFiRc845J6985Sv3ed2RkZEMDw/vdmHfhtZPzfcZ3jBzM9Q9n5lJP3WHI3P8FHyXZo7IcR3fWkfRLv3UHUrohhIyMPNMtKP008zl8RMlavuZnJ/97Gfz3e9+N3fdddeErr9q1ap84AMfaDtYt3visZ1vh9mp5mjyq0n8N6LuDHXPZ+bRT93jaZmTnkm+V0dPZuWg9Hd8ex1FO/RT9yihG0rIwMzSTkfpp5nL4ydK1NZP5ODgYN75znfm2muvzdOe9rQJ3WblypUZGhracRkc9OTPiThgTtKYNbnv0ZiVHNh5X9Seoe75zCz6qbv8Mo9lbJKnn49lNL9I549odBQTpZ+6SwndUEIGZo52O0o/zVweP1Gits6Uuueee/LII4/k+c9//o4/Gx0dza233pqPf/zjGRkZyaxZu/+E9fX1pa+vb2rSdpGBqTizMkl/52dW1p6h7vnMLPqpuzySqTj3u5GfpfNzv3UUE6WfuksJ3VBCBmaOdjtKP81cHj9RorbOlHrFK16RBx54IPfee++Oywte8IIsX748995775MeUNG5RRckzcktsdMcSxZ1/hp0tWeoez4zi37qLrdlTRqTPP28kZ7cls5fJVNHMVH6qbuU0A0lZGDm0FHdw+MnStTWmVJz5szJSSedtNufHXLIITn88MOf9OdMzuyFycKlyeCNnb1dZqM3WfiaZPaCmZuh7vnMLPqpuzyawTyQG3JSlnT0tsajeSIP5Ct5NA91nEFHMVH6qbuU0A0lZGDm0FHdw+MnSjTJN2NkOp1ySWd31KT14m+LL575GeqeD5Tr67msowdUSetFOm/OFZPOoKOAPSmhG0rIAJTH4ydKM+ml1C233JKPfexjUxCF8eaekZx2WWe3Pe2jrdvP9Ax1z2dm00/7t/X5dr6Qzh6V/FPenfX59qQz6Cg6pZ/2byV0QwkZmLl01P7L4ydK40ypwp180c47bGMfT7bc/vXTLmvdbn/JUPd8oFw354odD6xG88RTXnf717+Qi6fkt3zb6ShgT0rohhIyAOXx+ImSWEoVrtFonZ64bG3rubNptN4Cc/vbaO74uNH6+rK1res3GvtPhrrnA2W7OVfkspyRB/KVjGUso9mW0WxLM2MZzRMZzbaMZSwP5Cu5LGdM6QOqREcBe1ZCN5SQASiTx0+Uoq0XOqc+c89oXbYMJuuuSoY3JL8aTg7sb70d5qILp//F3urOUPd8oFzr8+2sz7dzaObnJbkwR+S4HJT+/CLD+Vk25LZcNakX5ZwIHQXsSQndUEIGoDweP1ECS6kZZvaC5NT3dXeGuucD5Xo0D+Ur+WCtGXQUsCcldEMJGYDyePxEnTx9DwAAAIDKWUoBAAAAUDlLKQAAAAAqZykFAAAAQOUazWazWeXA4eHhDAwMJI3kkHlVTm55fHPSHEsaPcnBc6ufL4MMpWWoe36SbN2UpJkMDQ2lv7+/nhCpv5+SMo5H3Rnqni+DDOOV0FH6SYZS5stQVgb91FLCsZBBhlLml5Jhov1U31IKYJxillIAe1DEP/oA9kA/AaXaVz/1Vphld86UkkGGIjLUPT/ZuUUvht/0df3PpAwy7KqojtJPXZ+h7vkylJVBP7WUcCxkkKGU+aVkmGg/1baUOvioZPlD1c+9dn6y9aetA1PHfBlkKC1D3fOT5Jp5reIsRV39lJRxPOrOUPd8GWQYr6SO0k8y1D1fhrIy6KeWEo6FDDKUMr+UDBPtJy90DgAAAEDlLKUAAAAAqJylFAAAAACVs5QCAAAAoHL1vfseM9aWjcm6NcnQ+uSJx5ID5iQDxyeLLkhmL5z++YdmQV6SC3Jkjs/TMie/zGN5JOtzW9bk0QxOfwCgWHX3UykZgPKU0A0eQwF7UkI/ydC9LKWYsE1rk/svTzbe0HprySRpjiaNWa2P73l/cvTSZPElydwzpn7+8TkjZ+finJylaWYsSdKTnoz9+uOleX/uz5dzcy7P+nx76gMAxaq7n0rJAJSnhG7wGArYkxL6SQY8fY99ajaT+y5LbjgzGbwxSbN1J22O/vrr2z9uJhtvTL78stadutmcugxn5+JckrU5KUvSk57MSm9mpTeNXT7uSU9OzmtySW7NK3PR1A0HilVCP5WQAShPKd3gMRQwXgn9JAPbWUqxTw9ckdzx7tbHzW1Pfd3tX7/9ktbtpsIrc1H+Vy5LkszKAU953e1fPy+Xe1AFXaDufiolA1CeErrBYyhgT0roJxnYrq2l1Pvf//40Go3dLieccMJ0ZaMAm9a27niduP2SZPOtk5t/fM7Iebm8o9uel8tzfF46uQDMGPqp+9TdT6VkYGbQUd2lhG7wGIqJ0k/dpYR+koFdtX2m1IknnpjNmzfvuPzrv/7rdOSiEPdfnjQ6fOWxRm/r9pNxdi7OaJ7o6LajecJv+rqMfuoudfdTKRmYOXRU9yihGzyGoh36qXuU0E8ysKu2D0Nvb2+OOuqo6chCYbZsbL3YWzp8zmxzW/KTLydbBpPZC9q//aFZkJOzND0dPst0Vg7I4rw2h2Z+Hs1DHX0PZhb91D3q7qdSMjCz6KjuUEI3eAxFu/RTdyihn2RgvLb/S7V+/frMmzcvxx57bJYvX56NGzdORy4KsG7Nzncf6FSjJ1l3VWe3fUku2PEOMZ1qZiwvyYWT+h7MHPqpe9TdT6VkYGbRUd2hhG7wGIp26afuUEI/ycB4bZ0p9Vu/9VtZs2ZNFi1alM2bN+cDH/hAXvrSl+b73/9+5syZs8fbjIyMZGRkZMfnw8PDk0tMZYbWT833Gd7Q2e2OzPFTML2ZI3LcFHwfSqefukvd/VRKBmaOdjtKP81cJXSDx1C0Qz91jxL6SQbGa2sptWTJkh0fL168OL/1W7+Vo48+Op///OfzB3/wB3u8zapVq/KBD3xgcimpxROP7Xw7zE41R5NfdfjfqadlTsennW/Xk1k5KP2T+h7MDPqpu9TdT6VkYOZot6P008xVQjd4DEU79FP3KKGfZGC8Sf3X6ulPf3p+4zd+Ixs27H1FuHLlygwNDe24DA4OTmYkFTpgTtKYNbnv0ZiVHNjh45lf5rGMTfLU87GM5hfRFt1IP+3f6u6nUjIwc+2ro/TTzFVCN3gMxWTop/1XCf0kA+NNaim1ZcuW/Md//Efmzp271+v09fWlv79/twszw8BUnPmdpL/DM78fyVScV9nIz+K8ym6kn/ZvdfdTKRmYufbVUfpp5iqhGzyGYjL00/6rhH6SgfHaWkpdcsklWbt2bR588MHcdttt+Z3f+Z3MmjUrb3jDG6YrHzVadEHSnNwv2dIcSxZ1+BqZt2VNGpM89byRntwWr0DXDfRTd6m7n0rJwMyho7pHCd3gMRTt0E/do4R+koHx2vqv1UMPPZQ3vOENWbRoUX73d383hx9+eG6//fYcccQR05WPGs1emCxcmjTaeuWxnRq9ydHLOn+bzEczmAdyQ0bzREe3H80TuT9f8lbGXUI/dZe6+6mUDMwcOqp7lNANHkPRDv3UPUroJxkYr63D8NnPfna6clCoUy5JNn65s9s2R5PFF09u/tdzWU7Jazu6bU9m5eZcMbkAzBj6qfvU3U+lZGBm0FHdpYRu8BiKidJP3aWEfpKBXU3uvF72e3PPSE67rLPbnvbR1u0nY32+nS+ks3v8P+XdWZ9vTy4AUKy6+6mUDEB5SugGj6GAPSmhn2RgV5ZS7NPJF+28w+7rFMftXz/tstbtpsLNuWLHg6p9nYa+/etfyMV+wwddoO5+KiUDUJ4SusFjKGBPSugnGdjOUop9ajRapycuW5ssfE2SRustMLe/jeaOjxutry9b27p+ozF1GW7OFbksZ+SBfCVjGctotmU029LMWEbzREazLWMZywP5Si7LGR5MQZcooZ9KyACUp5Ru8BgKGK+EfpKB7Tp8aS+60dwzWpctg8m6q5LhDcmvhpMD+1tvh7nowul9sbf1+XbW59s5NPPzklyYI3JcDkp/fpHh/Cwbcluu8oKc0KXq7qdSMgDlKaEbPIYC9qSEfpIBSynaNntBcur76pv/aB7KV/LB+gIAxaq7n0rJAJSnhG7wGArYkxL6SYbu5el7AAAAAFTOUgoAAACAyllKAQAAAFA5SykAAAAAKtdoNpvNKgcODw9nYGAgaSSHzKtycsvjm5PmWNLoSQ6eW/18GWQoLUPd85Nk66YkzWRoaCj9/f31hEj9/ZSUcTzqzlD3fBlkGK+EjtJPMpQyX4ayMuinlhKOhQwylDK/lAwT7af6llIA4xSzlALYgyL+0QewB/oJKNW++qm3wiy7c6aUDDIUkaHu+cnOLXox/Kav638mZZBhV0V1lH7q+gx1z5ehrAz6qaWEYyGDDKXMLyXDRPuptqXUwUclyx+qfu6185OtP20dmDrmyyBDaRnqnp8k18xrFWcp6uqnpIzjUXeGuufLIMN4JXWUfpKh7vkylJVBP7WUcCxkkKGU+aVkmGg/eaFzAAAAACpnKQUAAABA5SylAAAAAKhcfS90DgD7oS0bk3VrkqH1yROPJQfMSQaOTxZdkMxeWHc6AAAoh6UUAEyBTWuT+y9PNt7QeqeTJGmOJo1ZrY/veX9y9NJk8SXJ3DNqiwkAAMXw9D0AmIRmM7nvsuSGM5PBG5M0W8uo5uivv77942ay8cbkyy9rLa+apbyFNwAA1MRSCgAm4YErkjve3fq4ue2pr7v967df0rodAAB0M0spAOjQprWtBVMnbr8k2Xzr1OYBAICZpO2l1E9/+tO88Y1vzOGHH56DDjooJ598cu6+++7pyAbQFv1E1e6/PGl0+OqMjd7W7ekeOgoolX4C6tLWQ+lHH300p59+es4666zceOONOeKII7J+/foceuih05UPYEL0E1XbsrH1oubp8LWhmtuSn3w52TKYzF4wpdEokI4CSqWfgDq1tZT6yEc+kgULFuSqq67a8WfHHHPMlIcCaJd+omrr1rTeZW/7C5p3otGTrLsqOfV9UxaLQukooFT6CahTW0/f+9KXvpQXvOAFOe+883LkkUfmec97Xj71qU895W1GRkYyPDy82wVgquknqja0fmq+z/CGqfk+lK3djtJPQFX0E1CntpZSP/7xj7N69eocf/zx+epXv5q3vOUtecc73pGrr756r7dZtWpVBgYGdlwWLPAcBWDq6Seq9sRjkztLKmnd/lcey3eFdjtKPwFV0U9AndpaSo2NjeX5z39+Lr300jzvec/L//f//X/5P//n/+Rv//Zv93qblStXZmhoaMdlcHBw0qEBxtNPVO2AOUlj1uS+R2NWcmD/1OShbO12lH4CqqKfgDq1tZSaO3dunvvc5+72Z7/5m7+ZjRs37vU2fX196e/v3+0CMNX0E1UbOH5qvk//cVPzfShbux2ln4Cq6CegTm0tpU4//fSsW7dutz/70Y9+lKOPPnpKQwG0Sz9RtUUXJM2xyX2P5liy6MIpiUPhdBRQKv0E1KmtpdSf/Mmf5Pbbb8+ll16aDRs25Lrrrsvf//3fZ8WKFdOVD2BC9BNVm70wWbg0abT1PrY7NXqTo5cls70UR1fQUUCp9BNQp7aWUi984Qtz/fXX5zOf+UxOOumkfPCDH8zHPvaxLF++fLryAUyIfqIOp1ySNLd1dtvmaLL44qnNQ7l0FFAq/QTUqe3f7y5dujRLly6djiwAk6KfqNrcM5LTLktuv6T925720dbt6R46CiiVfgLq0taZUgDA7k6+qLWYSvb9VL7tXz/tstbtAACgm1lKAcAkNBqtp+EtW5ssfE2SRtKY1boku3zcaH192drW9RuNOlMDAED9Onx5VgBgV3PPaF22DCbrrkqGNyS/Gk4O7E/6j2u9y54XNQcAgJ0spQBgCs1ekJz6vrpTAABA+Tx9DwAAAIDKWUoBAAAAUDlLKQAAAAAqZykFAAAAQOUazWazWeXA4eHhDAwMJI3kkHlVTm55fHPSHEsaPcnBc6ufL4MMpWWoe36SbN2UpJkMDQ2lv7+/nhCpv5+SMo5H3Rnqni+DDOOV0FH6SYZS5stQVgb91FLCsZBBhlLml5Jhov1U31IKYJxillIAe1DEP/oA9kA/AaXaVz/1Vphld86UkkGGIjLUPT/ZuUUvht/0df3PpAwy7KqojtJPXZ+h7vkylJVBP7WUcCxkkKGU+aVkmGg/1baUOvioZPlD1c+9dn6y9aetA1PHfBlkKC1D3fOT5Jp5reIsRV39lJRxPOrOUPd8GWQYr6SO0k8y1D1fhrIy6KeWEo6FDDKUMr+UDBPtJy90DgAAAEDlLKUAAAAAqJylFAAAAACVs5QCAAAAoHKWUgAAAABUzlIKAAAAgMpZSgEAAABQOUspAAAAACpnKQUAAABA5dpaSj372c9Oo9F40mXFihXTlQ9gwnQUUCr9BJRKPwF16m3nynfddVdGR0d3fP79738/Z599ds4777wpDwbQLh0FlEo/AaXST0Cd2lpKHXHEEbt9/uEPfzjPec5z8rKXvWxKQwF0QkcBpdJPQKn0E1CntpZSu/rVr36Va665JhdddFEajcZerzcyMpKRkZEdnw8PD3c6EmDCJtJR+gmog34CSqWfgKp1/ELnX/ziF/Pzn/88F1xwwVNeb9WqVRkYGNhxWbBgQacjASZsIh2ln4A66CegVPoJqFrHS6krr7wyS5Ysybx5857yeitXrszQ0NCOy+DgYKcjASZsIh2ln4A66CegVPoJqFpHT9/7yU9+kptvvjn//M//vM/r9vX1pa+vr5MxAB2ZaEfpJ6Bq+gkolX4C6tDRmVJXXXVVjjzyyJxzzjlTnQdg0nQUUCr9BJRKPwF1aHspNTY2lquuuirnn39+ens7fp10gGmho4BS6SegVPoJqEvbS6mbb745GzduzJvf/ObpyAMwKToKKJV+Akqln4C6tL0Gf9WrXpVmszkdWQAmTUcBpdJPQKn0E1CXjt99DwAAAAA6ZSkFAAAAQOUspQAAAAConKUUAAAAAJWzlAIAAACgcpZSAAAAAFTOUgoAAACAyllKAQAAAFC5RrPZbFY5cHh4OAMDA0kjOWRelZNbHt+cNMeSRk9y8Nzq58sgQ2kZ6p6fJFs3JWkmQ0ND6e/vrydE6u+npIzjUXeGuufLIMN4JXSUfpKhlPkylJVBP7WUcCxkkKGU+aVkmGg/1beUAhinmKUUwB4U8Y8+gD3QT0Cp9tVPvRVm2Z0zpWSQoYgMdc9Pdm7Ri+E3fV3/MymDDLsqqqP0U9dnqHu+DGVl0E8tJRwLGWQoZX4pGSbaT7UtpQ4+Kln+UPVzr52fbP1p68DUMV8GGUrLUPf8JLlmXqs4S1FXPyVlHI+6M9Q9XwYZxiupo/STDHXPl6GsDPqppYRjIYMMpcwvJcNE+8kLnQMAAABQOUspAAAAACpnKQUAAABA5SylAAAAAKicpRQAAAAAlbOUAgAAAKByllIAAAAAVM5SCgAAAIDKtbWUGh0dzXvf+94cc8wxOeigg/Kc5zwnH/zgB9NsNqcrH8CE6CegZDoKKJV+AurU286VP/KRj2T16tW5+uqrc+KJJ+buu+/OhRdemIGBgbzjHe+YrowA+6SfgJLpKKBU+gmoU1tLqdtuuy3nnntuzjnnnCTJs5/97HzmM5/JnXfeOS3hACZKPwEl01FAqfQTUKe2nr73kpe8JN/4xjfyox/9KEly33335V//9V+zZMmSaQkHMFH6CSiZjgJKpZ+AOrV1ptR73vOeDA8P54QTTsisWbMyOjqaD33oQ1m+fPlebzMyMpKRkZEdnw8PD3eeFmAv9BNQsnY7Sj8BVdFPQJ3aOlPq85//fK699tpcd911+e53v5urr746l112Wa6++uq93mbVqlUZGBjYcVmwYMGkQwOMp5+AkrXbUfoJqIp+AurU1lLq3e9+d97znvfk93//93PyySfnf//v/50/+ZM/yapVq/Z6m5UrV2ZoaGjHZXBwcNKhAcbTT0DJ2u0o/QRURT8BdWrr6XuPP/54enp232PNmjUrY2Nje71NX19f+vr6OksHMEH6CShZux2ln4Cq6CegTm0tpZYtW5YPfehDWbhwYU488cR873vfyxVXXJE3v/nN05UPYEL0E1AyHQWUSj8BdWprKfU3f/M3ee9735u3vvWteeSRRzJv3rz80R/9Ud73vvdNVz6ACdFPQMl0FFAq/QTUqa2l1Jw5c/Kxj30sH/vYx6YpDkBn9BNQMh0FlEo/AXVq64XOAQAAAGAqWEoBAAAAUDlLKQAAAAAqZykFAAAAQOUspQAAAAConKUUAAAAAJWzlAIAAACgcpZSAAAAAFTOUgoAAACAyjWazWazyoFDQ0N5+tOfniQ5eG6Vk1sefzhJM0kjOfio6ufLIENpGeqenySPb279789//vMMDAzUEyL191NSyPHwMymDDLtnKKCj9JMMpcyXobAM+ilJIcdCBhkKmV9Mhgn2U+VLqYceeigLFiyociQwQwwODmb+/Pm1zddPwFOps6P0E/BU9BNQqn31U+VLqbGxsWzatClz5sxJo9Fo+/bDw8NZsGBBBgcH09/fPw0JZZgpGeqeL8PUZWg2m3nssccyb9689PTU96xi/STD/pSh7vn7U4YSOmqy/ZTUfzzqni+DDKVl0E871X0sSshQ93wZZJjqDBPtp97JhOxET0/PlGzx+/v7azs4MpSVoe75MkxNhjqftredfpJhf8xQ9/z9JUPdHTVV/ZTUfzzqni+DDKVl0E871X0sSshQ93wZZJjKDBPpJy90DgAAAEDlLKUAAAAAqNyMW0r19fXlL/7iL9LX1ydDl2eoe74MZWUoQQl/DzLIUMp8GcpT999F3fNlkKG0DHXPL0kJfxd1Z6h7vgwy1JWh8hc6BwAAAIAZd6YUAAAAADOfpRQAAAAAlbOUAgAAAKByM2op9Z3vfCezZs3KOeecU/nsCy64II1GY8fl8MMPz6tf/ercf//9lWd5+OGH8/a3vz3HHnts+vr6smDBgixbtizf+MY3pn32rn8PBxxwQJ75zGfm7LPPzqc//emMjY1N+/zxGXa9vPrVr65k/r5ybNiwoZL5Dz/8cN75znfmuOOOy9Oe9rQ885nPzOmnn57Vq1fn8ccfn/b5F1xwQV73utc96c9vueWWNBqN/PznP5/2DKXRUfppfI66Oqrufkrq7Sj99GT6ST+Nz6GfPIYqhX7ST+Nz6Kfu6qcZtZS68sor8/a3vz233nprNm3aVPn8V7/61dm8eXM2b96cb3zjG+nt7c3SpUsrzfDggw/m1FNPzTe/+c189KMfzQMPPJCbbropZ511VlasWFFJhu1/Dw8++GBuvPHGnHXWWXnnO9+ZpUuXZtu2bZVm2PXymc98ppLZ+8pxzDHHTPvcH//4x3ne856Xr33ta7n00kvzve99L9/5znfyp3/6p7nhhhty8803T3sGnqzbO0o/PTlHnR1VVz8lOqpE+kk/jc+hn/RTKfSTfhqfQz91Vz/11h1gorZs2ZLPfe5zufvuu/Pwww9nzZo1+fM///NKM/T19eWoo45Kkhx11FF5z3vek5e+9KX52c9+liOOOKKSDG9961vTaDRy55135pBDDtnx5yeeeGLe/OY3V5Jh17+HZz3rWXn+85+f0047La94xSuyZs2a/OEf/mGlGepUV463vvWt6e3tzd13373bz8Gxxx6bc889N95Us3o6Sj/tLUdd6sygo8qin/TT3nLURT+xnX7ST3vLURf9VL0Zc6bU5z//+ZxwwglZtGhR3vjGN+bTn/50rQdly5Ytueaaa3Lcccfl8MMPr2Tm//zP/+Smm27KihUrdvsh3e7pT396JTn25OUvf3lOOeWU/PM//3NtGbrFf//3f+drX/vaXn8OkqTRaFScim7vKP3EdjqqPPpJP9Gin8qjn/QTLd3cTzNmKXXllVfmjW98Y5LWKXVDQ0NZu3ZtpRluuOGGzJ49O7Nnz86cOXPypS99KZ/73OfS01PNX+OGDRvSbDZzwgknVDKvXSeccEIefPDBSmbteiy2Xy699NJKZj9VjvPOO2/aZ27/OVi0aNFuf/6MZzxjR44/+7M/m/YcyZ6Pw5IlSyqZXZpu7yj9tLsSOqqOfkrK6Sj9tJN+0k+70k/191Oio7bTT/ppV/qpO/tpRjx9b926dbnzzjtz/fXXJ0l6e3vze7/3e7nyyitz5plnVpbjrLPOyurVq5Mkjz76aD75yU9myZIlufPOO3P00UdP+/zST9drNpuVbW93PRbbHXbYYZXMfqoce9tqV+HOO+/M2NhYli9fnpGRkUpm7uk43HHHHTseXHQLHaWfxiuho0rqp6T6jtJPLfpJP42nn57MY6h66Cf9NJ5+erJu6KcZsZS68sors23btsybN2/HnzWbzfT19eXjH/94BgYGKslxyCGH5Ljjjtvx+T/8wz9kYGAgn/rUp/JXf/VX0z7/+OOPT6PRyA9/+MNpn9WJH/zgB5W9CNz4Y1GXOnIcd9xxaTQaWbdu3W5/fuyxxyZJDjrooMqy7On//0MPPVTZ/FLoKP00XgkdVVeGUjpKP7XoJ/00nn6qv58SHZXop0Q/jaefurOfin/63rZt2/KP//iPufzyy3PvvffuuNx3332ZN29eLe+4tl2j0UhPT09+8YtfVDLvsMMOy2//9m/nE5/4RLZu3fqkr9f59rHf/OY388ADD+T1r399bRm6xeGHH56zzz47H//4x/f4c0C1dFSLfmI7HVUO/dSin9hOP5VDP7XoJ7br5n4q/kypG264IY8++mj+4A/+4Enb8te//vW58sor88d//MeVZBkZGcnDDz+cpHVq58c//vFs2bIly5Ytq2R+knziE5/I6aefnhe96EX5y7/8yyxevDjbtm3L17/+9axevTo/+MEPpj3D9r+H0dHR/Nd//VduuummrFq1KkuXLs2b3vSmaZ+/a4Zd9fb25hnPeEYl8+v2yU9+Mqeffnpe8IIX5P3vf38WL16cnp6e3HXXXfnhD3+YU089te6IXUNH7aSfnpxjVzpKR1VNP+2kn56cY1f6ST9VTT/tpJ+enGNX+qkL+qlZuKVLlzZf85rX7PFrd9xxRzNJ87777pv2HOeff34zyY7LnDlzmi984Qub//RP/zTts8fbtGlTc8WKFc2jjz66eeCBBzaf9axnNV/72tc2v/Wtb0377F3/Hnp7e5tHHHFE85WvfGXz05/+dHN0dHTa54/PsOtl0aJFlczfNce5555b6cxdbdq0qfm2t72tecwxxzQPOOCA5uzZs5svetGLmh/96EebW7dunfb5e/v//61vfauZpPnoo49Oe4YS6KjddXs/jc9RV0fV3U/NZr0dpZ9a9NPu9JN+2s5jqPrpp93pJ/20XTf2U6PZLPzV1QAAAADY7xT/mlIAAAAA7H8spQAAAAConKUUAAAAAJWzlAIAAACgcpZSAAAAAFTOUgoAAACAyllKAQAAAFA5SykAAAAAKmcpBQAAAEDlLKUAAAAAqJylFAAAAACVs5QCAAAAoHKWUgAAAABUzlIKAAAAgMpZSgEAAABQOUspAAAAACpnKQUAAABA5XqrHjg2NpZNmzZlzpw5aTQaVY8HCtRsNvPYY49l3rx56empb1eun4A9KaGj9BOwJ/oJKNVE+6nypdSmTZuyYMGCqscCM8Dg4GDmz59f23z9BDyVOjtKPwFPRT8BpdpXP1W+lJozZ86Ojw+eW/X05PGHkzSTNJKDj6p+vgwylJah7vlJ8vjm1v/u2g91qLufkkKOh59JGWTYPUMBHaWfZChlvgyFZdBPSQo5FjLIUMj8YjJMsJ8qX0ptP6Xz4LnJGzdVPT25dn6y9afJIfOS5Q9VP18GGUrLUPf8JLlmXqu06j7lu+5+Sso4HnVnqHu+DDKMV0JH6ScZSpkvQ1kZ9FNLCcdCBhlKmV9Khon2kxc6BwAAAKByllIAAAAAVM5SCgAAAIDKWUoBAAAAUDlLKQAAAAAqZykFAAAAQOUspQAAAAConKUUAAAAAJVreyl16623ZtmyZZk3b14ajUa++MUvTkMsgPbpJ6BU+gkolX4C6tT2Umrr1q055ZRT8olPfGI68gB0TD8BpdJPQKn0E1Cn3nZvsGTJkixZsmQ6sgBMin4CSqWfgFLpJ6BObS+l2jUyMpKRkZEdnw8PD0/3SIAJ0U9AqfQTUCr9BEylaX+h81WrVmVgYGDHZcGCBdM9EmBC9BNQKv0ElEo/AVNp2pdSK1euzNDQ0I7L4ODgdI8EmBD9BJRKPwGl0k/AVJr2p+/19fWlr69vuscAtE0/AaXST0Cp9BMwlab9TCkAAAAAGK/tM6W2bNmSDRs27Pj8P//zP3PvvffmsMMOy8KFC6c0HEA79BNQKv0ElEo/AXVqeyl1991356yzztrx+UUXXZQkOf/887NmzZopCwbQLv0ElEo/AaXST0Cd2l5KnXnmmWk2m9ORBWBS9BNQKv0ElEo/AXXymlIAAAAAVM5SCgAAAIDKWUoBAAAAUDlLKQAAAAAqZykFAAAAQOUspQAAAAConKUUAAAAAJWzlAIAAACgco1ms9mscuDw8HAGBgaSRnLIvContzy+OWmOJY2e5OC51c+XQYbSMtQ9P0m2bkrSTIaGhtLf319PiNTfT0kZx6PuDHXPl0GG8UroKP0kQynzZSgrg35qKeFYyCBDKfNLyTDRfqpvKQUwTjFLKYA9KOIffQB7oJ+AUu2rn3orzLI7Z0rJIEMRGeqen+zcohfDb/q6/mdSBhl2VVRH6aeuz1D3fBnKyqCfWko4FjLIUMr8UjJMtJ9qW0odfFSy/KHq5147P9n609aBqWO+DDKUlqHu+UlyzbxWcZairn5KyjgedWeoe74MMoxXUkfpJxnqni9DWRn0U0sJx0IGGUqZX0qGifaTFzoHAAAAoHKWUgAAAABUzlIKAAAAgMpZSgEAAABQOUspAAAAACpnKQUAAABA5SylAAAAAKicpRQAAAAAlbOUAgAAAKBybS2lVq1alRe+8IWZM2dOjjzyyLzuda/LunXrpisbwITpJ6BkOgoolX4C6tTWUmrt2rVZsWJFbr/99nz961/PE088kVe96lXZunXrdOUDmBD9BJRMRwGl0k9AnXrbufJNN9202+dr1qzJkUcemXvuuSdnnHHGlAYDaId+Akqmo4BS6SegTm0tpcYbGhpKkhx22GF7vc7IyEhGRkZ2fD48PDyZkQATop+Aku2ro/QTUBf9BFSp4xc6Hxsby7ve9a6cfvrpOemkk/Z6vVWrVmVgYGDHZcGCBZ2OBJgQ/QSUbCIdpZ+AOugnoGodL6VWrFiR73//+/nsZz/7lNdbuXJlhoaGdlwGBwc7HQkwIfoJKNlEOko/AXXQT0DVOnr63tve9rbccMMNufXWWzN//vynvG5fX1/6+vo6CgfQLv0ElGyiHaWfgKrpJ6AObS2lms1m3v72t+f666/PLbfckmOOOWa6cgG0RT8BJdNRQKn0E1CntpZSK1asyHXXXZd/+Zd/yZw5c/Lwww8nSQYGBnLQQQdNS0CAidBPQMl0FFAq/QTUqa3XlFq9enWGhoZy5plnZu7cuTsun/vc56YrH8CE6CegZDoKKJV+AurU9tP3AEqkn4CS6SigVPoJqFPH774HAAAAAJ2ylAIAAACgcpZSAAAAAFTOUgoAAACAyllKAQAAAFA5SykAAAAAKmcpBQAAAEDlLKUAAAAAqFyj2Ww2qxw4PDycgYGBpJEcMq/KyS2Pb06aY0mjJzl4bvXzZZChtAx1z0+SrZuSNJOhoaH09/fXEyL191NSxvGoO0Pd82WQYbwSOko/yVDKfBnKyqCfWko4FjLIUMr8UjJMtJ/qW0oBjFPMUgpgD4r4Rx/AHugnoFT76qfeCrPszplSMshQRIa65yc7t+jF8Ju+rv+ZlEGGXRXVUfqp6zPUPV+GsjLop5YSjoUMMpQyv5QME+2n2pZSBx+VLH+o+rnXzk+2/rR1YOqYL4MMpWWoe36SXDOvVZylqKufkjKOR90Z6p4vgwzjldRR+kmGuufLUFYG/dRSwrGQQYZS5peSYaL95IXOAQAAAKicpRQAAAAAlbOUAgAAAKByllIAAAAAVM5SCgAAAIDKWUoBAAAAUDlLKQAAAAAqZykFAAAAQOXaWkqtXr06ixcvTn9/f/r7+/PiF784N95443RlA5gw/QSUTEcBpdJPQJ3aWkrNnz8/H/7wh3PPPffk7rvvzstf/vKce+65+bd/+7fpygcwIfoJKJmOAkqln4A69bZz5WXLlu32+Yc+9KGsXr06t99+e0488cQpDQbQDv0ElExHAaXST0Cd2lpK7Wp0dDRf+MIXsnXr1rz4xS+eykwAk6KfgJLpKKBU+gmoWttLqQceeCAvfvGL88tf/jKzZ8/O9ddfn+c+97l7vf7IyEhGRkZ2fD48PNxZUoB90E9AydrpKP0EVEk/AXVp+933Fi1alHvvvTd33HFH3vKWt+T888/Pv//7v+/1+qtWrcrAwMCOy4IFCyYVGGBv9BNQsnY6Sj8BVdJPQF3aXkodeOCBOe6443Lqqadm1apVOeWUU/LXf/3Xe73+ypUrMzQ0tOMyODg4qcAAe6OfgJK101H6CaiSfgLq0vFrSm03Nja22+mb4/X19aWvr2+yYwDapp+Akj1VR+knoE76CahKW0uplStXZsmSJVm4cGEee+yxXHfddbnlllvy1a9+dbryAUyIfgJKpqOAUuknoE5tLaUeeeSRvOlNb8rmzZszMDCQxYsX56tf/WrOPvvs6coHMCH6CSiZjgJKpZ+AOrW1lLryyiunKwfApOgnoGQ6CiiVfgLq1PYLnQMAAADAZFlKAQAAAFA5SykAAAAAKmcpBQAAAEDlLKUAAAAAqJylFAAAAACVs5QCAAAAoHKWUgAAAABUzlIKAAAAgMo1ms1ms8qBw8PDGRgYSBrJIfOqnNzy+OakOZY0epKD51Y/XwYZSstQ9/wk2bopSTMZGhpKf39/PSFSfz8lZRyPujPUPV8GGcYroaP0kwylzJehrAz6qaWEYyGDDKXMLyXDRPupvqUUwDjFLKUA9qCIf/QB7IF+Akq1r37qrTDL7pwpJYMMRWSoe36yc4teDL/p6/qfSRlk2FVRHaWfuj5D3fNlKCuDfmop4VjIIEMp80vJMNF+qm0pdfBRyfKHqp977fxk609bB6aO+TLIUFqGuucnyTXzWsVZirr6KSnjeNSdoe75MsgwXkkdpZ9kqHu+DGVl0E8tJRwLGWQoZX4pGSbaT17oHAAAAIDKWUoBAAAAUDlLKQAAAAAqZykFAAAAQOXqe/c9ZqxDsyAvyQU5MsfnaZmTX+axPJL1uS1r8mgGp33+lo3JujXJ0PrkiceSA+YkA8cniy5IZi+c9vHAXpRw35QBKFXdj58S/QTsmX6iTpZSTNjxOSNn5+KcnKVpZixJ0pOejP3646V5f+7Pl3NzLs/6fHvK529am9x/ebLxhtZbWyZJczRpzGp9fM/7k6OXJosvSeaeMeXjgb0o4b4pA1Cquh8/JfoJ2DP9RAk8fY8JOTsX55KszUlZkp70ZFZ6Myu9aezycU96cnJek0tya16Zi6ZsdrOZ3HdZcsOZyeCNSZqtomqO/vrr2z9uJhtvTL78slaxNZtTFgHYgxLumzIAJavz8VOin4C900+UwlKKfXplLsr/ymVJklk54Cmvu/3r5+XyKSuuB65I7nh36+Pmtqe+7vav335J63bA9CnhvikDUKq6Hz8l+gnYM/1ESSa1lPrwhz+cRqORd73rXVMUh9IcnzNyXi7v6Lbn5fIcn5dOav6mta3y6cTtlySbb53UeGYw/TS9SrhvysBMpZ/2f3U/fkr0E53TUfs3/URpOl5K3XXXXfm7v/u7LF68eCrzUJizc3FG80RHtx3NE5Pept9/edLo8JXPGr2t29N99NP0K+G+KQMzkX7qDnU/fkr0E53RUfs//URpOlpKbdmyJcuXL8+nPvWpHHrooVOdiUIcmgU5OUv3eUrn3szKAVmc1+bQzO/o9ls2tl7wbl+nc+5Nc1vyky8nW6p5wwgKoZ+mXwn3TRmYifRTd6j78VOin+iMjtr/6SdK1NFSasWKFTnnnHPyyle+cqrzUJCX5IId78LQqWbG8pJc2NFt163Z+Q4MnWr0JOuumtz3YGbRT9OvhPumDMxE+qk71P34KdFPdEZH7f/0EyVq+6S5z372s/nud7+bu+66a0LXHxkZycjIyI7Ph4eH2x1JTY7M8VPwXZo5Isd1dMuh9VMwPsnwhqn5PpRPP1WjhPumDMw0+ql71P34KdFPtK+djtJPM5d+okRt7SgHBwfzzne+M9dee22e9rSnTeg2q1atysDAwI7LggULOgpK9Z6WOemZ5Bs09mRWDkp/R7d94rGdbwnaqeZo8iv/newK+qk6Jdw3ZWAm0U/dpe7HT4l+oj3tdpR+mrn0EyVq6yfynnvuySOPPJLnP//56e3tTW9vb9auXZv/+3//b3p7ezM6+uSfrpUrV2ZoaGjHZXDQkz9nil/msYxN8vTOsYzmF+msMQ6YkzRmTWp8GrOSAzvvTGYQ/VSdEu6bMjCT6KfuUvfjp0Q/0Z52O0o/zVz6iRK19fS9V7ziFXnggQd2+7MLL7wwJ5xwQv7sz/4ss2Y9+aerr68vfX19k0tJLR7JVJxb2cjP0tm5lQNTcXZpkv7Ozy5lBtFP1SnhvikDM4l+6i51P35K9BPtabej9NPMpZ8oUVtnSs2ZMycnnXTSbpdDDjkkhx9+eE466aTpykhNbsuaNCZ5emcjPbktnb0K3aILkubkFvlpjiWLOn8dPmYQ/VSdEu6bMjCT6KfuUvfjp0Q/0R4d1T30EyWa5Ovesz97NIN5IDdkNE90dPvRPJH786U8moc6uv3shcnCpUmj7Zfjb2n0JkcvS2Z7mjtMqRLumzIApar78VOin4A900+UqMMfhZ1uueWWKYhBqb6ey3JKXtvRbXsyKzfniknNP+WSZOOXO7ttczRZfPGkxjPD6afpU8J9UwZmMv20f6v78VOin5gcHbX/0k+UxplSPKX1+Xa+kM7u9f+Ud2d9vj2p+XPPSE67rLPbnvbR1u2BqVfCfVMGoFR1P35K9BOwZ/qJ0lhKsU8354odxbWvUz23f/0LuXhKtuhJcvJFO0trX6d5bv/6aZe1bgdMnxLumzIApar78VOin4A900+UxFKKCbk5V+SynJEH8pWMZSyj2ZbRbEszYxnNExnNtoxlLA/kK7ksZ0xpYTUarVM0l61NFr4mSaP1NqDb30p0x8eN1teXrW1dv9GYsgjAHpRw35QBKFmdj58S/QTsnX6iFJN+TSm6x/p8O+vz7Rya+XlJLswROS4HpT+/yHB+lg25LVdN6kXv9mXuGa3LlsFk3VXJ8IbkV8PJgf2ttwRddKEXvIM6lHDflAEoVd2PnxL9BOyZfqIEllK07dE8lK/kg7XNn70gOfV9tY0H9qKE+6YMQKnqfvyU6Cdgz/QTdfL0PQAAAAAqZykFAAAAQOUspQAAAACoXKPZbDarHDg8PJyBgYGkkRwyr8rJLY9vTppjSaMnOXhu9fNlkKG0DHXPT5Ktm5I0k6GhofT399cTIvX3U1LG8ag7Q93zZZBhvBI6Sj/JUMp8GcrKoJ9aSjgWMshQyvxSMky0n+pbSgGMU8xSCmAPivhHH8Ae6CegVPvqp/refc+ZUjLIUESGuucnO7foxfCbvq7/mZRBhl0V1VH6qesz1D1fhrIy6KeWEo6FDDKUMr+UDBPtp9qWUgcflSx/qPq5185Ptv60dWDqmC+DDKVlqHt+klwzr1Wcpairn5IyjkfdGeqeL4MM45XUUfpJhrrny1BWBv3UUsKxkEGGUuaXkmGi/eSFzgEAAAConKUUAAAAAJWzlAIAAACgcpZSAAAAAFSuvnffY8basjFZtyYZWp888VhywJxk4Phk0QXJ7IX7//xSMgBPVsJ989AsyEtyQY7M8Xla5uSXeSyPZH1uy5o8msFqQgDFKaGfSsgAlKeEbighA/WwlGLCNq1N7r882XhD660lk6Q5mjRmtT6+5/3J0UuTxZckc8/Y/+aXkgF4shLum8fnjJydi3NylqaZsSRJT3oy9uuPl+b9uT9fzs25POvz7ekJARSnhH4qIQNQnhK6oYQM1MvT99inZjO577LkhjOTwRuTNFtF0Rz99de3f9xMNt6YfPllrWJpNveP+aVkAJ6slPvm2bk4l2RtTsqS9KQns9KbWelNY5ePe9KTk/OaXJJb88pcNLUBgOKU0E8lZADKU0I3lJCBMlhKsU8PXJHc8e7Wx81tT33d7V+//ZLW7faH+aVkAJ6shPvmK3NR/lcuS5LMygFPed3tXz8vl1tMwX6uhH4qIQNQnhK6oYQMlMFSiqe0aW3rzt+J2y9JNt86s+eXkgF4shLum8fnjJyXyzu67Xm5PMfnpZMPARSnhH4qIQNQnhK6oYQMlKOtpdT73//+NBqN3S4nnHDCdGWjAPdfnjQ6fOWxRm/r9jN5fikZ2Df91H1KuG+enYszmic6uu1onnC2VBfRUd2lhH4qIQMzg37qLiV0QwkZKEfbPwonnnhibr755p3foNdrpe+vtmxsveBcOnzebnNb8pMvJ1sGk9kLZt78UjIwcfqpe5Rw3zw0C3Jylqanw5OOZ+WALM5rc2jm59E81FkIZhQd1R1K6KcSMjCz6KfuUEI3lJCBsrT9SLq3tzdHHXXUjssznvGM6chFAdat2fkOCJ1q9CTrrpqZ80vJwMTpp+5Rwn3zJblgx7vsdaqZsbwkF07qezBz6KjuUEI/lZCBmUU/dYcSuqGEDJSl7R+H9evXZ968eTn22GOzfPnybNy48SmvPzIykuHh4d0uzAxD66fm+wxvmJnzS8nAxOmn7lHCffPIHD8FCZo5IsdNwfdhJmino/TTzFVCP5WQgZlFP3WHErqhhAyUpa2l1G/91m9lzZo1uemmm7J69er853/+Z1760pfmscce2+ttVq1alYGBgR2XBQucYzdTPPHYzrfk7FRzNPlVh/+dqnt+KRmYGP3UXUq4bz4tczp+6t52PZmVg9I/qe/BzNBuR+mnmauEfiohAzOHfuoeJXRDCRkoS1uPppcsWZLzzjsvixcvzm//9m/n//2//5ef//zn+fznP7/X26xcuTJDQ0M7LoODg5MOTTUOmJM0Zk3uezRmJQd2+O+tuueXkoGJ0U/dpYT75i/zWMYm+fS9sYzmF/Goqhu021H6aeYqoZ9KyMDMoZ+6RwndUEIGyjKpV7B7+tOfnt/4jd/Ihg17P3eur68vfX19kxlDTQam4pkpSfo7fGZK3fNLyUBn9NP+rYT75iOZivPPG/lZnH/ejfbVUfpp5iqhn0rIwMyln/ZfJXRDCRkoy6Sed7Bly5b8x3/8R+bOnTtVeSjIoguS5uROAkhzLFnU4Wv41j2/lAx0Rj/t30q4b96WNWlM8ul7jfTktnilzm6ko/ZfJfRTCRmYufTT/quEbighA2Vp69H0JZdckrVr1+bBBx/Mbbfdlt/5nd/JrFmz8oY3vGG68lGj2QuThUuTRofn0zV6k6OXdf5WnXXPLyUDE6OfuksJ981HM5gHckNG80RHtx/NE7k/X8qjeajzEMwYOqp7lNBPJWRg5tBP3aOEbighA2Vpayn10EMP5Q1veEMWLVqU3/3d383hhx+e22+/PUccccR05aNmp1ySNLd1dtvmaLL44pk9v5QM7Jt+6j4l3De/nssyKwd0dNuezMrNuWLyIZgRdFR3KaGfSsjAzKCfuksJ3VBCBsrR1n7ys5/97HTloFBzz0hOuyy5/ZL2b3vaR1u3n8nzS8nAvumn7lPCfXN9vp0v5OKcl8vbvu0/5d1Zn29PPgQzgo7qLiX0UwkZmBn0U3cpoRtKyEA5JvdiGHSFky9qlUay79Mst3/9tMtat9sf5peSAXiyEu6bN+eKfCGtX9nt66l827/+hVzsLCnYz5XQTyVkAMpTQjeUkIEyWEqxT41G6xTJZWuTha9J0mi9Def2t/Lc8XGj9fVla1vXbzT2j/mlZACerJT75s25IpfljDyQr2QsYxnNtoxmW5oZy2ieyGi2ZSxjeSBfyWU5w0IKukAJ/VRCBqA8JXRDCRkoQ4cvL0Y3mntG67JlMFl3VTK8IfnVcHJgf+stORddOL0vOFf3/FIyAE9Wwn1zfb6d9fl2Ds38vCQX5ogcl4PSn19kOD/LhtyWq7yoOXShEvqphAxAeUrohhIyUC9LKdo2e0Fy6vu6d34pGYAnK+G++WgeylfywXpDAMUpoZ9KyACUp4RuKCED9fD0PQAAAAAqZykFAAAAQOUspQAAAAConKUUAAAAAJVrNJvNZpUDh4eHMzAwkDSSQ+ZVObnl8c1Jcyxp9CQHz61+vgwylJah7vlJsnVTkmYyNDSU/v7+ekKk/n5KyjgedWeoe74MMoxXQkfpJxlKmS9DWRn0U0sJx0IGGUqZX0qGifZTfUspgHGKWUoB7EER/+gD2AP9BJRqX/3UW2GW3TlTSgYZishQ9/xk5xa9GH7T1/U/kzLIsKuiOko/dX2GuufLUFYG/dRSwrGQQYZS5peSYaL9VNtS6uCjkuUPVT/32vnJ1p+2Dkwd82WQobQMdc9PkmvmtYqzFHX1U1LG8ag7Q93zZZBhvJI6Sj/JUPd8GcrKoJ9aSjgWMshQyvxSMky0n7zQOQAAAACVs5QCAAAAoHKWUgAAAABUzlIKAAAAgMrV9+57AABAZbZsTNatSYbWJ088lhwwJxk4Pll0QTJ7Yd3pgG6mn7qXpRQAAOzHNq1N7r882XhD6+3Bk6Q5mjRmtT6+5/3J0UuTxZckc8+oLSbQhfQTnr4HAAD7oWYzue+y5IYzk8EbkzRb/9hrjv7669s/biYbb0y+/LLWPw6bzRpDA11BP7GdpRQAAOyHHrgiuePdrY+b2576utu/fvslrdsBTCf9xHZtL6V++tOf5o1vfGMOP/zwHHTQQTn55JNz9913T0c2gLboJ6BkOooqbVrb+gdcJ26/JNl869TmoWz6iSrpJ3bV1mtKPfroozn99NNz1lln5cYbb8wRRxyR9evX59BDD52ufAATop+Akukoqnb/5Umjd99nIOxJo7d1e6/f0h30E1XTT+yqraXURz7ykSxYsCBXXXXVjj875phjpjwUQLv0E1AyHUWVtmxsvWhwOnztlea25CdfTrYMJrMXTGk0CqSfqJJ+Yry2nr73pS99KS94wQty3nnn5cgjj8zznve8fOpTn5qubAATpp+AkukoqrRuzc53sepUoydZd9W+r8fMp5+okn5ivLZ+HH784x9n9erVOf744/PVr341b3nLW/KOd7wjV1999V5vMzIykuHh4d0uAFNNPwEla7ej9BOTMbR+ar7P8Iap+T6UTT9RJf3EeG09fW9sbCwveMELcumllyZJnve85+X73/9+/vZv/zbnn3/+Hm+zatWqfOADH5h8UoCnoJ+AkrXbUfqJyXjisZ1vq96p5mjyK7uGrqCfqJJ+Yry2zpSaO3dunvvc5+72Z7/5m7+ZjRs37vU2K1euzNDQ0I7L4OBgZ0kBnoJ+AkrWbkfpJybjgDlJY9bkvkdjVnJg/9TkoWz6iSrpJ8Zr60yp008/PevWrdvtz370ox/l6KOP3utt+vr60tfX11k6gAnST0DJ2u0o/cRkDBw/Nd+n/7ip+T6UTT9RJf3EeG2dKfUnf/Inuf3223PppZdmw4YNue666/L3f//3WbFixXTlA5gQ/QSUTEdRpUUXJM2xyX2P5liy6MIpiUPh9BNV0k+M19ZS6oUvfGGuv/76fOYzn8lJJ52UD37wg/nYxz6W5cuXT1c+gAnRT0DJdBRVmr0wWbg0abT1nIidGr3J0cu83Xq30E9UST8xXts/CkuXLs3SpUunIwvApOgnoGQ6iiqdckmy8cud3bY5miy+eGrzUDb9RJX0E7tq60wpAACgfHPPSE67rLPbnvbR1u0BpoN+YleWUgAAsB86+aKd//Db11Nltn/9tMtatwOYTvqJ7SylAABgP9RotJ7msmxtsvA1SRqtt1Lf/nbsOz5utL6+bG3r+o1GnamBbqCf2K7DlxcDAABmgrlntC5bBpN1VyXDG5JfDScH9rfeVn3RhV40GKiHfsJSCgAAusDsBcmp76s7BcCT6afu5el7AAAAAFTOUgoAAACAyllKAQAAAFA5SykAAAAAKtdoNpvNKgcODw9nYGAgaSSHzKtycsvjm5PmWNLoSQ6eW/18GWQoLUPd85Nk66YkzWRoaCj9/f31hEj9/ZSUcTzqzlD3fBlkGK+EjtJPMpQyX4ayMuinlhKOhQwylDK/lAwT7af6llIA4xSzlALYgyL+0QewB/oJKNW++qm3wiy7c6aUDDIUkaHu+cnOLXox/Kav638mZZBhV0V1lH7q+gx1z5ehrAz6qaWEYyGDDKXMLyXDRPuptqXUwUclyx+qfu6185OtP20dmDrmyyBDaRnqnp8k18xrFWcp6uqnpIzjUXeGuufLIMN4JXWUfpKh7vkylJVBP7WUcCxkkKGU+aVkmGg/eaFzAAAAACpnKQUAAABA5SylAAAAAKicpRQAAAAAlbOUAgAAAKByllIAAAAAVM5SCgAAAIDKWUoBAAAAULm2llLPfvaz02g0nnRZsWLFdOUDmDAdBZRKPwGl0k9AnXrbufJdd92V0dHRHZ9///vfz9lnn53zzjtvyoMBtEtHAaXST0Cp9BNQp7aWUkccccRun3/4wx/Oc57znLzsZS+b0lAAndBRQKn0E1Aq/QTUqePXlPrVr36Va665Jm9+85vTaDSmMhPApOkooFT6CSiVfgKq1taZUrv64he/mJ///Oe54IILnvJ6IyMjGRkZ2fH58PBwpyMBJmwiHaWfgDroJ6BU+gmoWsdnSl155ZVZsmRJ5s2b95TXW7VqVQYGBnZcFixY0OlIgAmbSEfpJ6AO+gkolX4CqtbRUuonP/lJbr755vzhH/7hPq+7cuXKDA0N7bgMDg52MhJgwibaUfoJqJp+Akqln4A6dPT0vauuuipHHnlkzjnnnH1et6+vL319fZ2MAejIRDtKPwFV009AqfQTUIe2z5QaGxvLVVddlfPPPz+9vR2/JBXAtNBRQKn0E1Aq/QTUpe2l1M0335yNGzfmzW9+83TkAZgUHQWUSj8BpdJPQF3aXoO/6lWvSrPZnI4sAJOmo4BS6SegVPoJqEvH774HAAAAAJ2ylAIAAACgcpZSAAAAAFTOUgoAAACAyllKAQAAAFA5SykAAAAAKmcpBQAAAEDlLKUAAAAAqJylFAAAAACVazSbzWaVA4eHhzMwMJA0kkPmVTm55fHNSXMsafQkB8+tfr4MMpSWoe75SbJ1U5JmMjQ0lP7+/npCpP5+Sso4HnVnqHu+DDKMV0JH6ScZSpkvQ1kZ9FNLCcdCBhlKmV9Khon2U31LKYBxillKAexBEf/oA9gD/QSUal/91Fthlt05U0oGGYrIUPf8ZOcWvRh+09f1P5MyyLCrojpKP3V9hrrny1BWBv3UUsKxkEGGUuaXkmGi/VTbUurgo5LlD1U/99r5ydaftg5MHfNlkKG0DHXPT5Jr5rWKsxR19VNSxvGoO0Pd82WQYbySOko/yVD3fBnKyqCfWko4FjLIUMr8UjJMtJ+80DkAAAAAlbOUAgAAAKByllIAAAAAVM5SCgAAAIDKWUoBAAAAUDlLKQAAAAAqZykFAAAAQOUspQAAAACoXFtLqdHR0bz3ve/NMccck4MOOijPec5z8sEPfjDNZnO68gFMiH4CSqajgFLpJ6BOve1c+SMf+UhWr16dq6++OieeeGLuvvvuXHjhhRkYGMg73vGO6coIsE/6CSiZjgJKpZ+AOrW1lLrtttty7rnn5pxzzkmSPPvZz85nPvOZ3HnnndMSDmCi9BNQMh0FlEo/AXVq6+l7L3nJS/KNb3wjP/rRj5Ik9913X/71X/81S5Ys2ettRkZGMjw8vNsFYKrpJ6Bk7XaUfgKqop+AOrV1ptR73vOeDA8P54QTTsisWbMyOjqaD33oQ1m+fPleb7Nq1ap84AMfmHRQgKein4CStdtR+gmoin4C6tTWmVKf//znc+211+a6667Ld7/73Vx99dW57LLLcvXVV+/1NitXrszQ0NCOy+Dg4KRDA4ynn4CStdtR+gmoin4C6tTWmVLvfve78573vCe///u/nyQ5+eST85Of/CSrVq3K+eefv8fb9PX1pa+vb/JJAZ6CfgJK1m5H6SegKvoJqFNbZ0o9/vjj6enZ/SazZs3K2NjYlIYCaJd+Akqmo4BS6SegTm2dKbVs2bJ86EMfysKFC3PiiSfme9/7Xq644oq8+c1vnq58ABOin4CS6SigVPoJqFNbS6m/+Zu/yXvf+9689a1vzSOPPJJ58+blj/7oj/K+971vuvIBTIh+Akqmo4BS6SegTm0tpebMmZOPfexj+djHPjZNcQA6o5+AkukooFT6CahTW68pBQAAAABTwVIKAAAAgMpZSgEAAABQOUspAAAAACpnKQUAAABA5SylAAAAAKicpRQAAAAAlbOUAgAAAKByjWaz2axy4NDQUJ7+9KcnSQ6eW+XklscfTtJM0kgOPqr6+TLIUFqGuucnyeObW//785//PAMDA/WESP39lBRyPPxMyiDD7hkK6Cj9JEMp82UoLIN+SlLIsZBBhkLmF5Nhgv1U+VLqoYceyoIFC6ocCcwQg4ODmT9/fm3z9RPwVOrsKP0EPBX9BJRqX/1U+VJqbGwsmzZtypw5c9JoNNq+/fDwcBYsWJDBwcH09/dPQ0IZZkqGuufLMHUZms1mHnvsscybNy89PfU9q1g/ybA/Zah7/v6UoYSOmmw/JfUfj7rnyyBDaRn00051H4sSMtQ9XwYZpjrDRPupdzIhO9HT0zMlW/z+/v7aDo4MZWWoe74MU5OhzqftbaefZNgfM9Q9f3/JUHdHTVU/JfUfj7rnyyBDaRn00051H4sSMtQ9XwYZpjLDRPrJC50DAAAAUDlLKQAAAAAqN+OWUn19ffmLv/iL9PX1ydDlGeqeL0NZGUpQwt+DDDKUMl+G8tT9d1H3fBlkKC1D3fNLUsLfRd0Z6p4vgwx1Zaj8hc4BAAAAYMadKQUAAADAzGcpBQAAAEDlLKUAAAAAqJylFAAAAACVm1FLqe985zuZNWtWzjnnnMpnX3DBBWk0Gjsuhx9+eF796lfn/vvvrzzLww8/nLe//e059thj09fXlwULFmTZsmX5xje+Me2zd/17OOCAA/LMZz4zZ599dj796U9nbGxs2uePz7Dr5dWvfnUl8/eVY8OGDZXMf/jhh/POd74zxx13XJ72tKflmc98Zk4//fSsXr06jz/++LTPv+CCC/K6173uSX9+yy23pNFo5Oc///m0ZyiNjtJP43PU1VF191NSb0fppyfTT/ppfA795DFUKfSTfhqfQz91Vz/NqKXUlVdembe//e259dZbs2nTpsrnv/rVr87mzZuzefPmfOMb30hvb2+WLl1aaYYHH3wwp556ar75zW/mox/9aB544IHcdNNNOeuss7JixYpKMmz/e3jwwQdz44035qyzzso73/nOLF26NNu2bas0w66Xz3zmM5XM3leOY445Ztrn/vjHP87znve8fO1rX8ull16a733ve/nOd76TP/3TP80NN9yQm2++edoz8GTd3lH66ck56uyouvop0VEl0k/6aXwO/aSfSqGf9NP4HPqpu/qpt+4AE7Vly5Z87nOfy913352HH344a9asyZ//+Z9XmqGvry9HHXVUkuSoo47Ke97znrz0pS/Nz372sxxxxBGVZHjrW9+aRqORO++8M4cccsiOPz/xxBPz5je/uZIMu/49POtZz8rzn//8nHbaaXnFK16RNWvW5A//8A8rzVCnunK89a1vTW9vb+6+++7dfg6OPfbYnHvuuWk2m5Vn6nY6Sj/tLUdd6sygo8qin/TT3nLURT+xnX7ST3vLURf9VL0Zc6bU5z//+ZxwwglZtGhR3vjGN+bTn/50rQdly5Ytueaaa3Lcccfl8MMPr2Tm//zP/+Smm27KihUrdvsh3e7pT396JTn25OUvf3lOOeWU/PM//3NtGbrFf//3f+drX/vaXn8OkqTRaFScim7vKP3EdjqqPPpJP9Gin8qjn/QTLd3cTzNmKXXllVfmjW98Y5LWKXVDQ0NZu3ZtpRluuOGGzJ49O7Nnz86cOXPypS99KZ/73OfS01PNX+OGDRvSbDZzwgknVDKvXSeccEIefPDBSmbteiy2Xy699NJKZj9VjvPOO2/aZ27/OVi0aNFuf/6MZzxjR44/+7M/m/YcyZ6Pw5IlSyqZXZpu7yj9tLsSOqqOfkrK6Sj9tJN+0k+70k/191Oio7bTT/ppV/qpO/tpRjx9b926dbnzzjtz/fXXJ0l6e3vze7/3e7nyyitz5plnVpbjrLPOyurVq5Mkjz76aD75yU9myZIlufPOO3P00UdP+/zST9drNpuVbW93PRbbHXbYYZXMfqoce9tqV+HOO+/M2NhYli9fnpGRkUpm7uk43HHHHTseXHQLHaWfxiuho0rqp6T6jtJPLfpJP42nn57MY6h66Cf9NJ5+erJu6KcZsZS68sors23btsybN2/HnzWbzfT19eXjH/94BgYGKslxyCGH5Ljjjtvx+T/8wz9kYGAgn/rUp/JXf/VX0z7/+OOPT6PRyA9/+MNpn9WJH/zgB5W9CNz4Y1GXOnIcd9xxaTQaWbdu3W5/fuyxxyZJDjrooMqy7On//0MPPVTZ/FLoKP00XgkdVVeGUjpKP7XoJ/00nn6qv58SHZXop0Q/jaefurOfin/63rZt2/KP//iPufzyy3PvvffuuNx3332ZN29eLe+4tl2j0UhPT09+8YtfVDLvsMMOy2//9m/nE5/4RLZu3fqkr9f59rHf/OY388ADD+T1r399bRm6xeGHH56zzz47H//4x/f4c0C1dFSLfmI7HVUO/dSin9hOP5VDP7XoJ7br5n4q/kypG264IY8++mj+4A/+4Enb8te//vW58sor88d//MeVZBkZGcnDDz+cpHVq58c//vFs2bIly5Ytq2R+knziE5/I6aefnhe96EX5y7/8yyxevDjbtm3L17/+9axevTo/+MEPpj3D9r+H0dHR/Nd//VduuummrFq1KkuXLs2b3vSmaZ+/a4Zd9fb25hnPeEYl8+v2yU9+Mqeffnpe8IIX5P3vf38WL16cnp6e3HXXXfnhD3+YU089te6IXUNH7aSfnpxjVzpKR1VNP+2kn56cY1f6ST9VTT/tpJ+enGNX+qkL+qlZuKVLlzZf85rX7PFrd9xxRzNJ87777pv2HOeff34zyY7LnDlzmi984Qub//RP/zTts8fbtGlTc8WKFc2jjz66eeCBBzaf9axnNV/72tc2v/Wtb0377F3/Hnp7e5tHHHFE85WvfGXz05/+dHN0dHTa54/PsOtl0aJFlczfNce5555b6cxdbdq0qfm2t72tecwxxzQPOOCA5uzZs5svetGLmh/96EebW7dunfb5e/v//61vfauZpPnoo49Oe4YS6KjddXs/jc9RV0fV3U/NZr0dpZ9a9NPu9JN+2s5jqPrpp93pJ/20XTf2U6PZLPzV1QAAAADY7xT/mlIAAAAA7H8spQAAAAConKUUAAAAAJWzlAIAAACgcpZSAAAAAFTOUgoAAACAyllKAQAAAFA5SykAAAAAKmcpBQAAAEDlLKUAAAAAqJylFAAAAACVs5QCAAAAoHL/P+MeJ8jHvWnsAAAAAElFTkSuQmCC\n"
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def single_turn(\n",
    "    current_boards: np, policy: GamePolicy\n",
    ") -> tuple[np.ndarray, np.ndarray]:\n",
    "    \"\"\"Execute a single turn on a board.\n",
    "\n",
    "    Places a new stone on the board. Turns captured enemy stones.\n",
    "\n",
    "    Args:\n",
    "        current_boards: The current board before the game.\n",
    "        policy: The game policy to be used.\n",
    "\n",
    "    Returns:\n",
    "        The new game board and the policy vector containing the index of the action used.\n",
    "    \"\"\"\n",
    "    policy_results = policy.get_policy(current_boards)\n",
    "\n",
    "    # if the constant VERIFY_POLICY is set to true the policy is verified. Should be good though.\n",
    "    # todo deactivate the policy verification after some testing.\n",
    "    if VERIFY_POLICY:\n",
    "        assert np.all(moves_possible(current_boards, policy_results)), (\n",
    "            current_boards[(moves_possible(current_boards, policy_results) == False)],\n",
    "            policy_results[(moves_possible(current_boards, policy_results) == False)],\n",
    "            np.where(moves_possible(current_boards, policy_results) == False),\n",
    "        )\n",
    "    return do_moves(current_boards, policy_results), policy_results\n",
    "\n",
    "\n",
    "%timeit single_turn(get_new_games(EXAMPLE_STACK_SIZE), RandomPolicy(1))\n",
    "VERIFY_POLICY = False  # type: ignore\n",
    "%timeit single_turn(get_new_games(EXAMPLE_STACK_SIZE), RandomPolicy(1))\n",
    "VERIFY_POLICY = True  # type: ignore\n",
    "plot_othello_boards(\n",
    "    single_turn(get_new_games(EXAMPLE_STACK_SIZE), RandomPolicy(1))[0][:8]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Simulate a stack of games\n",
    "This function will simulate a stack of games and return an array of policies and histories."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Exception in thread Thread-5 (_handle_workers):\n",
      "Traceback (most recent call last):\n",
      "  File \"C:\\Program Files\\Python310\\lib\\threading.py\", line 1016, in _bootstrap_inner\n",
      "    self.run()\n",
      "  File \"C:\\Program Files\\Python310\\lib\\threading.py\", line 953, in run\n",
      "    self._target(*self._args, **self._kwargs)\n",
      "  File \"C:\\Program Files\\Python310\\lib\\multiprocessing\\pool.py\", line 516, in _handle_workers\n",
      "    cls._maintain_pool(ctx, Process, processes, pool, inqueue,\n",
      "  File \"C:\\Program Files\\Python310\\lib\\multiprocessing\\pool.py\", line 340, in _maintain_pool\n",
      "    Pool._repopulate_pool_static(ctx, Process, processes, pool,\n",
      "  File \"C:\\Program Files\\Python310\\lib\\multiprocessing\\pool.py\", line 329, in _repopulate_pool_static\n",
      "    w.start()\n",
      "  File \"C:\\Program Files\\Python310\\lib\\multiprocessing\\process.py\", line 121, in start\n",
      "    self._popen = self._Popen(self)\n",
      "  File \"C:\\Program Files\\Python310\\lib\\multiprocessing\\context.py\", line 336, in _Popen\n",
      "    return Popen(process_obj)\n",
      "  File \"C:\\Program Files\\Python310\\lib\\multiprocessing\\popen_spawn_win32.py\", line 93, in __init__\n",
      "    reduction.dump(process_obj, to_child)\n",
      "  File \"C:\\Program Files\\Python310\\lib\\multiprocessing\\reduction.py\", line 60, in dump\n",
      "    ForkingPickler(file, protocol).dump(obj)\n",
      "  File \"C:\\Program Files\\Python310\\lib\\multiprocessing\\synchronize.py\", line 104, in __getstate__\n",
      "    h = context.get_spawning_popen().duplicate_for_child(sl.handle)\n",
      "  File \"C:\\Program Files\\Python310\\lib\\multiprocessing\\popen_spawn_win32.py\", line 99, in duplicate_for_child\n",
      "    return reduction.duplicate(handle, self.sentinel)\n",
      "  File \"C:\\Program Files\\Python310\\lib\\multiprocessing\\reduction.py\", line 79, in duplicate\n",
      "    return _winapi.DuplicateHandle(\n",
      "PermissionError: [WinError 5] Zugriff verweigert\n"
     ]
    }
   ],
   "source": [
    "from tqdm.notebook import tqdm\n",
    "\n",
    "\n",
    "def simulate_game(\n",
    "    nr_of_games: int,\n",
    "    policies: tuple[GamePolicy, GamePolicy],\n",
    ") -> tuple[np.ndarray, np.ndarray]:\n",
    "    \"\"\"Simulates a stack of games.\n",
    "\n",
    "    Args:\n",
    "        nr_of_games: The number of games that should be simulated.\n",
    "        policies: The policies that should be used to simulate the game.\n",
    "\n",
    "    Returns:\n",
    "        A stack of board histories and actions.\n",
    "    \"\"\"\n",
    "    board_history_stack = np.zeros((SIMULATE_TURNS, nr_of_games, 8, 8))\n",
    "    action_history_stack = np.zeros((SIMULATE_TURNS, nr_of_games, 2))\n",
    "    current_boards = get_new_games(nr_of_games)\n",
    "    for turn_index in range(SIMULATE_TURNS):\n",
    "        policy_index = turn_index % 2\n",
    "        policy = policies[policy_index]\n",
    "        board_history_stack[turn_index] = current_boards\n",
    "        if policy_index == 0:\n",
    "            current_boards = current_boards * -1\n",
    "        current_boards, action_taken = single_turn(current_boards, policy)\n",
    "        action_history_stack[turn_index] = action_taken\n",
    "\n",
    "        if policy_index == 0:\n",
    "            current_boards = current_boards * -1\n",
    "\n",
    "    return board_history_stack, action_history_stack\n",
    "\n",
    "\n",
    "simulation_results = simulate_game(1, (RandomPolicy(1), RandomPolicy(1)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "\n",
    "%timeit simulate_game(100, (RandomPolicy(1), RandomPolicy(1)))\n",
    "# simulate_game(EXAMPLE_STACK_SIZE, (RandomPolicy(1), RandomPolicy(1)))"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "policies_to_use = RandomPolicy(1), RandomPolicy(1)\n",
    "with Pool(3) as pool:\n",
    "    results = pool.map(simulate_game, [100, policies_to_use])"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "is_executing": true
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "a = np.array(\n",
    "    [\n",
    "        [\n",
    "            [-1, -1, -1, -1, 0, 0, 0, 0],\n",
    "            [1, 1, -1, 1, 1, 0, 0, 0],\n",
    "            [1, 1, -1, 1, 1, 1, 0, 0],\n",
    "            [0, 1, -1, 1, 1, 1, 0, 0],\n",
    "            [0, 1, 1, 1, 1, 1, 0, 0],\n",
    "            [-1, 1, 1, 1, 1, 0, 0, 0],\n",
    "            [0, 0, 0, 1, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "        ]\n",
    "    ],\n",
    "    dtype=int,\n",
    ")\n",
    "a"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "is_executing": true
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "is_executing": true
    }
   },
   "outputs": [],
   "source": [
    "RandomPolicy(1).get_policy(a)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "is_executing": true
    }
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "\n",
    "def create_test_game():\n",
    "    test_array = [\n",
    "        [\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 1, 2, 0, 0, 0],\n",
    "            [0, 0, 0, 2, 1, 0, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "        ],\n",
    "        [\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 2, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 2, 2, 0, 0, 0],\n",
    "            [0, 0, 0, 2, 1, 0, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "        ],\n",
    "        [\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 2, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 2, 2, 0, 0, 0],\n",
    "            [0, 0, 1, 1, 1, 0, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "        ],\n",
    "        [\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 2, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 2, 2, 0, 0, 0],\n",
    "            [0, 0, 2, 1, 1, 0, 0, 0],\n",
    "            [0, 2, 0, 0, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "        ],\n",
    "        [\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 1, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 1, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 1, 2, 0, 0, 0],\n",
    "            [0, 0, 2, 1, 1, 0, 0, 0],\n",
    "            [0, 2, 0, 0, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "        ],\n",
    "        [\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "            [0, 0, 2, 1, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 2, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 1, 2, 0, 0, 0],\n",
    "            [0, 0, 2, 1, 1, 0, 0, 0],\n",
    "            [0, 2, 0, 0, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "        ],\n",
    "        [\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "            [0, 0, 2, 1, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 2, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 1, 2, 0, 0, 0],\n",
    "            [0, 0, 2, 2, 2, 2, 0, 0],\n",
    "            [0, 2, 0, 0, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "        ],\n",
    "        [\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "            [0, 0, 2, 1, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 2, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 1, 1, 1, 0, 0],\n",
    "            [0, 0, 2, 2, 2, 2, 0, 0],\n",
    "            [0, 2, 0, 0, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "        ],\n",
    "        [\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "            [0, 0, 2, 1, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 2, 0, 2, 0, 0],\n",
    "            [0, 0, 0, 1, 2, 2, 0, 0],\n",
    "            [0, 0, 2, 2, 2, 2, 0, 0],\n",
    "            [0, 2, 0, 0, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "        ],\n",
    "        [\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "            [0, 0, 2, 1, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 2, 0, 2, 0, 0],\n",
    "            [0, 0, 0, 1, 2, 2, 0, 0],\n",
    "            [0, 0, 2, 2, 1, 2, 0, 0],\n",
    "            [0, 2, 0, 0, 0, 1, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "        ],\n",
    "        [\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "            [0, 0, 2, 1, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 2, 0, 2, 0, 0],\n",
    "            [0, 0, 0, 1, 2, 2, 0, 0],\n",
    "            [0, 0, 2, 2, 1, 2, 0, 0],\n",
    "            [0, 2, 0, 0, 0, 2, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 2, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "        ],\n",
    "        [\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "            [0, 0, 2, 1, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 2, 0, 2, 0, 0],\n",
    "            [0, 0, 0, 1, 2, 2, 0, 0],\n",
    "            [0, 1, 1, 1, 1, 2, 0, 0],\n",
    "            [0, 2, 0, 0, 0, 2, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 2, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "        ],\n",
    "        [\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "            [0, 0, 2, 1, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 2, 0, 2, 0, 0],\n",
    "            [0, 0, 0, 1, 2, 2, 0, 0],\n",
    "            [2, 2, 2, 2, 2, 2, 0, 0],\n",
    "            [0, 2, 0, 0, 0, 2, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 2, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "        ],\n",
    "        [\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "            [0, 0, 2, 1, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 2, 0, 2, 0, 0],\n",
    "            [0, 0, 0, 1, 1, 1, 1, 0],\n",
    "            [2, 2, 2, 2, 2, 2, 0, 0],\n",
    "            [0, 2, 0, 0, 0, 2, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 2, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "        ],\n",
    "        [\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "            [0, 0, 2, 1, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 2, 0, 2, 0, 0],\n",
    "            [0, 0, 0, 1, 1, 1, 1, 0],\n",
    "            [2, 2, 2, 1, 2, 2, 0, 0],\n",
    "            [0, 2, 0, 1, 0, 2, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 2, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "        ],\n",
    "        [\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "            [0, 0, 2, 1, 0, 0, 0, 0],\n",
    "            [0, 0, 0, 2, 2, 2, 0, 0],\n",
    "            [0, 0, 0, 2, 2, 1, 1, 0],\n",
    "            [2, 2, 2, 1, 2, 2, 0, 0],\n",
    "            [0, 2, 0, 1, 0, 2, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 2, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "        ],\n",
    "        [\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "            [0, 0, 2, 1, 0, 1, 0, 0],\n",
    "            [0, 0, 0, 2, 2, 1, 0, 0],\n",
    "            [0, 0, 0, 2, 2, 1, 1, 0],\n",
    "            [2, 2, 2, 1, 2, 2, 0, 0],\n",
    "            [0, 2, 0, 1, 0, 2, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 2, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "        ],\n",
    "        [\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "            [0, 0, 2, 1, 0, 1, 0, 0],\n",
    "            [0, 0, 0, 2, 2, 2, 2, 0],\n",
    "            [0, 0, 0, 2, 2, 2, 1, 0],\n",
    "            [2, 2, 2, 1, 2, 2, 0, 0],\n",
    "            [0, 2, 0, 1, 0, 2, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 2, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "        ],\n",
    "        [\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "            [0, 0, 2, 1, 0, 1, 0, 0],\n",
    "            [0, 0, 0, 2, 1, 2, 2, 0],\n",
    "            [0, 0, 0, 2, 2, 1, 1, 0],\n",
    "            [2, 2, 2, 1, 1, 1, 1, 0],\n",
    "            [0, 2, 0, 1, 0, 2, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 2, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "        ],\n",
    "        [\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "            [0, 0, 2, 1, 0, 1, 0, 0],\n",
    "            [0, 0, 0, 2, 1, 2, 2, 0],\n",
    "            [0, 0, 0, 2, 2, 1, 2, 0],\n",
    "            [2, 2, 2, 2, 2, 2, 2, 2],\n",
    "            [0, 2, 0, 1, 0, 2, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 2, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "        ],\n",
    "        [\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "            [0, 0, 2, 1, 0, 1, 0, 0],\n",
    "            [0, 0, 0, 2, 1, 2, 2, 0],\n",
    "            [0, 0, 0, 2, 1, 1, 2, 0],\n",
    "            [2, 2, 2, 2, 1, 2, 2, 2],\n",
    "            [0, 2, 0, 1, 1, 2, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 2, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "        ],\n",
    "        [\n",
    "            [0, 0, 0, 0, 2, 0, 0, 0],\n",
    "            [0, 0, 2, 2, 0, 2, 0, 0],\n",
    "            [0, 0, 0, 2, 1, 2, 2, 0],\n",
    "            [0, 0, 0, 2, 1, 1, 2, 0],\n",
    "            [2, 2, 2, 2, 1, 2, 2, 2],\n",
    "            [0, 2, 0, 1, 1, 2, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 2, 0, 0],\n",
    "            [0, 0, 0, 0, 0, 0, 0, 0],\n",
    "        ],\n",
    "    ]\n",
    "    test_array = np.array(test_array)\n",
    "\n",
    "    # swapp 2 by one. 2 was only there for homogenous formating and easier readability while coading.\n",
    "    test_array[test_array == 2] = -1\n",
    "    assert np.all(\n",
    "        np.count_nonzero(test_array, axis=(1, 2))\n",
    "        == np.arange(4, 4 + test_array.shape[0])\n",
    "    )\n",
    "\n",
    "    # validated that only one stone is added per turn\n",
    "    zero_array = test_array == 0\n",
    "    diff = zero_array != np.roll(zero_array, 1, axis=0)\n",
    "    turns = np.where(diff[1:])\n",
    "    arr = np.array(turns)[0]\n",
    "    assert len(arr) == len(set(arr))\n",
    "\n",
    "    return test_array"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "is_executing": true
    }
   },
   "outputs": [],
   "source": [
    "plot_othello_boards(create_test_game()[-3:])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "is_executing": true
    }
   },
   "outputs": [],
   "source": [
    "array = create_test_game()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Sources\n",
    "\n",
    "* Game rules and example board images [https://en.wikipedia.org/wiki/Reversi](https://en.wikipedia.org/wiki/Reversi)\n",
    "* Game rules and example game images [https://de.wikipedia.org/wiki/Othello_(Spiel)](https://de.wikipedia.org/wiki/Othello_(Spiel))\n",
    "* Game strategy examples [https://de.wikipedia.org/wiki/Computer-Othello](https://de.wikipedia.org/wiki/Computer-Othello)\n",
    "* Image for 8 directions [https://www.researchgate.net/journal/EURASIP-Journal-on-Image-and-Video-Processing-1687-5281](https://www.researchgate.net/journal/EURASIP-Journal-on-Image-and-Video-Processing-1687-5281)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "is_executing": true
    }
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}