Merge pull request #25 from CogitoNTNU/dev

Keeping main up to date with dev

docs/guide/venv.md

@@ -0,0 +1,25 @@
+# Development Environment
+
+We will be utilizing a virtual environment to develop the project. This will allow us to have a consistent development environment across all developers.In this case we will be using `venv` to create the virtual environment.
+
+## Getting Started
+
+* Have [Python](https://www.python.org/downloads/) installed
+  * Verify that python is installed by running `python --version`
+* Pip install the virtual environment package
+  * Verify that pip is installed by running `pip --version`
+  * Install the virtual environment package by running `pip install virtualenv`
+* In the root of the project, create a virtual environment by running `python -m venv .venv`
+* Activate the virtual environment
+  * On Windows, run `.venv\Scripts\activate`
+  * On Mac/Linux, run `source .venv/bin/activate`
+* Done! You should now have a fully functional development environment
+* To deactivate the virtual environment, run `deactivate`
+
+## Dependencies
+Once you have entered venv you need to make sure the dependencies are installed by running `pip install -r requirements.txt`.
+If you use a new pip dependency, make sure to add it to the `requirements.txt` file. This can be done by running:
+```bash
+pip freeze > requirements.txt
+``` 
+after you pip installed it locally, and then committing the changes.

docs/planning/12.03.2024.md

@@ -0,0 +1,14 @@
+# Cogito work night nr. 3
+
+## Agenda
+
+- Venv
+- [Mer klarere plan for Tetris](#tetris)
+- Progge
+- Hva skjer neste uke?
+
+
+## Tetris
+- **Headless:** ingen grafikk ting skjer sekvensielt. En viss boardstate og en ny block -> velger presist hvor den skal plasseres (blant mulige plasser) uten tidsbegrensning.
+  For development purpuses har vi en to print outs av brettet. Første viser brettet med den uplasserte blokken (øverst i midten), andre viser brettet etter at blokken er plassert. Bruker de objektene som vi har blitt enige om. Tanken er at vi kan bruke dette til å teste ut forskjellige algoritmer for å plassere blokken (ai/algorytme velger posisjonen).
+- **Grafisk:** pygame. Adapsjon av samme objekter som headless bare at vi nå styrer hvor blokken skal plasseres dynamisk. Blokken faller nedover med en viss hastighet og vi kan flytte den rundt med en viss hastighet (feks. et tastetrykk = forflytter blokken en rute). For å la agenten spille må vi lage et oversettelses lag mellom headless og grafisk, hvor vi kan sende input til grafisk for å manuvrere blokken til samme posisjon som headless ville plassert den.

docs/planning/agentDesign.md

@@ -0,0 +1,47 @@
+# Agent design
+
+## Environment
+
+When designing an AI agent for playing Tetris, it's essential to understand the task environment's properties. Here's a breakdown of these properties according to common AI environment classifications:
+
+### Observability (Fully Observable vs. Partially Observable)
+
+**Fully Observable:** In Tetris, the AI can see the entire game board at all times, including the positions and shapes of all placed pieces. It also knows the current piece and usually the next piece(s) to come. Therefore, Tetris is considered a fully observable environment.
+
+### Determinism (Deterministic vs. Stochastic)
+
+**Deterministic**
+The environment in Tetris is deterministic because the next state of the environment is completely determined by the current state and the action executed by the agent. There is no randomness in how the pieces respond to control inputs; however, the sequence of the Tetris pieces themselves can introduce some level of unpredictability if the sequence generation is not known in advance, yet the piece manipulation is deterministic.
+
+### Episodic vs. Sequential
+
+**Sequential:**
+Tetris is sequential because the decisions an agent makes depend on the previous actions, and these decisions accumulate over time to affect the final outcome. Each move affects future opportunities and challenges.
+
+### Static vs. Dynamic
+
+**Semi-Dynamic:**
+While the game board itself does not change while the agent is making a decision (the timer stops if the piece is not moved), the game introduces new pieces continually, which requires dynamic planning and response. Therefore, it's static during decision-making but dynamic overall because the game progresses with the addition of new pieces.
+
+### Discrete vs. Continuous
+
+**Discrete:** Both the time in which decisions are made and the actions available (e.g., moving pieces left, right, rotating them, or dropping them) are discrete. The game operates in steps defined by piece movements and placements.
+
+### Single-agent vs. Multi-agent
+
+**Single-agent:** Although some versions of Tetris feature competitive play against other players, the standard environment for Tetris is a single-agent setting where the AI competes against the game environment itself, not against other agents.
+Understanding these properties can significantly influence how you design your AI agent, from the decision-making algorithms used (like heuristic functions in deterministic environments) to handling the unpredictability of piece sequence and managing the game's progression.
+
+## Agent types
+
+Here are some suggested agent types for playing Tetris:
+
+- **Random Agent**: Makes random moves without any strategy.
+- **Heuristic Agent**: Uses a set of predefined rules or heuristics to evaluate the game state and make decisions. Usually makes the greedy choice based on the current state.
+- **Search-based Agent**: Uses search algorithms like iterative deepening, or Monte Carlo Tree Search to explore possible future states and make decisions based on the search results.
+- **Reinforcement Learning Agent**: Learns to play Tetris through trial and error, adjusting its strategy based on rewards and penalties received during gameplay.
+- Genetic Algorithm Agent: Uses genetic algorithms to evolve a population of agents over time, selecting the best-performing agents for reproduction and mutation.
+
+## Thoughts
+
+When doing search based agents it is worth to experiment with giving it foresight into several moves ahead to see if it can make better decisions. And maybe try different reward distributions to see if it can reliebly set up for tetrises.

main.py

@@ -1,8 +1,54 @@
-from src.game.TetrisGameManager import TetrisGameManager
-from src.game.board import Board
-
+from src.game.tetris import Tetris
+from src.game.TetrisGameManager import *
+from src.agents.agent import Agent, play_game
+from src.agents.agent_factory import create_agent
+from src.agents.heuristic import (
+    utility
+)
+from src.agents.heuristic_trainer import train
+from src.agents.geneticAlgAgentJon import GeneticAlgAgentJM
 
+def test():
+    # algAgent = GeneticAlgAgentJM()
+    # algAgent.number_of_selection(2)
+    # print(algAgent.getBestPop())
+
+    board = Tetris()
+    agent = create_agent("heuristic")
+    manager = TetrisGameManager(board)
+    manager.startDemo(agent)
+
 if __name__ == "__main__":
-    board = Board()
-    game = TetrisGameManager(board)
-    game.startGame()
+
+    # game = Tetris()
+    # agent: Agent = create_agent("heuristic")
+    # sum_rows_removed = 0
+    # for i in range(10):
+    #     end_board = play_game(agent, game, 7)
+    #     end_board.printBoard()
+    #     sum_rows_removed += end_board.rowsRemoved
+
+    # print(f"Average rows removed: {sum_rows_removed / 10}")
+
+    # possible_moves = game.getPossibleBoards()
+    # for boards in possible_moves:
+    #     print(utility(boards, 0, -1, 0, 0, 0))
+    #     boards.printBoard()
+
+    # board = Tetris()
+    # manager = TetrisGameManager(board)
+    # agent = create_agent("heuristic")
+
+    # # manager.startGame()
+
+    # # train()
+
+
+    # algAgent = GeneticAlgAgentJM()
+    # algAgent.number_of_selection(2)
+    # print(algAgent.getBestPop())
+
+    test()
+
+
+    # cProfile.run('main()', 'restats')  

src/agents/agent.py

@@ -6,7 +6,8 @@ class for all agents in the simulation.
 from abc import ABC, abstractmethod
 from typing import Any, Union
 
-from src.game.board import Action, Board
+from src.game.tetris import Action, Tetris
+from time import sleep
 
 
 class Agent(ABC):
@@ -21,7 +22,7 @@ def __subclasscheck__(cls, subclass: Any) -> bool:
         return hasattr(subclass, "result") and callable(subclass.result)
 
     @abstractmethod
-    def result(board: Board) -> Union[Action, list[Action]]:
+    def result(board: Tetris) -> Union[Action, list[Action]]:
         """
         Determines the next move for the agent based on the current state of the board.
 
@@ -34,7 +35,7 @@ def result(board: Board) -> Union[Action, list[Action]]:
         pass
 
 
-def play_game(agent: Agent, board: Board, actions_per_drop: int = 1) -> Board:
+def play_game(agent: Agent, board: Tetris, actions_per_drop: int = 1) -> Tetris:
     """
     Plays a game of Tetris with the given agent.
 
@@ -46,6 +47,8 @@ def play_game(agent: Agent, board: Board, actions_per_drop: int = 1) -> Board:
     Returns:
         The final state of the board after the game is over.
     """
+    #count = 0
+
     while not board.isGameOver():
         # Get the result of the agent's action
         for _ in range(actions_per_drop):
@@ -56,8 +59,38 @@ def play_game(agent: Agent, board: Board, actions_per_drop: int = 1) -> Board:
                     board.doAction(action)
             else:
                 board.doAction(result)
+
+            #count += 1
         # Advance the game by one frame
         board.doAction(Action.SOFT_DROP)
-        board.printBoard()
+        if board.blockHasLanded:
+            board.updateBoard()
+        #board.printBoard()
 
     return board
+
+def playGameDemoStepByStep(agent: Agent, board: Tetris) -> Tetris:
+    """
+    Plays a game of Tetris with the given agent where actions are slowed down for demonstration purposes.
+
+    Args:
+        agent (Agent): The agent to play the game.
+        board (Board): The initial state of the board.
+    """
+
+    # Get the result of the agent's action
+    result = agent.result(board)
+
+    if Action.HARD_DROP in result:
+        result.remove(Action.HARD_DROP)
+        result.append([Action.SOFT_DROP] * 20)
+    # Perform the action(s) on the board
+    if isinstance(result, list):
+        for action in result:
+            board.doAction(action, demo=True)
+    else:
+        board.doAction(action, demo=True)
+    # Advance the game by one frame
+    board.doAction(Action.SOFT_DROP)
+    if board.blockHasLanded:
+        board.updateBoard()

src/agents/agent_factory.py

@@ -1,13 +1,17 @@
 """ This module contains the factory function for creating agents. """
 
 from src.agents.agent import Agent
-from src.agents.randomAgent import RandomAgent
+from src.agents.random_agent import RandomAgent
+from src.agents.heuristic_agent import HeuristicAgent
+
 
 def create_agent(agent_type: str) -> Agent:
-    """ Create an agent of the specified type. """
+    """Create an agent of the specified type."""
 
-    match agent_type.lower():
-        case 'random':
-            return RandomAgent()
-        case _:
-            raise ValueError(f'Unknown agent type: {agent_type}')
+    if agent_type.lower() == "random":
+        return RandomAgent()
+    elif agent_type.lower() == "heuristic":
+        hyperparameters = [1,1,1,1,1]
+        return HeuristicAgent()
+    else:
+        raise ValueError(f"Unknown agent type: {agent_type}")

src/agents/geneticAlgAgent.py

@@ -0,0 +1,8 @@
+# From paper: https://codemyroad.wordpress.com/2013/04/14/tetris-ai-the-near-perfect-player/
+# the weigts the author got:
+#  a x (Aggregate Height) + b x (Complete Lines) + c x (Holes) + d x (Bumpiness)
+# a = -0.510066 b = 0.760666 c = -0.35663 d = -0.184483
+# TODO Read the part of the article about the genetic algorithm
+# TODO Create a fitness function
+# TODO Create a genetic algorithm based on the
+