1"""
  2
  3This is the documentation of Cooperative Cuisine.
  4
  5# About the package
  6
  7The package contains an environment for cooperation between players/agents. A PyGameGUI visualizes the game to
  8human or virtual agents in 2D. A 3D web-enabled version (for example for online studies, currently under development)
  9can be found [here](https://gitlab.ub.uni-bielefeld.de/scs/cocosy/godot-overcooked-3d-visualization).
 10
 11For a guided introduction to specific features, have a look at the `cooperative_cuisine.tutorials` page.
 12
 13
 14# Background / Literature
 15The overcooked/cooking domain is a well-established cooperation domain/task. There exists
 16environments designed for reinforcement learning agents as well as the game and adaptations of the game for human
 17players in a more "real-time"-like environment. They all mostly differ in the visual and graphics dimension. 2D
 18versions like [overcooked-ai](https://github.com/HumanCompatibleAI/overcooked_ai), [gym-cooking](
 19https://github.com/rosewang2008/gym-cooking) are the most well-known in the community. Besides, the general
 20adaptations of the original overcooked game. Cooperative Cuisine, we want to bring both worlds together: the
 21reinforcement learning and real-time playable environment with an appealing visualization. Enable the potential of
 22developing artificial agents that play with humans like a "real", cooperative, human partner.
 23
 24# Installation
 25
 26### Local Editable Installation
 27Easy with conda or mamba:
 28```bash
 29git clone https://gitlab.ub.uni-bielefeld.de/scs/cocosy/cooperative-cuisine.git
 30conda env create -f cooperative-cuisine/conda.recipe/environment.yaml
 31conda activate cocu
 32```
 33
 34Otherwise, you need to make sure graphviz is installed. Also, you can install it via pip directly:
 35```bash
 36conda install -c conda-forge pygraphviz
 37git clone https://gitlab.ub.uni-bielefeld.de/scs/cocosy/cooperative-cuisine.git
 38cd cooperative-cuisine
 39pip install -e .
 40```
 41With these installs, you can just checkout to other branches with git (and if dependencies do not change without `pip install -e .`), e.g., `git checkout dev`, and add new configs, images, etc. and change the code.
 42
 43
 44### Library Installation
 45The correct environment needs to be active:
 46
 47```bash
 48conda install -c conda-forge pygraphviz  # or find another way to install graphviz on your machine
 49pip install cooperative_cuisine@git+https://gitlab.ub.uni-bielefeld.de/scs/cocosy/cooperative-cuisine@main
 50```
 51You can now use the environment and/or simulator in your python code. Just by importing
 52it `import cooperative_cuisine`
 53
 54### Agent Installation
 55If you want to play with some agents, install them via
 56```bash
 57pip install cocu_agents@git+https://gitlab.ub.uni-bielefeld.de/scs/cocosy/cocu-agents@main
 58```
 59
 60
 61# Usage / Examples
 62 Our Cooperative Cuisine is designed for real time interaction but also for reinforcement
 63learning (gymnasium environment) with time independent step function calls. It focuses on configurability, extensibility and appealing visualization
 64options.
 65
 66## Human Player
 67Run it via the command line (in your pyenv/conda environment):
 68
 69```bash
 70cooperative-cuisine start  -s localhost -sp 8080 -g localhost -gp 8000
 71```
 72
 73If npm is installed, a web demo using this client will automatically be served by the study server (`--web`). The PixiJS visualization client can also be easily included in external projects. For more information, see the [README.md](cooperative_cuisine/pixijs_2d_vis/README.md) in the `pixijs_2d_vis` directory.
 74
 75
 76*The arguments shown are the defaults.*
 77
 78You can also start the **Game Server**, **Study Server** (Matchmaking),and the **PyGame GUI** individually in different terminals.
 79
 80```bash
 81cooperative-cuisine game-server -g localhost -gp 8000 --manager-ids SECRETKEY1 SECRETKEY2
 82
 83cooperative-cuisine study-server -s localhost -sp 8080 -g localhost -gp 8000 --manager-ids SECRETKEY1
 84
 85cooperative-cuisine gui -s localhost -sp 8080
 86```
 87
 88For more CLI arguments and a description of possible arguments, run `cooperative-cuisine -h`
 89
 90## Connect with agent and receive game state
 91Or you start a game server, create an environment and connect each player/agent via a websocket connection.
 92
 93To start a game server see above. Your own manager needs to create an environment.
 94```python
 95import requests
 96
 97from cooperative_cuisine import ROOT_DIR
 98from cooperative_cuisine.game_server import CreateEnvironmentConfig
 99from cooperative_cuisine.server_results import CreateEnvResult
100
101
102with open(ROOT_DIR / "configs" / "item_info.yaml", "r", encoding="utf-8") as file:
103    item_info = file.read()
104with open(ROOT_DIR / "configs" / "layouts" / "basic.layout", "r", encoding="utf-8") as file:
105    layout = file.read()
106with open(ROOT_DIR / "configs" / "environment_config.yaml", "r", encoding="utf-8") as file:
107    environment_config = file.read()
108
109create_env = CreateEnvironmentConfig(
110    manager_id="SECRETKEY1",
111    number_players=2,
112    environment_settings={"all_player_can_pause_game": False},
113    item_info_config=item_info,
114    environment_config=environment_config,
115    layout_config=layout,
116    seed=123456789,
117).model_dump(mode="json")
118
119post_result = requests.post("http://localhost:8000/manage/create_env", json=create_env)
120if post_result.status_code == 403:
121    raise ValueError(f"Forbidden Request: {post_result.json()['detail']}")
122env_info: CreateEnvResult = post_result.json()
123```
124
125Connect each player via a websocket (threaded or async).
126```python
127import json
128import dataclasses
129from websockets.sync.client import connect
130
131from cooperative_cuisine.action import Action, ActionType, InterActionData
132from cooperative_cuisine.utils import custom_asdict_factory
133
134
135p1_websocket = connect("ws://localhost:8000/ws/player/" + env_info["player_info"]["0"]["client_id"])
136
137# set player "0" as ready
138p1_websocket.send(json.dumps({"type": "ready", "player_hash": env_info["player_info"]["0"]["player_hash"]}))
139assert json.loads(p1_websocket.recv())["status"] == 200, "not accepted player"
140
141
142# get the state for player "0", call it on every frame/step
143p1_websocket.send(json.dumps({"type": "get_state", "player_hash": env_info["player_info"]["0"]["player_hash"]}))
144state = json.loads(p1_websocket.recv())
145
146# send an action for player "0"
147# --- movement ---
148action = Action(
149    player="0",
150    action_type=ActionType.MOVEMENT,
151    action_data=[0.0, 1.0],  # direction (here straight up)
152    duration=0.5  # seconds
153)
154# --- pickup/drop off ---
155action = Action(
156    player="0",
157    action_type=ActionType.PICK_UP_DROP,
158    action_data=None,
159)
160# --- interact ---
161action = Action(
162    player="0",
163    action_type=ActionType.INTERACT,
164    action_data=InterActionData.START  # InterActionData.STOP when to stop the interaction
165)
166
167p1_websocket.send(json.dumps({
168    "type": "action",
169    "player_hash": env_info["player_info"]["0"]["player_hash"],
170    "action": dataclasses.asdict(
171        action, dict_factory=custom_asdict_factory
172    ),
173}))
174p1_websocket.recv()
175
176```
177
178Stop the environment if you want the game to end before the time is up.
179```python
180requests.post(
181    "http://localhost:8000/manage/stop_env",
182    json={
183        "manager_id": "SECRETKEY1",
184        "env_id": env_info["env_id"],
185        "reason": "closed environment",
186    },
187)
188```
189
190## Direct integration into your code
191You can use the `cooperative_cuisine.environment.Environment` class
192and call the `step`, `get_state`, and `perform_action` methods directly.
193
194```python
195import json
196from datetime import timedelta
197
198from cooperative_cuisine import ROOT_DIR
199from cooperative_cuisine.action import Action
200from cooperative_cuisine.environment import Environment
201
202env = Environment(
203    env_config=ROOT_DIR / "configs" / "environment_config.yaml",
204    layout_config=ROOT_DIR / "configs" / "layouts" / "basic.layout",
205    item_info=ROOT_DIR / "configs" / "item_info.yaml"
206)
207
208env.add_player("0")
209env.add_player("1")
210
211while True:
212    # adapt this to real time if needed with time.sleep etc.
213    env.step(timedelta(seconds=0.1))
214
215    player_0_state = env.get_state("0")
216    if player_0_state["ended"]:
217        break
218
219    action = Action(...)  # Please refer to the above code but remember to use np.array instead of list for the movement direction vector
220    env.perform_action(action)
221```
222
223# JSON State
224The JSON schema for the state of the environment for a player can be generated by running the `state_representation.py`
225```bash
226python state_representation.py
227```
228Should look like - to have an interactive view, have a look at [jsonschemaviewer](https://navneethg.github.io/jsonschemaviewer/):
229```
230{"$defs": {"CookingEquipmentState": {"description": "Format of the state representation of cooking equipment.", "properties": {"id": {"title": "Id", "type": "string"}, "category": {"anyOf": [{"const": "Item"}, {"const": "ItemCookingEquipment"}], "title": "Category"}, "type": {"title": "Type", "type": "string"}, "progress_percentage": {"anyOf": [{"type": "number"}, {"type": "integer"}], "title": "Progress Percentage"}, "inverse_progress": {"title": "Inverse Progress", "type": "boolean"}, "active_effects": {"items": {"$ref": "#/$defs/EffectState"}, "title": "Active Effects", "type": "array"}, "content_list": {"items": {"$ref": "#/$defs/ItemState"}, "title": "Content List", "type": "array"}, "content_ready": {"anyOf": [{"$ref": "#/$defs/ItemState"}, {"type": "null"}]}}, "required": ["id", "category", "type", "progress_percentage", "inverse_progress", "active_effects", "content_list", "content_ready"], "title": "CookingEquipmentState", "type": "object"}, "CounterState": {"description": "Format of the state representation of a counter.", "properties": {"id": {"title": "Id", "type": "string"}, "category": {"const": "Counter", "title": "Category"}, "type": {"title": "Type", "type": "string"}, "pos": {"items": {"type": "number"}, "title": "Pos", "type": "array"}, "orientation": {"items": {"type": "number"}, "title": "Orientation", "type": "array"}, "occupied_by": {"anyOf": [{"items": {"anyOf": [{"$ref": "#/$defs/ItemState"}, {"$ref": "#/$defs/CookingEquipmentState"}]}, "type": "array"}, {"$ref": "#/$defs/ItemState"}, {"$ref": "#/$defs/CookingEquipmentState"}, {"type": "null"}], "title": "Occupied By"}, "active_effects": {"items": {"$ref": "#/$defs/EffectState"}, "title": "Active Effects", "type": "array"}}, "required": ["id", "category", "type", "pos", "orientation", "occupied_by", "active_effects"], "title": "CounterState", "type": "object"}, "EffectState": {"description": "Format of the state representation of an effect (fire).", "properties": {"id": {"title": "Id", "type": "string"}, "type": {"title": "Type", "type": "string"}, "progress_percentage": {"anyOf": [{"type": "number"}, {"type": "integer"}], "title": "Progress Percentage"}, "inverse_progress": {"title": "Inverse Progress", "type": "boolean"}}, "required": ["id", "type", "progress_percentage", "inverse_progress"], "title": "EffectState", "type": "object"}, "ItemState": {"description": "Format of the state representation of an item.", "properties": {"id": {"title": "Id", "type": "string"}, "category": {"anyOf": [{"const": "Item"}, {"const": "ItemCookingEquipment"}], "title": "Category"}, "type": {"title": "Type", "type": "string"}, "progress_percentage": {"anyOf": [{"type": "number"}, {"type": "integer"}], "title": "Progress Percentage"}, "inverse_progress": {"title": "Inverse Progress", "type": "boolean"}, "active_effects": {"items": {"$ref": "#/$defs/EffectState"}, "title": "Active Effects", "type": "array"}}, "required": ["id", "category", "type", "progress_percentage", "inverse_progress", "active_effects"], "title": "ItemState", "type": "object"}, "KitchenInfo": {"description": "Format of the state representation of basic information of the kitchen.", "properties": {"width": {"title": "Width", "type": "number"}, "height": {"title": "Height", "type": "number"}}, "required": ["width", "height"], "title": "KitchenInfo", "type": "object"}, "OrderState": {"description": "Format of the state representation of an order.", "properties": {"id": {"title": "Id", "type": "string"}, "category": {"const": "Order", "title": "Category"}, "meal": {"title": "Meal", "type": "string"}, "start_time": {"anyOf": [{"format": "date-time", "type": "string"}, {"type": "string"}], "title": "Start Time"}, "max_duration": {"title": "Max Duration", "type": "number"}, "score": {"anyOf": [{"type": "number"}, {"type": "integer"}], "title": "Score"}}, "required": ["id", "category", "meal", "start_time", "max_duration", "score"], "title": "OrderState", "type": "object"}, "PlayerState": {"description": "Format of the state representation of a player.", "properties": {"id": {"title": "Id", "type": "string"}, "pos": {"items": {"type": "number"}, "title": "Pos", "type": "array"}, "facing_direction": {"items": {"type": "number"}, "title": "Facing Direction", "type": "array"}, "holding": {"anyOf": [{"$ref": "#/$defs/ItemState"}, {"$ref": "#/$defs/CookingEquipmentState"}, {"type": "null"}], "title": "Holding"}, "current_nearest_counter_pos": {"anyOf": [{"items": {"type": "number"}, "type": "array"}, {"type": "null"}], "title": "Current Nearest Counter Pos"}, "current_nearest_counter_id": {"anyOf": [{"type": "string"}, {"type": "null"}], "title": "Current Nearest Counter Id"}}, "required": ["id", "pos", "facing_direction", "holding", "current_nearest_counter_pos", "current_nearest_counter_id"], "title": "PlayerState", "type": "object"}, "ViewRestriction": {"description": "Format of the state representation of a view restriction from the players perspectives.\nCurrently, as a view cone, like a flashlight in the dark.", "properties": {"direction": {"items": {"type": "number"}, "title": "Direction", "type": "array"}, "position": {"items": {"type": "number"}, "title": "Position", "type": "array"}, "angle": {"title": "Angle", "type": "integer"}, "counter_mask": {"anyOf": [{"items": {"type": "boolean"}, "type": "array"}, {"type": "null"}], "title": "Counter Mask"}, "range": {"anyOf": [{"type": "number"}, {"type": "null"}], "title": "Range"}}, "required": ["direction", "position", "angle", "counter_mask", "range"], "title": "ViewRestriction", "type": "object"}}, "description": "The format of the returned state representation.", "properties": {"players": {"items": {"$ref": "#/$defs/PlayerState"}, "title": "Players", "type": "array"}, "counters": {"items": {"$ref": "#/$defs/CounterState"}, "title": "Counters", "type": "array"}, "kitchen": {"$ref": "#/$defs/KitchenInfo"}, "score": {"anyOf": [{"type": "number"}, {"type": "integer"}], "title": "Score"}, "orders": {"items": {"$ref": "#/$defs/OrderState"}, "title": "Orders", "type": "array"}, "ended": {"title": "Ended", "type": "boolean"}, "env_time": {"format": "date-time", "title": "Env Time", "type": "string"}, "remaining_time": {"title": "Remaining Time", "type": "number"}, "view_restrictions": {"anyOf": [{"items": {"$ref": "#/$defs/ViewRestriction"}, "type": "array"}, {"type": "null"}], "title": "View Restrictions"}, "served_meals": {"items": {"maxItems": 2, "minItems": 2, "prefixItems": [{"type": "string"}, {"type": "string"}], "type": "array"}, "title": "Served Meals", "type": "array"}, "info_msg": {"items": {"maxItems": 2, "minItems": 2, "prefixItems": [{"type": "string"}, {"type": "string"}], "type": "array"}, "title": "Info Msg", "type": "array"}}, "required": ["players", "counters", "kitchen", "score", "orders", "ended", "env_time", "remaining_time", "view_restrictions", "served_meals", "info_msg"], "title": "StateRepresentation", "type": "object"}
231```
232
233The BaseModel and TypedDicts can be found in `cooperative_cuisine.state_representation`. The
234`cooperative_cuisine.state_representation.StateRepresentation` represents the json state that the `get_state`
235returns.
236
237## Generate images from JSON states
238You might have stored some json states and now you want to visualize them via the
239pygame-2d visualization. You can do that by running the `drawing.py` script and referencing a json file.
240```bash
241cooperative-cuisine screenshot --state my_state.json
242```
243- You can specify a different visualization config with `-v` or `--visualization_config`.
244- You can specify the name of the output file with `-o` or `--output_file`. The default is `screenshot.jpg`.
245
246## Generate images/videos from recordings
247You can record json states or only the actions and the environment config via hooks and the recording class.
248
249If you want to generate images or complete videos, see `cooperative_cuisine.pygame_2d_vis.video_replay`.
250
251# Configuration
252
253The environment configuration is currently done with 3 config files + GUI configuration.
254
255## Item Config
256
257The item config defines which ingredients, cooking equipment and meals can exist and how meals and processed ingredients
258can be cooked/created.
259
260For example
261
262    CuttingBoard:
263      type: Equipment
264
265    Stove:
266      type: Equipment
267
268    Pot:
269      type: Equipment
270      equipment: Stove
271
272    Tomato:
273      type: Ingredient
274
275    ChoppedTomato:
276      type: Ingredient
277      needs: [ Tomato ]
278      seconds: 4.0
279      equipment: CuttingBoard
280
281    TomatoSoup:
282      type: Meal
283      needs: [ ChoppedTomato, ChoppedTomato, ChoppedTomato ]
284      seconds: 6.0
285      equipment: Pot
286
287
288## Layout Config
289
290You can define the layout of the kitchen via a layout file. The position of counters are based on a grid system, even
291when the players do not move grid steps but continuous steps. Each character defines a different type of counter. Which
292character is mapped to which counter is defined in the Environment config.
293
294For example
295
296```
297#QU#FO#TNLB#
298@__________M
299|__________K
300$__________I
301#__A____A__D
302C__________E
303#__________G
304C__________#
305##PS+#X##S+#
306```
307
308## Environment Config
309
310The environment config details how a level/environment is defined. Here, the available plates, meals, order and player
311configuration is done.
312
313For example
314
315```yaml
316plates:
317  clean_plates: 1
318  dirty_plates: 2
319  plate_delay: [ 5, 10 ]
320  # range of seconds until the dirty plate arrives.
321
322game:
323  time_limit_seconds: 300
324  undo_dispenser_pickup: true
325  validate_recipes: true
326
327layout_chars:
328  _: Free
329  hash: Counter
330  A: Agent
331  P: PlateDispenser
332  C: CuttingBoard
333  X: Trashcan
334  W: ServingWindow
335  S: Sink
336  +: SinkAddon
337  U: Pot  # with Stove
338  T: Tomato
339
340orders:  # how to create orders
341  meals:
342    all: true
343  ...
344
345player_config:
346  radius: 0.4
347  speed_units_per_seconds: 6
348  interaction_range: 1.6
349  restricted_view: False
350  view_angle: 70
351  view_range: 4  # in grid units, can be "null"
352
353effect_manager:  # fire effect
354  ...
355
356hook_callbacks:  # scores, recording, msgs, etc.
357  ...
358```
359
360## PyGame Visualization Config
361
362Here the visualisation for all objects is defined. Reference the images or define a list of base shapes that represent
363the counters, ingredients, meals and players. Cooperative Cuisine comes with images for the defined meals.
364You can extend it easily. Just have a look at the default [`visualisation.yml`](https://gitlab.ub.uni-bielefeld.de/scs/cocosy/cooperative-cuisine/-/blob/main/cooperative_cuisine/pygame_2d_vis/visualization.yaml?ref_type=heads).
365Further, here, you can configure the size of the visualization like screen width, button sizes, fps, colors, etc.
366
367You can change the localisation (language) of the visualization by changing the `language` key in the visualization
368config, we provide "en" and "de" translations. You can add own languages by adding i18 translation files to the
369`pygame_2d_vis/locals` folder and `continue_{lang}.png`, `controls_{lang}.png`, `try_{lang}.png` images in the `pygame_2d_vis/gui_images` folder (we provide a `tutorial.drawio` template file.
370You can edit it online at [draw.io](https://app.diagrams.net/)).
371
372## Study Config
373
374You can setup a study with a study config.
375It defines which levels the player will play after they connect to the study server.
376Further, you define how many players play together within an environment.
377
378```yaml
379levels:
380  - config_path: STUDY_DIR/level1/level1_config.yaml
381    layout_path: LAYOUTS_DIR/overcooked-1/1-1-far-apart.layout
382    item_info_path: STUDY_DIR/level1/level1_item_info.yaml
383    name: "Level 1-1: Far Apart"
384
385  - config_path: CONFIGS_DIR/environment_config.yaml
386    layout_path: LAYOUTS_DIR/basic.layout
387    item_info_path: CONFIGS_DIR/item_info.yaml
388    name: "Basic"
389
390  - config_path: STUDY_DIR/level2/level2_config.yaml
391    layout_path: LAYOUTS_DIR/overcooked-1/1-4-bottleneck.layout
392    item_info_path: STUDY_DIR/level2/level2_item_info.yaml
393    name: "Level 1-4: Bottleneck"
394
395num_players: 1
396num_bots: 0
397```
398`STUDY_DIR`, `LAYOUTS_DIR`, `CONFIG_DIR`, etc. will be replaced accordingly to their names.
399
400# Citation
401
402# Structure of the Documentation
403The API documentation follows the file and content structure in the repo.
404On the left you can find the navigation panel that brings you to the implementation of
405- the **action**s the player can perform,
406- the **argument parser** definitions for the cli,
407- the **base agent** class, for abstracting some agent logic (e.g. path planning),
408- the **counter factory** converts the characters in the layout file to counter instances,
409- the **counters**, including the kitchen utility objects like dispenser, cooking counter (stove, deep fryer, oven),
410  sink, etc.,
411- the **effect manager**, how the fire effect is defined,
412- the **environment**, handles the incoming actions and provides the state,
413- the **game server**, which can manage several running environments and can communicates via FastAPI post requests and
414websockets,
415- the **hooks**, you can adapt and extend recording, scoring, msgs to players, etc. via hooks,
416- the **hook addons** to extend the environment for custom features,
417- the **info msgs** to players, based on hook events, short text msgs can be generated,
418- the game **items**, the holdable ingredients, cooking equipment, composed ingredients, and meals,
419- the calculation of player **movement**,
420- the **orders**, how to sample incoming orders and their attributes,
421- the **player**/agent, that interacts in the environment,
422- the **pygame 2d visualization**, GUI, drawing, and video generation,
423- the **pixijs 2d visualization** for web apps,
424- the **recording**, via hooks, actions, environment configs, states, etc. can be recorded in files,
425- the **scores**, via hooks, events can affect the scores,
426- type hints are defined in **state representation** for the json state and **server results** for the data returned by
427the game server in post requests.
428- the **study server**, the match making server,
429- **util**ity code,
430- the config **validation** and graph generation,
431- the interface for different **voice chat** APIs (currently custom build CoCuVc und Mumble).
432
433# License
434Cooperative Cuisine © 2025 by [Social Cognitive Systems Group](https://scs.techfak.uni-bielefeld.de/) is licensed under [`CC BY-NC-SA 4.0`](https://creativecommons.org/licenses/by-nc-sa/4.0/?ref=chooser-v1) [![License: CC BY-NC-SA 4.0](https://licensebuttons.net/l/by-nc-sa/4.0/80x15.png)](https://creativecommons.org/licenses/by-nc-sa/4.0/)
435"""
436import os
437from pathlib import Path
438
439
440ROOT_DIR = Path(os.path.dirname(os.path.abspath(__file__)))  # This is your Project Root
441"""A path variable to get access to the layouts coming with the package. For example,
442```python 
443from cooperative_cuisine import ROOT_DIR
444
445environment_config_path = ROOT_DIR / "configs" / "environment_config.yaml"
446```
447"""