Major update, made single main file for both multi and single agent, added argsparse, polished everything

This commit is contained in:
Vasilis Valatsos 2023-11-29 11:53:30 +01:00
parent 6a84d0b3f4
commit 84000dd28b
46 changed files with 1590 additions and 744 deletions

3
.gitignore vendored
View file

@ -161,3 +161,6 @@ cython_debug/
# Mac bs
.DS_store
# Random stuff
__pycache__/

386
LICENSE
View file

@ -1,21 +1,373 @@
MIT License
Mozilla Public License Version 2.0
==================================
Copyright (c) 2023 Vasilis Valatsos
1. Definitions
--------------
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
1.1. "Contributor"
means each individual or legal entity that creates, contributes to
the creation of, or owns Covered Software.
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
1.2. "Contributor Version"
means the combination of the Contributions of others (if any) used
by a Contributor and that particular Contributor's Contribution.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
1.3. "Contribution"
means Covered Software of a particular Contributor.
1.4. "Covered Software"
means Source Code Form to which the initial Contributor has attached
the notice in Exhibit A, the Executable Form of such Source Code
Form, and Modifications of such Source Code Form, in each case
including portions thereof.
1.5. "Incompatible With Secondary Licenses"
means
(a) that the initial Contributor has attached the notice described
in Exhibit B to the Covered Software; or
(b) that the Covered Software was made available under the terms of
version 1.1 or earlier of the License, but not also under the
terms of a Secondary License.
1.6. "Executable Form"
means any form of the work other than Source Code Form.
1.7. "Larger Work"
means a work that combines Covered Software with other material, in
a separate file or files, that is not Covered Software.
1.8. "License"
means this document.
1.9. "Licensable"
means having the right to grant, to the maximum extent possible,
whether at the time of the initial grant or subsequently, any and
all of the rights conveyed by this License.
1.10. "Modifications"
means any of the following:
(a) any file in Source Code Form that results from an addition to,
deletion from, or modification of the contents of Covered
Software; or
(b) any new file in Source Code Form that contains any Covered
Software.
1.11. "Patent Claims" of a Contributor
means any patent claim(s), including without limitation, method,
process, and apparatus claims, in any patent Licensable by such
Contributor that would be infringed, but for the grant of the
License, by the making, using, selling, offering for sale, having
made, import, or transfer of either its Contributions or its
Contributor Version.
1.12. "Secondary License"
means either the GNU General Public License, Version 2.0, the GNU
Lesser General Public License, Version 2.1, the GNU Affero General
Public License, Version 3.0, or any later versions of those
licenses.
1.13. "Source Code Form"
means the form of the work preferred for making modifications.
1.14. "You" (or "Your")
means an individual or a legal entity exercising rights under this
License. For legal entities, "You" includes any entity that
controls, is controlled by, or is under common control with You. For
purposes of this definition, "control" means (a) the power, direct
or indirect, to cause the direction or management of such entity,
whether by contract or otherwise, or (b) ownership of more than
fifty percent (50%) of the outstanding shares or beneficial
ownership of such entity.
2. License Grants and Conditions
--------------------------------
2.1. Grants
Each Contributor hereby grants You a world-wide, royalty-free,
non-exclusive license:
(a) under intellectual property rights (other than patent or trademark)
Licensable by such Contributor to use, reproduce, make available,
modify, display, perform, distribute, and otherwise exploit its
Contributions, either on an unmodified basis, with Modifications, or
as part of a Larger Work; and
(b) under Patent Claims of such Contributor to make, use, sell, offer
for sale, have made, import, and otherwise transfer either its
Contributions or its Contributor Version.
2.2. Effective Date
The licenses granted in Section 2.1 with respect to any Contribution
become effective for each Contribution on the date the Contributor first
distributes such Contribution.
2.3. Limitations on Grant Scope
The licenses granted in this Section 2 are the only rights granted under
this License. No additional rights or licenses will be implied from the
distribution or licensing of Covered Software under this License.
Notwithstanding Section 2.1(b) above, no patent license is granted by a
Contributor:
(a) for any code that a Contributor has removed from Covered Software;
or
(b) for infringements caused by: (i) Your and any other third party's
modifications of Covered Software, or (ii) the combination of its
Contributions with other software (except as part of its Contributor
Version); or
(c) under Patent Claims infringed by Covered Software in the absence of
its Contributions.
This License does not grant any rights in the trademarks, service marks,
or logos of any Contributor (except as may be necessary to comply with
the notice requirements in Section 3.4).
2.4. Subsequent Licenses
No Contributor makes additional grants as a result of Your choice to
distribute the Covered Software under a subsequent version of this
License (see Section 10.2) or under the terms of a Secondary License (if
permitted under the terms of Section 3.3).
2.5. Representation
Each Contributor represents that the Contributor believes its
Contributions are its original creation(s) or it has sufficient rights
to grant the rights to its Contributions conveyed by this License.
2.6. Fair Use
This License is not intended to limit any rights You have under
applicable copyright doctrines of fair use, fair dealing, or other
equivalents.
2.7. Conditions
Sections 3.1, 3.2, 3.3, and 3.4 are conditions of the licenses granted
in Section 2.1.
3. Responsibilities
-------------------
3.1. Distribution of Source Form
All distribution of Covered Software in Source Code Form, including any
Modifications that You create or to which You contribute, must be under
the terms of this License. You must inform recipients that the Source
Code Form of the Covered Software is governed by the terms of this
License, and how they can obtain a copy of this License. You may not
attempt to alter or restrict the recipients' rights in the Source Code
Form.
3.2. Distribution of Executable Form
If You distribute Covered Software in Executable Form then:
(a) such Covered Software must also be made available in Source Code
Form, as described in Section 3.1, and You must inform recipients of
the Executable Form how they can obtain a copy of such Source Code
Form by reasonable means in a timely manner, at a charge no more
than the cost of distribution to the recipient; and
(b) You may distribute such Executable Form under the terms of this
License, or sublicense it under different terms, provided that the
license for the Executable Form does not attempt to limit or alter
the recipients' rights in the Source Code Form under this License.
3.3. Distribution of a Larger Work
You may create and distribute a Larger Work under terms of Your choice,
provided that You also comply with the requirements of this License for
the Covered Software. If the Larger Work is a combination of Covered
Software with a work governed by one or more Secondary Licenses, and the
Covered Software is not Incompatible With Secondary Licenses, this
License permits You to additionally distribute such Covered Software
under the terms of such Secondary License(s), so that the recipient of
the Larger Work may, at their option, further distribute the Covered
Software under the terms of either this License or such Secondary
License(s).
3.4. Notices
You may not remove or alter the substance of any license notices
(including copyright notices, patent notices, disclaimers of warranty,
or limitations of liability) contained within the Source Code Form of
the Covered Software, except that You may alter any license notices to
the extent required to remedy known factual inaccuracies.
3.5. Application of Additional Terms
You may choose to offer, and to charge a fee for, warranty, support,
indemnity or liability obligations to one or more recipients of Covered
Software. However, You may do so only on Your own behalf, and not on
behalf of any Contributor. You must make it absolutely clear that any
such warranty, support, indemnity, or liability obligation is offered by
You alone, and You hereby agree to indemnify every Contributor for any
liability incurred by such Contributor as a result of warranty, support,
indemnity or liability terms You offer. You may include additional
disclaimers of warranty and limitations of liability specific to any
jurisdiction.
4. Inability to Comply Due to Statute or Regulation
---------------------------------------------------
If it is impossible for You to comply with any of the terms of this
License with respect to some or all of the Covered Software due to
statute, judicial order, or regulation then You must: (a) comply with
the terms of this License to the maximum extent possible; and (b)
describe the limitations and the code they affect. Such description must
be placed in a text file included with all distributions of the Covered
Software under this License. Except to the extent prohibited by statute
or regulation, such description must be sufficiently detailed for a
recipient of ordinary skill to be able to understand it.
5. Termination
--------------
5.1. The rights granted under this License will terminate automatically
if You fail to comply with any of its terms. However, if You become
compliant, then the rights granted under this License from a particular
Contributor are reinstated (a) provisionally, unless and until such
Contributor explicitly and finally terminates Your grants, and (b) on an
ongoing basis, if such Contributor fails to notify You of the
non-compliance by some reasonable means prior to 60 days after You have
come back into compliance. Moreover, Your grants from a particular
Contributor are reinstated on an ongoing basis if such Contributor
notifies You of the non-compliance by some reasonable means, this is the
first time You have received notice of non-compliance with this License
from such Contributor, and You become compliant prior to 30 days after
Your receipt of the notice.
5.2. If You initiate litigation against any entity by asserting a patent
infringement claim (excluding declaratory judgment actions,
counter-claims, and cross-claims) alleging that a Contributor Version
directly or indirectly infringes any patent, then the rights granted to
You by any and all Contributors for the Covered Software under Section
2.1 of this License shall terminate.
5.3. In the event of termination under Sections 5.1 or 5.2 above, all
end user license agreements (excluding distributors and resellers) which
have been validly granted by You or Your distributors under this License
prior to termination shall survive termination.
************************************************************************
* *
* 6. Disclaimer of Warranty *
* ------------------------- *
* *
* Covered Software is provided under this License on an "as is" *
* basis, without warranty of any kind, either expressed, implied, or *
* statutory, including, without limitation, warranties that the *
* Covered Software is free of defects, merchantable, fit for a *
* particular purpose or non-infringing. The entire risk as to the *
* quality and performance of the Covered Software is with You. *
* Should any Covered Software prove defective in any respect, You *
* (not any Contributor) assume the cost of any necessary servicing, *
* repair, or correction. This disclaimer of warranty constitutes an *
* essential part of this License. No use of any Covered Software is *
* authorized under this License except under this disclaimer. *
* *
************************************************************************
************************************************************************
* *
* 7. Limitation of Liability *
* -------------------------- *
* *
* Under no circumstances and under no legal theory, whether tort *
* (including negligence), contract, or otherwise, shall any *
* Contributor, or anyone who distributes Covered Software as *
* permitted above, be liable to You for any direct, indirect, *
* special, incidental, or consequential damages of any character *
* including, without limitation, damages for lost profits, loss of *
* goodwill, work stoppage, computer failure or malfunction, or any *
* and all other commercial damages or losses, even if such party *
* shall have been informed of the possibility of such damages. This *
* limitation of liability shall not apply to liability for death or *
* personal injury resulting from such party's negligence to the *
* extent applicable law prohibits such limitation. Some *
* jurisdictions do not allow the exclusion or limitation of *
* incidental or consequential damages, so this exclusion and *
* limitation may not apply to You. *
* *
************************************************************************
8. Litigation
-------------
Any litigation relating to this License may be brought only in the
courts of a jurisdiction where the defendant maintains its principal
place of business and such litigation shall be governed by laws of that
jurisdiction, without reference to its conflict-of-law provisions.
Nothing in this Section shall prevent a party's ability to bring
cross-claims or counter-claims.
9. Miscellaneous
----------------
This License represents the complete agreement concerning the subject
matter hereof. If any provision of this License is held to be
unenforceable, such provision shall be reformed only to the extent
necessary to make it enforceable. Any law or regulation which provides
that the language of a contract shall be construed against the drafter
shall not be used to construe this License against a Contributor.
10. Versions of the License
---------------------------
10.1. New Versions
Mozilla Foundation is the license steward. Except as provided in Section
10.3, no one other than the license steward has the right to modify or
publish new versions of this License. Each version will be given a
distinguishing version number.
10.2. Effect of New Versions
You may distribute the Covered Software under the terms of the version
of the License under which You originally received the Covered Software,
or under the terms of any subsequent version published by the license
steward.
10.3. Modified Versions
If you create software not governed by this License, and you want to
create a new license for such software, you may create and use a
modified version of this License if you rename the license and remove
any references to the name of the license steward (except to note that
such modified license differs from this License).
10.4. Distributing Source Code Form that is Incompatible With Secondary
Licenses
If You choose to distribute Source Code Form that is Incompatible With
Secondary Licenses under the terms of this version of the License, the
notice described in Exhibit B of this License must be attached.
Exhibit A - Source Code Form License Notice
-------------------------------------------
This Source Code Form is subject to the terms of the Mozilla Public
License, v. 2.0. If a copy of the MPL was not distributed with this
file, You can obtain one at http://mozilla.org/MPL/2.0/.
If it is not possible or desirable to put the notice in a particular
file, then You may include the notice in a location (such as a LICENSE
file in a relevant directory) where a recipient would be likely to look
for such a notice.
You may add additional accurate notices of copyright ownership.
Exhibit B - "Incompatible With Secondary Licenses" Notice
---------------------------------------------------------
This Source Code Form is "Incompatible With Secondary Licenses", as
defined by the Mozilla Public License, v. 2.0.

View file

@ -1,2 +1,46 @@
# PneuMARL
A PyGame2 Zelda-like embedded with MARL algorithm.
# Pneuma: Reinforcement Learning Platform
## Introduction
Pneuma is a Reinforcement Learning platform created as part of a thesis project. It is developed using PyGame and offers a customizable environment for testing and implementing reinforcement learning algorithms.
## Installation
To install Pneuma, clone this repository and install the requirements (`requirements.txt`)
After cloning, you can edit the agents, create your own, and modify pneuma.py (the main file). Additionally, consider editing player.setup_agent() for further customization.
Note
- [] TODO: Separate the update logic from the network logic inside the player.
## Usage
To run Pneuma, use the command-line interface with the following options:
- `--no_seed`: If set to True, runs the program without a seed. Default is False.
- `--seed [int]`: Specifies the seed for the random number generator. Default is 1.
- `--n_episodes [int]`: Defines the number of episodes. Default is 300.
- `--ep_length [int]`: Sets the length of each episode. Default is 5000.
- `--n_players [int]`: Number of players. Default is 1.
- `--chkpt_path [str]`: Path for saving/loading agent models. Default is "agents/saved_models".
- `--figure_path [str]`: Path for saving figures. Default is "figures".
- `--horizon [int]`: Number of steps per update. Default is 200.
- `--show_pg`: If True, opens a PyGame window on the desktop. Default is False.
- `--no_load`: If True, ignores saved models. Default is False.
- `--gamma [float]`: The gamma parameter for PPO. Default is 0.99.
- `--alpha [float]`: The alpha parameter for PPO. Default is 0.0003.
- `--policy_clip [float]`: The policy clip. Default is 0.2.
- `--batch_size [int]`: Size of each batch. Default is 64.
- `--n_epochs [int]`: Number of epochs. Default is 10.
- `--gae_lambda [float]`: The lambda parameter of the GAE. Default is 0.95.
### Example Command
```bash
$ python pneuma.py --seed 42 --n_episodes 300 --ep_length 5000 --n_players 2 --no_load
```
## License
Pneuma is licensed under the Mozilla Public License 2.0.

View file

View file

@ -8,15 +8,19 @@ class Agent:
def __init__(self, input_dims, n_actions, gamma=0.99, alpha=0.0003,
policy_clip=0.2, batch_size=64, N=2048, n_epochs=10,
gae_lambda=0.95):
gae_lambda=0.95, chkpt_dir='tmp/ppo'):
self.gamma = gamma
self.policy_clip = policy_clip
self.n_epochs = n_epochs
self.gae_lambda = gae_lambda
self.actor = ActorNetwork(input_dims, n_actions, alpha)
self.critic = CriticNetwork(input_dims, alpha)
self.actor = ActorNetwork(
input_dims, n_actions, alpha, chkpt_dir=chkpt_dir)
self.critic = CriticNetwork(
input_dims, alpha, chkpt_dir=chkpt_dir)
self.memory = PPOMemory(batch_size)
def remember(self, state, action, probs, vals, reward, done):
@ -79,17 +83,17 @@ class Agent:
weighted_probs = advantage[batch] * prob_ratio
weighted_clipped_probs = T.clamp(
prob_ratio, 1-self.policy_clip, 1+self.policy_clip)*advantage[batch]
actor_loss = -T.min(weighted_probs,
self.actor_loss = -T.min(weighted_probs,
weighted_clipped_probs).mean()
returns = advantage[batch] + values[batch]
critic_loss = (returns - critic_value)**2
critic_loss = critic_loss.mean()
self.critic_loss = (returns - critic_value)**2
self.critic_loss = self.critic_loss.mean()
total_loss = actor_loss + 0.5*critic_loss
self.total_loss = self.actor_loss + 0.5*self.critic_loss
self.actor.optimizer.zero_grad()
self.critic.optimizer.zero_grad()
total_loss.backward()
self.total_loss.backward()
self.actor.optimizer.step()
self.critic.optimizer.step()

View file

@ -82,7 +82,9 @@ class ActorNetwork(nn.Module):
T.save(self.state_dict(), os.path.join(self.chkpt_dir, filename))
def load_checkpoint(self, filename='actor_torch_ppo'):
self.load_state_dict(T.load(os.path.join(self.chkpt_dir, filename), map_location=self.device))
self.load_state_dict(
T.load(os.path.join(self.chkpt_dir, filename),
map_location=self.device))
class CriticNetwork(nn.Module):
@ -114,4 +116,6 @@ class CriticNetwork(nn.Module):
T.save(self.state_dict(), os.path.join(self.chkpt_dir, filename))
def load_checkpoint(self, filename='critic_torch_ppo'):
self.load_state_dict(T.load(os.path.join(self.chkpt_dir, filename), map_location=self.device))
self.load_state_dict(
T.load(os.path.join(self.chkpt_dir, filename),
map_location=self.device))

BIN
agents/saved_models/A0 Normal file

Binary file not shown.

BIN
agents/saved_models/C0 Normal file

Binary file not shown.

View file

@ -0,0 +1 @@
# This is a folder with all the saved models.

View file

@ -5,40 +5,40 @@
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,392,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,390,-1,-1,-1,-1,-1,-1,391,-1,-1,-1,391,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,390,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,391,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,393,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,400,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,393,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,390,-1,-1,-1,400,400,-1,-1,400,-1,400,-1,-1,400,-1,-1,-1,400,-1,-1,-1,400,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,390,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,390,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,393,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,390,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,390,-1,-1,-1,-1,-1,-1,400,-1,-1,-1,-1,-1,-1,400,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,400,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,400,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,400,-1,-1,-1,390,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,500,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,400,-1,-1,-1,-1,400,-1,-1,-1,-1,-1,400,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,393,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,390,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,390,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,390,-1,-1,-1,-1,-1,390,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,400,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,400,-1,-1,-1,390,-1,-1,-1,-1,-1,-1,391,-1,-1,-1,391,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,390,-1,-1,400,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,391,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,400,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,400,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,400,393,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,400,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,400,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,393,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,400,-1,390,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,400,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,390,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,400,-1,-1,-1,-1,390,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,393,-1,-1,-1,-1,400,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,390,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,400,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,390,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,400,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,390,-1,400,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,500,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,400,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,400,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,400,-1,393,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,390,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,390,400,400,400,400,400,400,400,400,400,400,400,400,400,400,400,400,400,400,400,400,400,400,400,400,400,400,400,400,400,400,400,400,400,400,400,400,400,400,400,400,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,400,-1,-1,-1,-1,-1,-1,-1,390,-1,-1,-1,-1,-1,390,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,390,-1,393,-1,-1,400,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,393,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,400,-1,-1,400,-1,-1,400,-1,-1,-1,-1,400,-1,-1,-1,390,-1,400,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,390,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,393,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,400,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,400,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,393,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,390,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,400,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,390,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,391,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,400,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,390,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,400,-1,-1,390,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,393,-1,-1,-1,-1,-1,-1,-1,400,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,400,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,400,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,400,-1,-1,393,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,400,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,400,390,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,400,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,400,-1,-1,-1,-1,-1,390,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,400,-1,391,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,400,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,400,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,400,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,393,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,393,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,400,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,393,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,393,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,392,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1

1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
5 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
6 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
7 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 392 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
8 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 400 -1 -1 -1 390 -1 -1 -1 -1 -1 -1 391 -1 -1 -1 391 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
9 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 390 -1 -1 -1 400 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 391 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
10 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 400 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
11 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 400 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
12 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 400 393 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
13 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 400 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
14 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 400 -1 -1 400 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 393 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
15 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 400 -1 390 -1 -1 -1 400 -1 400 -1 -1 -1 400 -1 -1 400 -1 -1 -1 400 -1 -1 -1 -1 400 -1 -1 -1 -1 400 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
16 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 400 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 390 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
17 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 400 -1 -1 -1 -1 390 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
18 -1 -1 -1 -1 -1 -1 -1 -1 -1 393 -1 -1 -1 -1 -1 400 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 390 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
19 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 400 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 390 -1 -1 -1 -1 -1 -1 400 -1 -1 -1 -1 -1 -1 -1 400 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
20 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 400 400 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 400 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
21 -1 -1 -1 -1 -1 -1 -1 -1 400 -1 -1 -1 -1 390 -1 -1 400 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 500 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
22 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 400 -1 -1 -1 -1 -1 -1 -1 400 -1 -1 -1 -1 -1 400 -1 -1 -1 -1 -1 -1 400 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
23 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 400 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
24 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 400 -1 393 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 390 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
25 -1 -1 -1 -1 -1 -1 390 -1 400 -1 400 -1 400 -1 400 -1 400 -1 400 -1 400 -1 400 -1 400 -1 400 -1 400 -1 400 -1 400 -1 400 -1 400 -1 400 -1 400 -1 400 -1 400 -1 400 -1 400 -1 400 -1 400 -1 400 -1 400 -1 400 -1 400 -1 400 -1 400 -1 400 -1 400 -1 400 -1 400 -1 400 -1 400 -1 400 -1 400 -1 400 -1 400 -1 400 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
26 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 400 -1 -1 -1 -1 -1 -1 -1 390 -1 -1 -1 -1 -1 390 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 400 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
27 -1 -1 -1 -1 -1 -1 -1 -1 -1 390 -1 393 -1 -1 400 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 393 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
28 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 400 -1 -1 -1 -1 400 -1 -1 -1 400 -1 -1 -1 400 -1 -1 -1 -1 -1 400 -1 -1 -1 -1 390 -1 400 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
29 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 400 -1 -1 390 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 393 -1 -1 -1 -1 -1 -1 -1 -1 400 -1 -1 -1 -1 -1 400 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
30 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 400 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 400 400 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
31 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 400 -1 -1 393 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 400 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
32 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 400 390 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 400 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 400 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
33 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 400 -1 -1 -1 -1 -1 390 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 400 -1 391 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
34 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 400 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
35 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 400 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
36 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
37 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
38 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
-1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 400 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
39 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
40 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
41 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 393 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 400 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
-1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 393 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
42 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
43 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 393 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 393 -1 -1 -1 -1 -1 -1 -1 -1 -1
44 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 392 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1

View file

@ -3,11 +3,11 @@
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,395,395,395,395,-1,-1,-1,-1,-1,395,395,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,395,395,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,395,395,395,-1,-1,395,395,395,395,-1,-1,395,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,395,395,395,395,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,395,395,395,395,395,395,-1,-1,-1,-1,-1,-1,-1,395,-1,-1,395,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,395,395,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,395,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,395,395,-1,395,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,395,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,395,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,395,-1,395,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,395,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,395,395,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,395,-1,395,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,395,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,395,395,395,395,-1,-1,-1,-1,-1,-1,-1,-1,-1,395,-1,395,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,395,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,395,395,-1,-1,-1,-1,-1,-1,-1,395,395,-1,395,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,395,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,395,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,395,395,-1,395,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,395,-1,-1,-1,-1,-1,-1,-1,-1,395,395,395,395,395,-1,-1,-1
-1,-1,-1,-1,-1,395,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,395,-1,395,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,395,-1,-1,-1,-1,-1,-1,-1,-1,395,-1,-1,-1,395,-1,-1,-1
-1,-1,-1,-1,-1,395,395,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,395,-1,395,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,395,-1,-1,-1,-1,-1,-1,-1,-1,395,-1,700,-1,395,-1,-1,-1
-1,-1,-1,-1,-1,-1,395,395,395,395,-1,-1,-1,-1,-1,-1,-1,-1,-1,395,-1,395,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,395,-1,-1,-1,-1,-1,-1,-1,-1,395,-1,-1,-1,395,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,395,395,-1,-1,-1,-1,-1,-1,-1,395,395,-1,395,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,395,-1,-1,-1,-1,-1,-1,-1,-1,395,395,395,395,395,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,395,-1,-1,-1,-1,-1,-1,395,395,-1,-1,-1,395,395,395,395,395,395,395,395,-1,-1,395,395,395,395,395,395,395,395,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,395,395,-1,-1,-1,-1,-1,395,-1,-1,-1,-1,395,-1,-1,-1,-1,-1,-1,395,-1,-1,395,-1,-1,-1,-1,-1,-1,395,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,395,-1,-1,-1,-1,-1,395,395,395,395,395,395,395,395,395,395,395,395,395,-1,-1,395,395,395,395,395,395,395,395,395,395,395,395,395,395,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1

1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 395 395 395 395 395 395 395 395 395 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
3 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 395 395 395 395 -1 -1 -1 -1 -1 395 395 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 395 395 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
4 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 395 395 395 -1 -1 395 395 395 395 -1 -1 395 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 395 395 395 395 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
5 -1 -1 -1 -1 -1 395 395 395 395 395 395 -1 -1 -1 -1 -1 -1 -1 395 -1 -1 395 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 395 395 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
6 -1 -1 -1 -1 -1 395 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 395 395 -1 395 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 395 -1 -1 -1 -1 -1 -1 -1 -1 -1 395 -1 395 -1 395 -1 395 -1 395 -1 -1 -1
7 -1 -1 -1 -1 -1 395 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 395 -1 395 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 395 -1 -1 -1 -1 -1 -1 -1 -1 -1 395 -1 -1 -1 -1 395 -1 -1 -1
8 -1 -1 -1 -1 -1 395 395 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 395 -1 395 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 395 -1 -1 -1 -1 -1 -1 -1 -1 -1 395 -1 -1 700 -1 -1 395 -1 -1 -1
9 -1 -1 -1 -1 -1 -1 395 395 395 395 -1 -1 -1 -1 -1 -1 -1 -1 -1 395 -1 395 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 395 -1 -1 -1 -1 -1 -1 -1 -1 -1 395 -1 -1 -1 -1 395 -1 -1 -1
10 -1 -1 -1 -1 -1 -1 -1 -1 -1 395 395 -1 -1 -1 -1 -1 -1 -1 395 395 -1 395 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 395 -1 -1 -1 -1 -1 -1 -1 -1 -1 395 -1 395 -1 395 -1 395 -1 395 -1 -1 -1
11 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 395 -1 -1 -1 -1 -1 -1 395 395 -1 -1 -1 395 395 395 395 395 395 395 395 -1 -1 395 395 395 395 395 395 395 395 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
12 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 395 395 -1 -1 -1 -1 -1 395 -1 -1 -1 -1 395 -1 -1 -1 -1 -1 -1 395 -1 -1 395 -1 -1 -1 -1 -1 -1 395 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
13 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 395 -1 -1 -1 -1 -1 395 395 395 395 395 395 395 395 395 395 395 395 395 -1 -1 395 395 395 395 395 395 395 395 395 395 395 395 395 395 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1

61
camera.py Normal file
View file

@ -0,0 +1,61 @@
import os
import pygame
from utils.resource_loader import import_assets
class Camera(pygame.sprite.Group):
def __init__(self):
super().__init__()
# General Setup
self.display_surface = pygame.display.get_surface()
self.half_width = self.display_surface.get_size()[0] // 2
self.half_height = self.display_surface.get_size()[1] // 2
self.offset = pygame.math.Vector2(100, 200)
# Creating the floor
image_path = import_assets(os.path.join('graphics',
'tilemap',
'ground.png'))
self.floor_surf = pygame.image.load(
import_assets(
os.path.join('graphics',
'tilemap',
'ground.png')
)
).convert()
self.floor_rect = self.floor_surf.get_rect(topleft=(0, 0))
def custom_draw(self, entity):
self.sprite_type = entity.sprite_type
# Getting the offset
if hasattr(entity, 'animation'):
self.offset.x = entity.animation.rect.centerx - self.half_width
self.offset.y = entity.animation.rect.centery - self.half_height
else:
self.offset.x = entity.rect.centerx - self.half_width
self.offset.y = entity.rect.centery - self.half_height
# Drawing the floor
floor_offset_pos = self.floor_rect.topleft - self.offset
self.display_surface.blit(self.floor_surf, floor_offset_pos)
for sprite in sorted(self.sprites(),
key=lambda sprite: sprite.animation.rect.centery
if hasattr(sprite, 'animation')
else sprite.rect.centery):
if hasattr(sprite, 'animation'):
offset_pos = sprite.animation.rect.topleft - self.offset
self.display_surface.blit(sprite.animation.image, offset_pos)
else:
offset_pos = sprite.rect.topleft - self.offset
self.display_surface.blit(sprite.image, offset_pos)

View file

@ -1,12 +1,41 @@
import os
script_dir = os.path.dirname(os.path.abspath(__file__))
asset_path = os.path.join(
script_dir, '../..', 'assets')
monster_data = {
'squid': {'id': 1, 'health': .1, 'exp': 1, 'attack': .5, 'attack_type': 'slash', 'speed': 3, 'knockback': 20, 'attack_radius': 80, 'notice_radius': 360},
'raccoon': {'id': 2, 'health': .3, 'exp': 2.5, 'attack': .8, 'attack_type': 'claw', 'speed': 2, 'knockback': 20, 'attack_radius': 120, 'notice_radius': 400},
'spirit': {'id': 3, 'health': .1, 'exp': 1.1, 'attack': .6, 'attack_type': 'thunder', 'speed': 4, 'knockback': 20, 'attack_radius': 60, 'notice_radius': 350},
'bamboo': {'id': 4, 'health': .07, 'exp': 1.2, 'attack': .2, 'attack_type': 'leaf_attack', 'speed': 3, 'knockback': 20, 'attack_radius': 50, 'notice_radius': 300}}
'squid': {'id': 1,
'health': .1,
'exp': 1,
'attack': .5,
'attack_type': 'slash',
'speed': 3,
'knockback': 20,
'attack_radius': 80,
'notice_radius': 360},
'raccoon': {'id': 2,
'health': .3,
'exp': 2.5,
'attack': .8,
'attack_type': 'claw',
'speed': 2,
'knockback': 20,
'attack_radius': 120,
'notice_radius': 400},
'spirit': {'id': 3,
'health': .1,
'exp': 1.1,
'attack': .6,
'attack_type': 'thunder',
'speed': 4,
'knockback': 20,
'attack_radius': 60,
'notice_radius': 350},
'bamboo': {'id': 4,
'health': .07,
'exp': 1.2,
'attack': .2,
'attack_type': 'leaf_attack',
'speed': 3,
'knockback': 20,
'attack_radius': 50,
'notice_radius': 300}
}

View file

@ -1,9 +1,22 @@
import os
script_dir = os.path.dirname(os.path.abspath(__file__))
asset_path = os.path.join(
script_dir, '../..', 'assets')
from utils.resource_loader import import_assets
magic_data = {
'flame': {'strength': 5, 'cost': .020, 'graphic': f"{asset_path}/graphics/particles/flame/fire.png"},
'heal': {'strength': 20, 'cost': .010, 'graphic': f"{asset_path}/graphics/particles/heal/heal.png"}}
'flame': {'strength': 5, 'cost': .020, 'graphic': import_assets(
os.path.join('graphics',
'particles',
'flame',
'fire.png')
)
},
'heal': {'strength': 20, 'cost': .010, 'graphic': import_assets(
os.path.join('graphics',
'particles',
'heal',
'heal.png')
)
}
}

View file

@ -1,13 +1,43 @@
import os
script_dir = os.path.dirname(os.path.abspath(__file__))
asset_path = os.path.join(
script_dir, '../..', 'assets')
from utils.resource_loader import import_assets
weapon_data = {
'sword': {'cooldown': 100, 'damage': 15, 'graphic': f"{asset_path}/graphics/weapons/sword/full.png"},
'lance': {'cooldown': 400, 'damage': 30, 'graphic': f"{asset_path}/graphics/weapons/lance/full.png"},
'axe': {'cooldown': 300, 'damage': 20, 'graphic': f"{asset_path}/graphics/weapons/axe/full.png"},
'rapier': {'cooldown': 50, 'damage': 8, 'graphic': f"{asset_path}/graphics/weapons/rapier/full.png"},
'sai': {'cooldown': 80, 'damage': 10, 'graphic': f"{asset_path}/graphics/weapons/sai/full.png"}
'sword': {'cooldown': 100, 'damage': 15, 'graphic': import_assets(
os.path.join('graphics',
'weapons',
'sword',
'full.png')
)
},
'lance': {'cooldown': 400, 'damage': 30, 'graphic': import_assets(
os.path.join('graphics',
'weapons',
'lance',
'full.png')
)
},
'axe': {'cooldown': 300, 'damage': 20, 'graphic': import_assets(
os.path.join('graphics',
'weapons',
'axe',
'full.png')
)
},
'rapier': {'cooldown': 50, 'damage': 8, 'graphic': import_assets(
os.path.join('graphics',
'weapons',
'rapier',
'full.png')
)
},
'sai': {'cooldown': 80, 'damage': 10, 'graphic': import_assets(
os.path.join('graphics',
'weapons',
'sai',
'full.png')
)
},
}

View file

@ -1,4 +1,3 @@
import os
import pygame
from random import randint
@ -17,9 +16,14 @@ class MagicPlayer:
if player.stats.health >= player.stats.stats['health']:
player.stats.health = player.stats.stats['health']
self.animation_player.generate_particles(
'aura', player.rect.center, groups)
'aura',
player.rect.center,
groups)
self.animation_player.generate_particles(
'heal', player.rect.center + pygame.math.Vector2(0, -50), groups)
'heal',
player.rect.center + pygame.math.Vector2(0, -50),
groups)
def flame(self, player, cost, groups):
if player.stats.energy >= cost:

View file

@ -8,49 +8,118 @@ from random import choice
class AnimationPlayer:
def __init__(self):
script_dir = os.path.dirname(os.path.abspath(__file__))
asset_path = os.path.join(
script_dir, '..', 'assets')
self.frames = {
# magic
'flame': import_folder(f'{asset_path}/graphics/particles/flame/frames'),
'aura': import_folder(f'{asset_path}/graphics/particles/aura'),
'heal': import_folder(f'{asset_path}/graphics/particles/heal/frames'),
# Spells
'flame': import_folder(os.path.join('graphics',
'particles',
'flame',
'frames')),
# attacks
'claw': import_folder(f'{asset_path}/graphics/particles/claw'),
'slash': import_folder(f'{asset_path}/graphics/particles/slash'),
'sparkle': import_folder(f'{asset_path}/graphics/particles/sparkle'),
'leaf_attack': import_folder(f'{asset_path}/graphics/particles/leaf_attack'),
'thunder': import_folder(f'{asset_path}/graphics/particles/thunder'),
'aura': import_folder(os.path.join('graphics',
'particles',
'aura')),
# monster deaths
'squid': import_folder(f'{asset_path}/graphics/particles/smoke_orange'),
'raccoon': import_folder(f'{asset_path}/graphics/particles/raccoon'),
'spirit': import_folder(f'{asset_path}/graphics/particles/nova'),
'bamboo': import_folder(f'{asset_path}/graphics/particles/bamboo'),
'heal': import_folder(os.path.join('graphics',
'particles',
'heal',
'frames')),
# leafs
# Attacks
'claw': import_folder(os.path.join('graphics',
'particles',
'claw')),
'slash': import_folder(os.path.join('graphics',
'particles',
'slash')),
'sparkle': import_folder(os.path.join('graphics',
'particles',
'sparkle')),
'leaf_attack': import_folder(os.path.join('graphics',
'particles',
'leaf_attack')),
'thunder': import_folder(os.path.join('graphics',
'particles',
'thunder')),
# Monster Deaths
'squid': import_folder(os.path.join('graphics',
'particles',
'smoke_orange')),
'raccoon': import_folder(os.path.join('graphics',
'particles',
'raccoon')),
'spirit': import_folder(os.path.join('graphics',
'particles',
'nova')),
'bamboo': import_folder(os.path.join('graphics',
'particles',
'bamboo')),
# Leafs
'leaf': (
import_folder(f'{asset_path}/graphics/particles/leaf1'),
import_folder(f'{asset_path}/graphics/particles/leaf2'),
import_folder(f'{asset_path}/graphics/particles/leaf3'),
import_folder(f'{asset_path}/graphics/particles/leaf4'),
import_folder(f'{asset_path}/graphics/particles/leaf5'),
import_folder(f'{asset_path}/graphics/particles/leaf6'),
self.reflect_images(import_folder(
f'{asset_path}/graphics/particles/leaf1')),
self.reflect_images(import_folder(
f'{asset_path}/graphics/particles/leaf2')),
self.reflect_images(import_folder(
f'{asset_path}/graphics/particles/leaf3')),
self.reflect_images(import_folder(
f'{asset_path}/graphics/particles/leaf4')),
self.reflect_images(import_folder(
f'{asset_path}/graphics/particles/leaf5')),
self.reflect_images(import_folder(
f'{asset_path}/graphics/particles/leaf6'))
import_folder(os.path.join('graphics',
'particles',
'leaf1')),
import_folder(os.path.join('graphics',
'particles',
'leaf2')),
import_folder(os.path.join('graphics',
'particles',
'leaf3')),
import_folder(os.path.join('graphics',
'particles',
'leaf4')),
import_folder(os.path.join('graphics',
'particles',
'leaf5')),
import_folder(os.path.join('graphics',
'particles',
'leaf6')),
self.reflect_images(
import_folder(os.path.join('graphics',
'particles',
'leaf1'))),
self.reflect_images(
import_folder(os.path.join('graphics',
'particles',
'leaf2'))),
self.reflect_images(
import_folder(
os.path.join('graphics',
'particles',
'leaf3'))),
self.reflect_images(
import_folder(
os.path.join('graphics',
'particles',
'leaf4'))),
self.reflect_images(
import_folder(
os.path.join('graphics',
'particles',
'leaf5'))),
self.reflect_images(
import_folder(
os.path.join('graphics',
'particles',
'leaf6')))
)
}

View file

@ -1,22 +1,24 @@
import os
import pygame
from utils.resource_loader import import_assets
class Weapon(pygame.sprite.Sprite):
def __init__(self, player, groups):
super().__init__(groups)
script_dir = os.path.dirname(os.path.abspath(__file__))
asset_path = os.path.join(
script_dir, '..', 'assets')
self.sprite_type = 'weapon'
direction = player._input.status.split('_')[0]
# Graphic
full_path = f"{asset_path}/graphics/weapons/{player._input.combat.weapon}/{direction}.png"
self.image = pygame.image.load(full_path).convert_alpha()
self.image = pygame.image.load(import_assets(os.path.join(
'graphics',
'weapons',
player._input.combat.weapon,
f"{direction}.png"))
).convert_alpha()
# Sprite Placement
if direction == 'right':

View file

View file

@ -38,9 +38,13 @@ class InputHandler:
self.possible_actions = [0, 1, 2, 3, 4, 5]
self.action = 10
def check_input(self, button, speed, hitbox, obstacle_sprites, rect, player):
self.action = 10
def check_input(self,
button,
speed,
hitbox,
obstacle_sprites,
rect,
player):
if not self.attacking and self.can_move:
@ -105,9 +109,13 @@ class InputHandler:
# Rotating Weapons
if button == 6 and self.can_rotate_weapon:
self.can_rotate_weapon = False
self.weapon_rotation_time = pygame.time.get_ticks()
if self.combat.weapon_index < len(list(weapon_data.keys())) - 1:
if self.combat.weapon_index\
< len(list(weapon_data.keys())) - 1:
self.combat.weapon_index += 1
else:
self.combat.weapon_index = 0
@ -131,23 +139,34 @@ class InputHandler:
self.vulnerable = vulnerable
if self.attacking:
if current_time - self.attack_time > self.attack_cooldown + weapon_data[self.combat.weapon]['cooldown']:
if current_time - self.attack_time\
> self.attack_cooldown\
+ weapon_data[self.combat.weapon]['cooldown']:
self.attacking = False
if self.combat.current_attack:
self.combat.delete_attack_sprite()
if not self.can_rotate_weapon:
if current_time - self.weapon_rotation_time > self.rotate_attack_cooldown:
if current_time - self.weapon_rotation_time\
> self.rotate_attack_cooldown:
self.can_rotate_weapon = True
if not self.can_swap_magic:
if current_time - self.magic_swap_time > self.rotate_attack_cooldown:
if current_time - self.magic_swap_time\
> self.rotate_attack_cooldown:
self.can_swap_magic = True
if not vulnerable:
if current_time - self.combat.hurt_time >= self.combat.invulnerability_duration:
if current_time - self.combat.hurt_time\
>= self.combat.invulnerability_duration:
self.combat.vulnerable = True
if not self.can_move:
if current_time - self.move_time >= self.move_cooldown:
if current_time - self.move_time\
>= self.move_cooldown:
self.can_move = True

View file

@ -2,7 +2,7 @@ import os
import pygame
from math import sin
from utils.resource_loader import import_folder
from utils.resource_loader import import_folder, import_assets
from configs.system.window_config import HITBOX_OFFSET
@ -17,40 +17,47 @@ class AnimationHandler:
self.name = name
def import_assets(self, position):
script_dir = os.path.dirname(os.path.abspath(__file__))
asset_path = os.path.join(
script_dir, '../..', 'assets', 'graphics')
if self.sprite_type == 'player':
character_path = f"{asset_path}/player"
# Import Graphic Assets
if self.sprite_type == 'player':
self.image = pygame.image.load(
f"{character_path}/down/down_0.png").convert_alpha()
import_assets(os.path.join('graphics',
'player',
'down',
'down_0.png'))).convert_alpha()
self.rect = self.image.get_rect(topleft=position)
self.hitbox = self.rect.inflate(HITBOX_OFFSET[self.sprite_type])
self.animations = {
'up': [], 'down': [], 'left': [], 'right': [],
'up_idle': [], 'down_idle': [], 'left_idle': [], 'right_idle': [],
'up_attack': [], 'down_attack': [], 'left_attack': [], 'right_attack': []
'up': [], 'down': [],
'left': [], 'right': [],
'up_idle': [], 'down_idle': [],
'left_idle': [], 'right_idle': [],
'up_attack': [], 'down_attack': [],
'left_attack': [], 'right_attack': []
}
for animation in self.animations.keys():
full_path = f"{character_path}/{animation}"
self.animations[animation] = import_folder(full_path)
self.animations[animation]\
= import_folder(os.path.join('graphics',
'player',
animation
))
elif self.sprite_type == 'enemy':
self.status = 'idle'
character_path = f"{asset_path}/monsters/{self.name}"
self.animations = {'idle': [], 'move': [], 'attack': []}
for animation in self.animations.keys():
self.animations[animation] = import_folder(
f"{character_path}/{animation}")
self.animations[animation]\
= import_folder(os.path.join('graphics',
'monsters',
self.name,
animation))
self.image = self.animations[self.status][self.frame_index]
self.rect = self.image.get_rect(topleft=position)

View file

@ -11,11 +11,12 @@ class Enemy(pygame.sprite.Sprite):
def __init__(self, name, position, groups, visible_sprites, obstacle_sprites):
super().__init__(groups)
self.sprite_type = "enemy"
self.name = name
self.visible_sprites = visible_sprites
self.position = position
# Setup Graphics
self.animation_player = AnimationPlayer()
self.animation = AnimationHandler(self.sprite_type, self.name)

View file

@ -3,6 +3,8 @@ import pygame
from configs.system.window_config import HITBOX_OFFSET
from utils.resource_loader import import_assets
class Observer(pygame.sprite.Sprite):
@ -11,12 +13,11 @@ class Observer(pygame.sprite.Sprite):
self.sprite_type = 'camera'
script_dir = os.path.dirname(os.path.abspath(__file__))
asset_path = os.path.join(
script_dir, '..', 'assets')
self.image = pygame.image.load(
f"{asset_path}/graphics/observer.png").convert_alpha()
import_assets(os.path.join('graphics',
'observer.png'))
).convert_alpha()
self.rect = self.image.get_rect(topleft=position)
self.hitbox = self.rect.inflate(HITBOX_OFFSET[self.sprite_type])

View file

@ -1,3 +1,4 @@
import os
import pygame
import numpy as np
from random import randint
@ -15,66 +16,101 @@ from agents.ppo.agent import Agent
class Player(pygame.sprite.Sprite):
def __init__(self,
player_id,
role,
position,
groups,
obstacle_sprites,
visible_sprites,
attack_sprites,
attackable_sprites,
role,
player_id):
attackable_sprites
):
super().__init__(groups)
# Setup Sprites
self.sprite_type = 'player'
self.status = 'down'
self.initial_position = position
self.player_id = player_id
self.distance_direction_from_enemy = None
# Sprite Setup
self.sprite_type = "player"
self.obstacle_sprites = obstacle_sprites
self.visible_sprites = visible_sprites
self.attack_sprites = attack_sprites
self.obstacle_sprites = obstacle_sprites
self.attackable_sprites = attackable_sprites
# Setup Graphics
# Graphics Setup
self.animation_player = AnimationPlayer()
self.animation = AnimationHandler(self.sprite_type)
self.animation.import_assets(position)
self.image = self.animation.image
self.rect = self.animation.rect
# Setup Inputs
# Input Setup
self._input = InputHandler(
self.sprite_type, self.animation_player) # , self.status)
self.sprite_type, self.animation_player)
# Setup Stats
self.role = role
self.stats = StatsHandler(self.sprite_type, self.role)
self.distance_direction_from_enemy = None
def setup_agent(self,
gamma,
alpha,
policy_clip,
batch_size,
N,
n_epochs,
gae_lambda,
chkpt_dir,
no_load=False):
# Setup AI
self.score = 0
self.learn_iters = 0
self.n_steps = 0
self.N = 20
self.get_current_state()
self.agent = Agent(
input_dims=len(self.state_features),
n_actions=len(self._input.possible_actions),
gamma=gamma,
alpha=alpha,
policy_clip=policy_clip,
batch_size=batch_size,
N=N,
n_epochs=n_epochs,
gae_lambda=gae_lambda,
chkpt_dir=chkpt_dir
)
print(
f"\nAgent initialized on player {self.player_id} using {self.agent.actor.device}.")
if not no_load:
print("Attempting to load models ...")
try:
self.agent.load_models(
actr_chkpt=f"A{self.player_id}",
crtc_chkpt=f"C{self.player_id}"
)
print("Models loaded ...\n")
except FileNotFoundError:
print(
f"FileNotFound for player {self.player_id}.\
\nSkipping loading ...\n")
def get_status(self):
if self._input.movement.direction.x == 0 and self._input.movement.direction.y == 0:
if 'idle' not in self.status and 'attack' not in self.status:
self.status += '_idle'
if self._input.movement.direction.x == 0\
and self._input.movement.direction.y == 0:
if 'idle' not in self._input.status and 'attack' not in self._input.status:
self._input.status += '_idle'
if self._input.attacking:
self._input.movement.direction.x = 0
self._input.movement.direction.y = 0
if 'attack' not in self.status:
if 'idle' in self.status:
self.status = self.status.replace('idle', 'attack')
if 'attack' not in self._input.status:
if 'idle' in self._input.status:
self._input.status = self._input.status.replace(
'idle', 'attack')
else:
self.status += '_attack'
self._input.status += '_attack'
else:
if 'attack' in self.status:
self.status = self.status.replace('_attack', '')
if 'attack' in self._input.status:
self._input.status = self._input.status.replace('_attack', '')
def attack_logic(self):
if self.attack_sprites:
@ -123,11 +159,12 @@ class Player(pygame.sprite.Sprite):
2*np.exp(-nearest_dist**2),
np.exp(-nearest_enemy.stats.health),
-np.exp(-self.stats.health**2)
if not self.is_dead() > 0 else -1
]
self.state_features = [
np.exp(-self.rect.center[0]),
np.exp(-self.rect.center[1]),
np.exp(-self.animation.rect.center[0]),
np.exp(-self.animation.rect.center[1]),
self._input.movement.direction.x,
self._input.movement.direction.y,
self.stats.health/self.stats.stats['health'],
@ -153,42 +190,23 @@ class Player(pygame.sprite.Sprite):
self.state_features = np.array(self.state_features)
def get_max_num_states(self):
self.get_current_state()
self.num_features = len(self.state_features)
def setup_agent(self):
print(f"Initializing agent on player {self.player_id} ...")
self.agent = Agent(
input_dims=len(self.state_features),
n_actions=len(self._input.possible_actions),
batch_size=5,
n_epochs=4)
print(
f" Agent initialized using {self.agent.actor.device}. Attempting to load models ...")
try:
self.agent.load_models(
actr_chkpt=f"player_actor", crtc_chkpt=f"player_critic")
print("Models loaded ...\n")
except FileNotFoundError:
print("FileNotFound for agent. Skipping loading...\n")
def is_dead(self):
if self.stats.health <= 0:
self.stats.health = 0
self.animation.import_assets((3264, 448))
return True
else:
return False
def update(self):
if not self.is_dead():
# Get the current state
self.get_current_state()
# Choose action based on current state
action, probs, value = self.agent.choose_action(self.state_features)
action, probs, value\
= self.agent.choose_action(self.state_features)
self.n_steps += 1
# Apply chosen action
self._input.check_input(action,
self.stats.speed,
@ -201,25 +219,19 @@ class Player(pygame.sprite.Sprite):
self.agent.remember(self.state_features, action,
probs, value, self.stats.exp, self.is_dead())
if self.n_steps % self.N == 0:
self.agent.learn()
self.learn_iters += 1
self.get_current_state()
# Refresh objects based on input
self.status = self._input.status
# Animate
self.get_status()
self.animation.animate(self.status, self._input.combat.vulnerable)
self.image = self.animation.image
self.rect = self.animation.rect
# Cooldowns and Regen
self.stats.health_recovery()
self.stats.energy_recovery()
self._input.cooldowns(self._input.combat.vulnerable)
if self.is_dead():
self.stats.exp = max(-1, self.stats.exp - .5)
else:
self.stats.exp = max(0, self.stats.exp - .01)
# Refresh player based on input and animate
self.get_status()
self.animation.animate(
self._input.status, self._input.combat.vulnerable)
self.image = self.animation.image
self.rect = self.animation.rect
self._input.cooldowns(self._input.combat.vulnerable)

View file

@ -1,15 +1,26 @@
import pygame
from configs.system.window_config import *
from configs.system.window_config import TILESIZE,\
HITBOX_OFFSET
class Tile(pygame.sprite.Sprite):
class Terrain(pygame.sprite.Sprite):
def __init__(self,
position,
groups,
sprite_type,
surface=pygame.Surface((TILESIZE, TILESIZE))
):
def __init__(self, position, groups, sprite_type, surface=pygame.Surface((TILESIZE, TILESIZE))):
super().__init__(groups)
self.sprite_type = sprite_type
self.position = position
self.image = surface
if sprite_type == 'object':
# Offset
self.rect = self.image.get_rect(

BIN
figures/actor_loss.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 21 KiB

BIN
figures/critic_loss.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 23 KiB

BIN
figures/score.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 21 KiB

BIN
figures/total_loss.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 21 KiB

51
game.py
View file

@ -1,49 +1,58 @@
import pygame
import os
import sys
import pygame
from level.level import Level
from configs.system.window_config import WIDTH, HEIGHT, WATER_COLOR, FPS
from level import Level
from configs.system.window_config import WIDTH,\
HEIGHT,\
WATER_COLOR,\
FPS
class Game:
def __init__(self, n_players):
def __init__(self, show_pg=False, n_players=1,):
print(f"Initializing Pneuma with {n_players} player(s).\
\nShowing PyGame screen: {'True' if show_pg else 'False'}")
pygame.init()
if show_pg:
self.screen = pygame.display.set_mode(
(WIDTH, HEIGHT)) # , pygame.HIDDEN)
(WIDTH, HEIGHT)
)
pygame.display.set_caption('Pneuma')
else:
self.screen = pygame.display.set_mode(
(WIDTH, HEIGHT),
pygame.HIDDEN
)
img = pygame.image.load('assets/graphics/icon.png')
pygame.display.set_caption("Pneuma")
img = pygame.image.load(os.path.join('assets',
'graphics',
'icon.png'))
pygame.display.set_icon(img)
self.clock = pygame.time.Clock()
self.level = Level(n_players)
self.max_num_players = len(self.level.player_sprites)
def calc_score(self):
self.scores = [0 for _ in range(self.max_num_players)]
for player in self.level.player_sprites:
self.scores[player.player_id] = player.stats.exp
def run(self):
self.clock = pygame.time.Clock()
for event in pygame.event.get():
if event.type == pygame.QUIT:
self.quit()
if event.type == pygame.KEYDOWN:
elif event.type == pygame.KEYDOWN:
if event.key == pygame.K_m:
self.level.toggle_menu()
self.level.observer.update()
self.level.pause()
self.screen.fill(WATER_COLOR)
self.level.run()
self.level.run('observer', self.clock.get_fps())
pygame.display.update()
self.clock.tick(FPS)

View file

@ -1,18 +1,25 @@
import pygame
from configs.game.weapon_config import *
from configs.game.spell_config import *
from configs.game.weapon_config import weapon_data
from configs.game.spell_config import magic_data
from .ui_settings import *
from .ui_settings import UI_FONT,\
UI_FONT_SIZE,\
HEALTH_BAR_WIDTH,\
HEALTH_COLOR,\
ENERGY_BAR_WIDTH,\
ENERGY_COLOR,\
BAR_HEIGHT,\
UI_BG_COLOR,\
UI_BORDER_COLOR_ACTIVE,\
UI_BORDER_COLOR,\
TEXT_COLOR,\
ITEM_BOX_SIZE
class UI:
def __init__(self):
script_dir = os.path.dirname(os.path.abspath(__file__))
asset_path = os.path.join(
script_dir, '../..', 'assets')
# General info
self.display_surface = pygame.display.get_surface()
self.font = pygame.font.Font(UI_FONT, UI_FONT_SIZE)
@ -66,7 +73,7 @@ class UI:
pygame.draw.rect(self.display_surface,
UI_BORDER_COLOR, text_rect.inflate(10, 10), 4)
else:
text_surf = self.font.render(f"OBSERVER", False, TEXT_COLOR)
text_surf = self.font.render("OBSERVER", False, TEXT_COLOR)
x = self.display_surface.get_size()[0] - 20
y = self.display_surface.get_size()[1] - 20
text_rect = text_surf.get_rect(bottomright=(x, y))
@ -104,9 +111,17 @@ class UI:
def display(self, player):
if player.sprite_type == 'player':
self.show_bar(
player.stats.health, player.stats.stats['health'], self.health_bar_rect, HEALTH_COLOR)
player.stats.health,
player.stats.stats['health'],
self.health_bar_rect,
HEALTH_COLOR)
self.show_bar(
player.stats.energy, player.stats.stats['energy'], self.energy_bar_rect, ENERGY_COLOR)
player.stats.energy,
player.stats.stats['energy'],
self.energy_bar_rect,
ENERGY_COLOR)
self.show_exp(player.stats.exp)
self.weapon_overlay(player._input.combat.weapon_index,
player._input.can_rotate_weapon)

View file

@ -1,29 +1,26 @@
import os
from utils.resource_loader import import_assets
script_dir = os.path.dirname(os.path.abspath(__file__))
asset_path = os.path.join(
script_dir, '..', 'assets')
# ui
# UI
BAR_HEIGHT = 20
HEALTH_BAR_WIDTH = 200
ENERGY_BAR_WIDTH = 140
ITEM_BOX_SIZE = 80
UI_FONT = f"{asset_path}/font/joystix.ttf"
UI_FONT = import_assets(path=os.path.join('font', 'joystix.ttf'))
UI_FONT_SIZE = 18
# general colors
# General Colors
WATER_COLOR = '#71ddee'
UI_BG_COLOR = '#222222'
UI_BORDER_COLOR = '#111111'
TEXT_COLOR = '#EEEEEE'
# ui colors
# UI Colors
HEALTH_COLOR = 'red'
ENERGY_COLOR = 'blue'
UI_BORDER_COLOR_ACTIVE = 'gold'
# Upgrade menu
# Upgrade Menu
TEXT_COLOR_SELECTED = '#111111'
BAR_COLOR = '#EEEEEE'
BAR_COLOR_SELECTED = '#111111'

View file

@ -1,6 +1,14 @@
import pygame
from .ui_settings import UI_FONT, UI_FONT_SIZE, TEXT_COLOR, TEXT_COLOR_SELECTED, UPGRADE_BG_COLOR_SELECTED, UI_BORDER_COLOR, UI_BG_COLOR, BAR_COLOR_SELECTED, BAR_COLOR
from .ui_settings import UI_FONT,\
UI_FONT_SIZE,\
TEXT_COLOR,\
TEXT_COLOR_SELECTED,\
UPGRADE_BG_COLOR_SELECTED,\
UI_BORDER_COLOR,\
UI_BG_COLOR,\
BAR_COLOR_SELECTED,\
BAR_COLOR
class Upgrade:

305
level.py Normal file
View file

@ -0,0 +1,305 @@
import os
import pygame
import numpy as np
from random import choice
from configs.system.window_config import TILESIZE
from utils.debug import debug
from utils.resource_loader import import_csv_layout, import_folder
from interface.ui import UI
from entities.observer import Observer
from entities.player import Player
from entities.enemy import Enemy
from entities.terrain import Terrain
from camera import Camera
class Level:
def __init__(self, n_players):
self.paused = False
self.done = False
# Get display surface
self.display_surface = pygame.display.get_surface()
# Setup Sprite groups
self.visible_sprites = Camera()
self.obstacle_sprites = pygame.sprite.Group()
self.attack_sprites = pygame.sprite.Group()
self.attackable_sprites = pygame.sprite.Group()
# Map generation
self.n_players = n_players
self.generate_map()
# Handle generated entities
self.get_entities()
self.get_distance_direction()
self.dead_players = np.zeros(self.n_players)
# Setup UI
self.ui = UI()
def generate_map(self):
self.possible_player_locations = []
player_id = 0
self.layouts = {
'boundary': import_csv_layout(os.path.join('map',
'FloorBlocks.csv')),
'grass': import_csv_layout(os.path.join('map',
'Grass.csv')),
'objects': import_csv_layout(os.path.join('map',
'Objects.csv')),
'entities': import_csv_layout(os.path.join('map',
'Entities.csv'))
}
self.graphics = {
'grass': import_folder(os.path.join('graphics', 'grass')),
'objects': import_folder(os.path.join('graphics', 'objects'))
}
for style, layout in self.layouts.items():
for row_index, row in enumerate(layout):
for col_index, col in enumerate(row):
if int(col) != -1:
x = col_index * TILESIZE
y = row_index * TILESIZE
# Generate unpassable terrain
if style == 'boundary':
if col != '700':
Terrain((x, y),
[self.obstacle_sprites,
self.visible_sprites],
'invisible')
if col == '700':
print(f"Prison set at:{(x, y)}")
# Generate grass
if style == 'grass':
random_grass_image = choice(self.graphics['grass'])
Terrain((x, y), [
self.visible_sprites,
self.obstacle_sprites,
self.attackable_sprites
],
'grass',
random_grass_image)
# Generate objects like trees and statues
# if style == 'objects':
# surface = self.graphics['objects'][int(col)]
# Terrain((x, y), [
# self.visible_sprites,
# self.obstacle_sprites
# ],
# 'object',
# surface)
# Generate observer, players and monsters
if style == 'entities':
# Generate observer
if col == '500':
self.observer = Observer(
(x, y),
[self.visible_sprites]
)
# Generate player(s)
# TODO: Make a way to generate players in random locations
elif col == '400':
self.possible_player_locations.append((x, y))
# Monster generation
else:
if col == '390':
monster_name = 'bamboo'
elif col == '391':
monster_name = 'spirit'
elif col == '392':
monster_name = 'raccoon'
elif col == ' 393':
monster_name = 'squid'
Enemy(name=monster_name,
position=(x, y),
groups=[self.visible_sprites,
self.attackable_sprites],
visible_sprites=self.visible_sprites,
obstacle_sprites=self.obstacle_sprites)
for player_id in range(self.n_players):
Player(
player_id,
'tank',
choice(self.possible_player_locations),
[self.visible_sprites],
self.obstacle_sprites,
self.visible_sprites,
self.attack_sprites,
self.attackable_sprites
)
def reset(self):
for grass in self.grass_sprites:
grass.kill()
for enemy in self.enemy_sprites:
enemy.kill()
for style, layout in self.layouts.items():
for row_index, row in enumerate(layout):
for col_index, col in enumerate(row):
if int(col) != -1:
x = col_index * TILESIZE
y = row_index * TILESIZE
# Regenerate grass
if style == 'grass':
random_grass_image = choice(
self.graphics['grass'])
Terrain((x, y), [
self.visible_sprites,
self.obstacle_sprites,
self.attackable_sprites
],
'grass',
random_grass_image)
if style == 'entities':
if col == '500':
continue
if col == '400':
continue
else:
if col == '390':
monster_name = 'bamboo'
elif col == '391':
monster_name = 'spirit'
elif col == '392':
monster_name = 'raccoon'
elif col == ' 393':
monster_name = 'squid'
Enemy(monster_name,
(x, y),
[self.visible_sprites,
self.attackable_sprites],
self.visible_sprites,
self.obstacle_sprites)
for player in self.player_sprites:
player.animation.import_assets(
choice(self.possible_player_locations))
player.stats.health\
= player.stats.stats['health']
player.stats.energy\
= player.stats.stats['energy']
self.get_entities()
self.get_distance_direction()
self.dead_players = np.zeros(self.n_players)
self.done = False
def get_entities(self):
self.player_sprites = [sprite
for sprite in self.visible_sprites.sprites()
if sprite.sprite_type == 'player']
self.enemy_sprites = [sprite
for sprite in self.visible_sprites.sprites()
if sprite.sprite_type == 'enemy']
self.grass_sprites = [sprite
for sprite in self.visible_sprites.sprites()
if sprite.sprite_type == 'grass']
def get_distance_direction(self):
for player in self.player_sprites:
player.distance_direction_from_enemy = []
for enemy in self.enemy_sprites:
enemy.distance_direction_from_player = []
for player in self.player_sprites:
if not player.is_dead():
player_vector = pygame.math.Vector2(
player.animation.rect.center
)
for enemy in self.enemy_sprites:
enemy_vector = pygame.math.Vector2(
enemy.animation.rect.center
)
distance\
= (player_vector - enemy_vector).magnitude()
if distance > 0:
direction\
= (player_vector - enemy_vector).normalize()
else:
direction\
= pygame.math.Vector2()
enemy.distance_direction_from_player.append(
(distance, direction, player))
player.distance_direction_from_enemy.append(
(distance, -direction, enemy))
def apply_damage_to_player(self):
for enemy in self.enemy_sprites:
for distance, _, player in enemy.distance_direction_from_player:
if (distance < enemy.stats.attack_radius
and player._input.combat.vulnerable):
player.stats.health -= enemy.stats.attack
player._input.combat.vulnerable = False
player._input.combat.hurt_time = pygame.time.get_ticks()
def toggle_pause(self):
self.paused = not self.paused
def run(self, who='observer', fps='v0.9'):
# Draw the game
self.visible_sprites.custom_draw(self.observer)
self.ui.display(self.observer)
debug(f"{fps}")
if not self.paused:
# Update the game
for player in self.player_sprites:
if player.stats.health > 0:
player.attack_logic()
self.get_entities()
self.get_distance_direction()
self.visible_sprites.update()
self.apply_damage_to_player()
else:
debug('PAUSED')
for player in self.player_sprites:
self.dead_players[player.player_id] = player.is_dead()
self.done = True if (self.dead_players.all() == 1
or self.enemy_sprites == []) else False

View file

View file

@ -1,42 +0,0 @@
import os
import pygame
class Camera(pygame.sprite.Group):
def __init__(self):
super().__init__()
# General Setup
self.display_surface = pygame.display.get_surface()
self.half_width = self.display_surface.get_size()[0] // 2
self.half_height = self.display_surface.get_size()[1] // 2
self.offset = pygame.math.Vector2(100, 200)
# Creating the floor
script_dir = os.path.dirname(os.path.abspath(__file__))
image_path = os.path.join(
script_dir, '..', 'assets', 'graphics', 'tilemap', 'ground.png')
self.floor_surf = pygame.image.load(image_path).convert()
self.floor_rect = self.floor_surf.get_rect(topleft=(0, 0))
def custom_draw(self, player):
self.sprite_type = player.sprite_type
# Getting the offset
self.offset.x = player.rect.centerx - self.half_width
self.offset.y = player.rect.centery - self.half_height
# Drawing the floor
floor_offset_pos = self.floor_rect.topleft - self.offset
self.display_surface.blit(self.floor_surf, floor_offset_pos)
for sprite in sorted(self.sprites(), key=lambda sprite: sprite.rect.centery):
offset_pos = sprite.rect.topleft - self.offset
self.display_surface.blit(sprite.image, offset_pos)
def enemy_update(self, player):
enemy_sprites = [sprite for sprite in self.sprites() if hasattr(
sprite, 'sprite_type') and sprite.sprite_type == 'enemy']
for enemy in enemy_sprites:
enemy.enemy_update(player)

View file

@ -1,229 +0,0 @@
import os
import pygame
import numpy as np
from random import choice
from configs.system.window_config import TILESIZE
from utils.debug import debug
from utils.resource_loader import import_csv_layout, import_folder
from interface.ui import UI
from entities.observer import Observer
from entities.player import Player
from entities.enemy import Enemy
from .terrain import Tile
from .camera import Camera
class Level:
def __init__(self, n_players, reset=False):
# General Settings
self.game_paused = False
self.done = False
# Get display surface
self.display_surface = pygame.display.get_surface()
# Sprite Group setup
self.visible_sprites = Camera()
self.obstacle_sprites = pygame.sprite.Group()
self.attack_sprites = pygame.sprite.Group()
self.attackable_sprites = pygame.sprite.Group()
# Sprite setup and entity generation
self.create_map(n_players)
self.get_players_enemies()
self.get_distance_direction()
if not reset:
for player in self.player_sprites:
player.get_max_num_states()
player.setup_agent()
else:
for player in self.player_sprites:
player.get_max_num_states()
self.dead_players = np.zeros(len(self.player_sprites))
# UI setup
self.ui = UI()
def create_map(self, n_players):
player_id = 0
script_dir = os.path.dirname(os.path.abspath(__file__))
asset_path = os.path.join(
script_dir, '..', 'assets')
layouts = {
'boundary': import_csv_layout(f"{asset_path}/map/FloorBlocks.csv"),
'grass': import_csv_layout(f"{asset_path}/map/Grass.csv"),
'objects': import_csv_layout(f"{asset_path}/map/Objects.csv"),
'entities': import_csv_layout(f"{asset_path}/map/Entities.csv")
}
graphics = {
'grass': import_folder(f"{asset_path}/graphics/grass"),
'objects': import_folder(f"{asset_path}/graphics/objects")
}
for style, layout in layouts.items():
for row_index, row in enumerate(layout):
for col_index, col in enumerate(row):
if col != '-1':
x = col_index * TILESIZE
y = row_index * TILESIZE
if style == 'boundary':
Tile((x, y), [self.obstacle_sprites], 'invisible')
if style == 'grass':
random_grass_image = choice(graphics['grass'])
Tile((x, y), [self.visible_sprites, self.obstacle_sprites,
self.attackable_sprites], 'grass', random_grass_image)
#
# if style == 'objects':
# surf = graphics['objects'][int(col)]
# Tile((x, y), [self.visible_sprites,
# self.obstacle_sprites], 'object', surf)
if style == 'entities':
# The numbers represent their IDs in .csv files generated from TILED.
if col == '500':
self.observer = Observer(
(x, y), [self.visible_sprites])
elif col == '400':
if choice([0, 1]) == 1 and player_id <= n_players:
# Player Generation
Player(
(x, y),
[self.visible_sprites],
self.obstacle_sprites,
self.visible_sprites,
self.attack_sprites,
self.attackable_sprites,
'tank',
player_id)
player_id += 1
elif col == '401':
# Player Generation
Player(
(x, y),
[self.visible_sprites],
self.obstacle_sprites,
self.visible_sprites,
self.attack_sprites,
self.attackable_sprites,
'warrior',
player_id)
player_id += 1
elif col == '402':
# Player Generation
Player(
(x, y),
[self.visible_sprites],
self.obstacle_sprites,
self.visible_sprites,
self.attack_sprites,
self.attackable_sprites,
'mage',
player_id)
player_id += 1
else:
# Monster Generation
if col == '390':
monster_name = 'bamboo'
elif col == '391':
monster_name = 'spirit'
elif col == '392':
monster_name = 'raccoon'
elif col == ' 393':
monster_name = 'squid'
Enemy(monster_name,
(x, y),
[
self.visible_sprites,
self.attackable_sprites
],
self.visible_sprites,
self.obstacle_sprites)
def get_players_enemies(self):
self.player_sprites = [sprite for sprite in self.visible_sprites.sprites(
) if hasattr(sprite, 'sprite_type') and sprite.sprite_type in ('player')]
self.enemy_sprites = [sprite for sprite in self.visible_sprites.sprites(
) if hasattr(sprite, 'sprite_type') and sprite.sprite_type in ('enemy')]
def get_distance_direction(self):
for player in self.player_sprites:
player.distance_direction_from_enemy = []
for enemy in self.enemy_sprites:
enemy.distance_direction_from_player = []
for player in self.player_sprites:
player_vector = pygame.math.Vector2(player.rect.center)
for enemy in self.enemy_sprites:
enemy_vector = pygame.math.Vector2(enemy.rect.center)
distance = (player_vector - enemy_vector).magnitude()
if distance > 0:
direction = (player_vector - enemy_vector).normalize()
else:
direction = pygame.math.Vector2()
enemy.distance_direction_from_player.append(
(distance, direction, player))
player.distance_direction_from_enemy.append(
(distance, -direction, enemy))
def apply_damage_to_player(self):
for enemy in self.enemy_sprites:
for distance, _, player in enemy.distance_direction_from_player:
if distance < enemy.stats.attack_radius and player._input.combat.vulnerable:
player.stats.health -= enemy.stats.attack
player._input.combat.vulnerable = False
player._input.combat.hurt_time = pygame.time.get_ticks()
def toggle_menu(self):
self.game_paused = not self.game_paused
def run(self, who='observer'):
# Draw the game
if who == 'observer':
self.visible_sprites.custom_draw(self.observer)
self.ui.display(self.observer)
else:
self.visible_sprites.custom_draw(self.player)
self.ui.display(self.aaa)
debug('v0.8')
if not self.game_paused:
# Update the game
for player in self.player_sprites:
player.attack_logic()
self.get_players_enemies()
self.get_distance_direction()
self.visible_sprites.update()
self.apply_damage_to_player()
else:
debug('PAUSED')
for player in self.player_sprites:
if player.is_dead():
self.dead_players[player.player_id] = True
self.done = True if self.dead_players.all() == 1 else False

View file

@ -1,83 +0,0 @@
import random
import torch as T
import numpy as np
import matplotlib.pyplot as plt
from game import Game
from tqdm import tqdm
from os import environ
environ['PYGAME_HIDE_SUPPORT_PROMPT'] = '1'
random.seed(1)
np.random.seed(1)
T.manual_seed(1)
n_episodes = 2000
game_len = 5000
figure_file = 'plots/scores_mp.png'
game = Game()
agent_list = [0 for _ in range(game.max_num_players)]
score_history = np.zeros(
shape=(game.max_num_players, n_episodes))
best_score = np.zeros(game.max_num_players)
avg_score = np.zeros(game.max_num_players)
for i in tqdm(range(n_episodes)):
# TODO: Make game.level.reset_map() so we don't __init__ everything all the time (such a waste)
if i != 0:
game.level.__init__(reset=True)
# TODO: Make game.level.reset_map() so we don't pull out and load the agent every time (There is -definitevly- a better way)
for player in game.level.player_sprites:
player.stats.exp = score_history[player.player_id][i-1]
for agent in agent_list:
player.agent = agent_list[player.player_id]
agent_list = [0 for _ in range(game.max_num_players)]
for j in range(game_len):
if not game.level.done:
game.run()
game.calc_score()
for player in game.level.player_sprites:
if player.is_dead():
agent_list[player.player_id] = player.agent
player.kill()
# if (j == game_len-1 or game.level.done) and game.level.enemy_sprites != []:
# for player in game.level.player_sprites:
# for enemy in game.level.enemy_sprites:
# player.stats.exp *= .95
for player in game.level.player_sprites:
if not player.is_dead():
agent_list[player.player_id] = player.agent
exp_points = player.stats.exp
score_history[player.player_id][i] = exp_points
avg_score[player.player_id] = np.mean(
score_history[player.player_id])
if avg_score[player.player_id] > best_score[player.player_id]:
best_score[player.player_id] = avg_score[player.player_id]
print(f"Saving models for agent {player.player_id}...")
player.agent.save_models(
actr_chkpt=f"player_{player.player_id}_actor", crtc_chkpt=f"player_{player.player_id}_critic")
print("Models saved ...\n")
print(
f"\nCumulative score for player {player.player_id}: {score_history[0][i]}\nAverage score for player {player.player_id}: {avg_score[player.player_id]}\nBest score for player {player.player_id}: {best_score[player.player_id]}")
plt.plot(score_history)
plt.savefig(figure_file)
game.quit()
plt.show()

Binary file not shown.

Before

Width:  |  Height:  |  Size: 19 KiB

264
pneuma.py Normal file
View file

@ -0,0 +1,264 @@
import os
import argparse
import torch as T
import numpy as np
import matplotlib.pyplot as plt
from tqdm import tqdm
from game import Game
if __name__ == "__main__":
# Create parser
parser = argparse.ArgumentParser(
prog='Pneuma',
description='A Reinforcement Learning platform made with PyGame'
)
# Add args
parser.add_argument('--no_seed',
default=False,
action="store_true",
help="Set to True to run without a seed.")
parser.add_argument('--seed',
type=int,
default=1,
help="The seed for the RNG.")
parser.add_argument('--n_episodes',
type=int,
default=300,
help="Number of episodes.")
parser.add_argument('--ep_length',
type=int,
default=5000,
help="Length of each episode.")
parser.add_argument('--n_players',
type=int,
default=1,
help="Number of players.")
parser.add_argument('--chkpt_path',
type=str,
default="agents/saved_models",
help="Save/load location for agent models.")
parser.add_argument('--figure_path',
type=str,
default="figures",
help="Save location for figures.")
parser.add_argument('--horizon',
type=int,
default=200,
help="The number of steps per update")
parser.add_argument('--show_pg',
default=False,
action="store_true",
help="Set to True to open PyGame window on desktop")
parser.add_argument('--no_load',
default=False,
action="store_true",
help="Set to True to ignore saved models")
parser.add_argument('--gamma',
type=float,
default=0.99,
help="The gamma parameter for PPO")
parser.add_argument('--alpha',
type=float,
default=0.0003,
help="The alpha parameter for PPO")
parser.add_argument('--policy_clip',
type=float,
default=0.2,
help="The policy clip")
parser.add_argument('--batch_size',
type=int,
default=64,
help="The size of each batch")
parser.add_argument('--n_epochs',
type=int,
default=10,
help="The number of epochs")
parser.add_argument('--gae_lambda',
type=float,
default=0.95,
help="The lambda parameter of the GAE")
args = parser.parse_args()
np.random.seed(args.seed)
T.manual_seed(args.seed)
n_episodes = args.n_episodes
episode_length = args.ep_length
n_players = args.n_players
chkpt_path = args.chkpt_path
figure_folder = args.figure_path
horizon = args.horizon
learnings_per_episode = int(episode_length/horizon)
learn_iters = 0
show_pygame = args.show_pg
# Setup AI stuff
score_history = np.zeros(shape=(n_players, n_episodes))
best_score = np.zeros(n_players)
actor_loss = np.zeros(shape=(n_players,
n_episodes))
critic_loss = np.zeros(shape=(n_players,
n_episodes))
total_loss = np.zeros(shape=(n_players,
n_episodes))
game = Game(show_pg=show_pygame, n_players=n_players)
print("Initializing agents ...")
for player in game.level.player_sprites:
player.setup_agent(
gamma=args.gamma,
alpha=args.alpha,
policy_clip=args.policy_clip,
batch_size=args.batch_size,
N=args.horizon,
n_epochs=args.n_epochs,
gae_lambda=args.gae_lambda,
chkpt_dir=chkpt_path,
no_load=args.no_load
)
# Episodes start
for episode in tqdm(range(n_episodes),
dynamic_ncols=True):
# This handles agent continuity, as well as score persistence
game.level.reset()
episode_actor_loss = np.zeros(
shape=(n_players, learnings_per_episode))
episode_critic_loss = np.zeros(
shape=(n_players, learnings_per_episode))
episode_total_loss = np.zeros(
shape=(n_players, learnings_per_episode))
# Main game loop
for step in tqdm(range(episode_length),
leave=False,
ascii=True,
dynamic_ncols=True):
if not game.level.done:
game.run()
if step % horizon == 0:
for player in game.level.player_sprites:
player.agent.learn()
episode_actor_loss[player.player_id][learn_iters % learnings_per_episode]\
= player.agent.actor_loss
episode_critic_loss[player.player_id][learn_iters % learnings_per_episode]\
= player.agent.critic_loss
episode_total_loss[player.player_id][learn_iters % learnings_per_episode]\
= player.agent.total_loss
learn_iters += 1
# Gather information about the episode
for player in game.level.player_sprites:
# Update score
score_history[player.player_id][episode] = player.stats.exp
# Update actor/critic loss
actor_loss[player.player_id][episode] = np.mean(
episode_actor_loss)
critic_loss[player.player_id][episode] = np.mean(
episode_critic_loss)
total_loss[player.player_id][episode] = np.mean(
episode_total_loss)
# Check for new best score
if player.stats.exp > best_score[player.player_id]:
print(f"\nNew best score:\t {player.stats.exp}\
\nOld best score: \t {best_score[player.player_id]}")
best_score[player.player_id] = player.stats.exp
print(f"Saving models for player {player.player_id}...")
# Save models
player.agent.save_models(
f"A{player.player_id}",
f"C{player.player_id}")
print(f"Models saved to {chkpt_path}")
else:
print(f"\nScore this round for player {player.player_id}:\
{player.stats.exp}")
# End of training session
print("End of episodes.\
\n Exiting game...")
game.quit()
plt.figure()
plt.title("Player Performance")
plt.xlabel("Episode")
plt.ylabel("Score")
plt.legend([f"Player {num}" for num in range(n_players)])
for player_score in score_history:
plt.plot(player_score)
plt.savefig(f"{figure_folder}/score.png")
plt.figure()
plt.suptitle("Actor Loss")
plt.xlabel("Episode")
plt.ylabel("Loss")
plt.legend([f"Agent {num}" for num in range(n_players)])
for actor in actor_loss:
plt.plot(actor)
plt.savefig(f"{figure_folder}/actor_loss.png")
plt.figure()
plt.suptitle("Critic Loss")
plt.xlabel("Episode")
plt.ylabel("Loss")
plt.legend([f"Agent {num}" for num in range(n_players)])
for critic in critic_loss:
plt.plot(critic)
plt.savefig(f"{figure_folder}/critic_loss.png")
plt.figure()
plt.suptitle("Total Loss")
plt.xlabel("Episode")
plt.ylabel("Loss")
plt.legend([f"Agent {num}" for num in range(n_players)])
for total in total_loss:
plt.plot(total)
plt.savefig(f"{figure_folder}/total_loss.png")
plt.show()

View file

@ -1,85 +0,0 @@
import torch as T
import numpy as np
import matplotlib.pyplot as plt
from game import Game
from tqdm import tqdm
from os import environ
environ['PYGAME_HIDE_SUPPORT_PROMPT'] = '1'
np.random.seed(1)
T.manual_seed(1)
n_episodes = 300
game_len = 10000
n_players = 8
figure_file = 'plots/score_sp.png'
game = Game(n_players)
agent = game.level.player_sprites[0].agent
score_history = np.zeros(shape=(game.max_num_players, n_episodes))
best_score = 0
avg_score = np.zeros(n_episodes)
for i in tqdm(range(n_episodes)):
# TODO: Make game.level.reset_map() so we don't __init__ everything all the time (such a waste)
if i != 0:
game.level.__init__(n_players, reset=True)
# TODO: Make game.level.reset_map() so we don't pull out and load the agent every time (There is -definitevly- a better way)
for player in game.level.player_sprites:
player.stats.exp = score_history[player.player_id][i-1]
player.agent = agent
for j in tqdm(range(game_len)):
if not game.level.done:
game.run()
game.calc_score()
for player in game.level.player_sprites:
if player.is_dead():
player.kill()
# if (j == game_len-1 or game.level.done) and game.level.enemy_sprites != []:
# for player in game.level.player_sprites:
# for enemy in game.level.enemy_sprites:
# player.stats.exp *= .95
for player in game.level.player_sprites:
exp_points = player.stats.exp
score_history[player.player_id][i] = exp_points
avg_score[i] = np.mean(score_history)
if avg_score[i] >= best_score:
print(
f"\nNew Best score: {avg_score[i]}\
\nOld Best score: {best_score}"
)
best_score = avg_score[i]
print("Saving models for agent...")
agent.save_models(
actr_chkpt="player_actor", crtc_chkpt="player_critic")
print("Models saved ...\n")
else:
print(
f"Average score of round: {avg_score[i]}\
\nBest score: {np.mean(best_score)}"
)
print("\nEpisodes done, saving models...")
agent.save_models(
actr_chkpt="player_actor", crtc_chkpt="player_critic")
print("Models saved ...\n")
plt.plot(avg_score)
plt.savefig(figure_file)
game.quit()
plt.show()

Binary file not shown.

Binary file not shown.

Binary file not shown.

View file

@ -4,6 +4,7 @@ pygame.init()
font = pygame.font.Font(None, 30)
def debug(info, y=10, x=10):
display_surface = pygame.display.get_surface()
debug_surf = font.render(str(info), True, 'White')

View file

@ -1,9 +1,14 @@
import pygame
from csv import reader
from os import walk
import os
def import_csv_layout(path):
script_dir = os.path.dirname(os.path.abspath(__file__))
path = os.path.join(script_dir,
'..',
'assets',
path)
terrain_map = []
with open(path) as level_map:
layout = reader(level_map, delimiter=',')
@ -13,11 +18,25 @@ def import_csv_layout(path):
def import_folder(path):
script_dir = os.path.dirname(os.path.abspath(__file__))
path = os.path.join(script_dir,
'..',
'assets',
path)
surface_list = []
for _, __, img_files in walk(path):
for _, __, img_files in os.walk(path):
for image in img_files:
full_path = f"{path}/{image}"
full_path = os.path.join(path, image)
image_surf = pygame.image.load(full_path).convert_alpha()
surface_list.append(image_surf)
return surface_list
def import_assets(path):
script_dir = os.path.dirname(os.path.abspath(__file__))
return os.path.join(script_dir,
'..',
'assets', path)