Ai

Motivated by my internal drive, I am deeply passionate about exploring AI systems in game development, with a focus on enhancing user experiences and streamlining production processes.

Pose Transfer - MSc Thesis Project

Recently I completed my master's in AI and Ml from Liverpool Jhon Mores University. I have taken Pose Transfer as my thesis topic. Here is the content of my video submission.

Win Rate Prediction for OZMM

Overview:
Win rate prediction involves iteratively tuning game levels to achieve a balance between actual and expected performance. This process often requires multiple iterations to stabilize win rates, making it both time-consuming and resource-intensive.

Problem Statement:
The goal is to predict the win rate of a level during its design phase, reducing the need for extensive manual iterations and optimizing the design process.

Proposed Approach:

Leverage the dataset of approximately 6,400 levels with their corresponding win rates.
Implement a machine learning model to predict win rates for new levels based on historical data.

Challenges:

The level data is stored in text format, which must be converted into a structured tabular format suitable for machine learning.
Mapping the transformed data to corresponding win rates requires careful preprocessing and feature extraction.

Example level file:

Text to Tabular format Conversion

I have created a script to convert this data into a tabular format, but the process is quite time-consuming.

Model Building

Approach 1: Linear Regression
I initially used a linear regression model to predict the win rate, but it was unsuccessful. Upon analysis, I realized that there is no linear relationship between the input variables (X) and the output variable (Y).

Approach 2: Tree-Based Models
Next, I experimented with various tree-based models, including Random Forest, Gradient Boosting, and XGBoost. While all models faced challenges, XGBoost outperformed the other two.

Evaluation Metrics:
To measure model performance, I utilized standard metrics such as Mean Squared Error (MSE) and R² Score.

Graph of actual winrate vs predicted winrate:

Approach 3: Deep Neural Network (DNN)

I implemented a Deep Neural Network (DNN) to predict the win rate. The model was fine-tuned using techniques such as dropout and normalization.

The training process lasted for approximately 22,000 epochs, taking about two days. However, the model failed to perform well, with a training loss of 50–55 and a validation loss of 150–160. This highlighted the need for more data to make the DNN approach effective.

To address this, I simulated additional data, increasing the dataset size from 6,400 to 32,000 samples.

After retraining the model, performance improved significantly, with a noticeable increase in the R² score.

However, the model still failed on the test data, indicating an overfitting issue.

Approach 3: Improvements

To tackle the overfitting issue, I incorporated batch normalization and additional dropout layers into the network architecture. Additionally, I transformed the win rate data into categorical bins and analyzed the distribution across these bins. This analysis revealed a significant class imbalance in the data.

I simulated the data while maintaining its original distribution and verified it using statistical measures.

This approach allowed me to achieve a balanced dataset with respect to win rate.

:

I also included two additional metrics: low win rate range and high win rate range into the data.

Result:

After training for about 500 epochs, the results during both training and validation were very promising:

I also tested the model with new data, and the performance was solid. The error was reduced to 36, and the R² score improved to 77%.

The result is very close compared with actual winrate.

Graph of Actual and predicted winrate

Next Steps:

I will develop a basic interface and provide it to the designers for testing on future levels. The results will then be compared with the actual player win rates.

Additionally, I plan to enhance the model by incorporating another metric: win rate bins. This can be predicted using a classification model.

Conclusion:

I took this project personally, dedicating my own time, weekends, and extra hours to complete it. Having completed my master's in AI and ML, I wanted to apply those skills to a real-world problem, which I hadn't had the chance to do before. This project has been one of the most challenging one, and I truly enjoyed the process.