Balancing
Player vs Player
Skill Level
This is again a really hard problem. When two players play against each other where one is vastly better than the other it makes little fun for both. One has no challenge, one has no chance to win. Pairing players with similar skill level is important.
Skill level shows another problem and that is the gap between the worst and best players. Look at some common PVP games like CSGO, AoE2, Dota2, League of Legends. You will notice that the worst players and best players, will play your game differently. Some gameplay features or tactics can be hard to use but offer a high payout when they are used correctly. In lower elos these tactics will have a low win-rate because they are hard and only better players know how to use.
Now the win-rate of builds/tactics is different depending on the skill level. A player that mastered a gameplay feature or tactic can reach a win-rate higher than 60%. Is this balanced? How would you handle this?
If you weaken the tactic that overall has a 45% win rate but because in high elo its win rate is much higher. The win rate in lower elo will go also down.
Gameplay
When each gameplay feature would be the same and homogonous the game itself would be quite boring. That's why most player vs player games offer a different set of starting options someone can choose and other options he can take in the course of the game.
Here a big problems shows. Most games are not "simple" like chess. Games like League of Legends (MOBA) have so many different options and combination of options a player can choose that defining what combinations are too strong or too weak is impossible.
Why? Because in the end you will only get the data which combination of options resulted in which win-rate. You won't understand why the player choose certain combinations. You actually can't deduce from the win-rate alone which combination is too strong or too weak.
Imagine in League of Legends that a certain anti health item build has a too high win-rate (57%). You now believe that this build is too strong and weaken its anti-health effect. Now the build win rate drops to 48%. Now you think you did a better job, while It's now weaker it's much closer to 50% then before. Sadly now a life steal build that was before around 47% Now skyrockets to 60%, because anti got so weak that players are now not even buying it.
If you weaken something, something else will get stronger or weaker. This relation is not easy to understand or to see on the first glance.
Take your favorite game PVP and think of some changes you would do to balance it better. Now think of the change in percentage that would occur up or down in win rate. It will most certainly be wrong.
Player versus Environment
Here we again face a difficult problem. I personally believe that in a PvE game we want to maximize fun not balance. This becomes a big problem when our PvE mode can also be played against players. Like in paradox or total war games. Here there is now difference between the game mode besides that an AI faction is now played by a player.
I don't think this really matters. When a game is not balanced for multiplayer but can be played as a multiplayer game. The players should create their own rule sets and standards to how to play the game. Banning certain tactics that are too overpowered.
To make this easy for players to create their own rule set the game should allow for specifying one. So the game engine itself can enforce the rules. It should be easy to tweak values and disable, enable features. With these options players will not complain about unbalanced features to you but instead to the maker of the rule set.
This allows the designer of the game to maximize fun for the player, while players that wish to play competitive can maximize their fun my creating a good rule set.