Back in August, UTUK Editor Jamie Kennedy conducted a study on the performance of FUT 14 in-forms (IFs) versus the rare golds, by comparing three different TOTS IFs (Hazard, Ibrahimovic and Suarez) with their non in-form counterparts. The study was named ‘FUT 14 Does Scripting / Handicapping Exist?? An Unbiased Statistics Report‘. This article will look to build upon the three player’s previously analysed and take new information on goal and match data for all player cards currently on sale, which have been used in at least 500 matches.
With the new information available, it’s possible to get a better understanding of the relationship between stats and performance. Besides that, the new data allows us to poke deeper into the handicap claim related to star players in general, which was the original subject of Jamie’s article.
The noble art of comparison
To compare the performance of some cards, we need to understand how the card stats impact the player’s performance. This is unfortunately a little more complicated than just comparing two cards with different performance stats, as the stats aren’t the only parameter affecting the measured performance or for that matter the actual performance.
I have illustrated this in the chart below:
- Player positioning: One of the annoyances about in-forms is position changes. As an example, Jeremy Menez is a RW but his in-form is a CF. Obviously a CF ought to score more goals than a RW, so we cannot use the goal efficiency increase to measure the impact of the in-form stat increase.
- Statistical inaccuracy: All cards in this survey have been used in at least 500 matches, maybe more. At a sample size of 500, statistical inaccuracy can account for performance gaps as large as 8%. Another issue is that although the cards have played 500 games, these games aren’t played by 500 different players. In fact, the number of human players involved may be a lot smaller. Some of the silver in-forms are quite rare (in some cases only about 10 are up for sale at the same time), so chances are that a few players may have a huge impact on the data.
- Availability of relevant performance data #1: The only performance data available are goal and assist statistics. Obviously, neither goal scoring nor assists are valid performance measures for goalkeepers and defenders. Goals and assists are relevant measures for some midfielders, but the issue with midfielders is that they may take on various roles depending on position and formation. Thus, I don’t think that the available performance data are relevant to midfielders, defenders or goalkeepers.
- Availability of relevant performance data #2: No matter what performance goals we use, the team mates will impact the result. I find this issue more prevalent with regards to assists, because getting an assist requires someone else to score a goal. Obviously, pairing Bale up with Messi will lead to more assists than pairing him up with Ciaran O’Connor (the lowest rated striker in the game). If there was a measure called “Chance creation”, I would definitely use that rather than assists.
- Quality of teammates: This obviously does play a role, and obviously, we may see that some players / cards score more goals, because they are part of a better team than the cards we are comparing against. This, however, shouldn’t make any difference as long as we compare two versions of the same card, unless there is a league (or nation!) change involved.
- Human skill and level of opposition: The goals-per-match ratio is the product of three factors: [Card stats] + [Player skill] – [Quality of opposition]. No card scores goals on their own, and obviously good cards score more goals when the opponent is bad or the player controlling them is skilled. If the players using IF Neymar on average are more skilled than the players who are using regular Neymar, this will have a positive impact on IF Neymar’s goal rate when compared to regular Neymar. And if the average opponents people get when using IF Neymar are stronger than the average opponents people face when using regular Neymar, this will have a negative impact on IF Neymar’s goal rate, when compared to regular Neymar.
We can definitely overcome some of the challenges above by only comparing attackers and players in the same position. What about human skill and level of opposition? Can’t we simply assume that these factors impact the measured performance equally of both the cards we want to compare?
Starting with the human skill aspect, this definitely could play a role for some in-forms because different versions of a card may appeal more to some people than to others. In FUT 14, Steven Caulker (74 rated IF silver) used to cost 10 x more than his 77-rated gold in-form. Because of the huge price difference, one could speculate that the 74-rated silver version had a higher appeal to expert gamers than the cheaper, 77-rated gold version. Hence, the 74-rated card may have performed better than the 77-rated card because of the skill level of the owners. There are definitely examples of the opposite as well.
Next, there is the skill level of the opponents. The fact is that most in-forms have a higher rating than the original card. Thus, players using the in-form version will get matched with slightly better opponent squads on average when playing friendlies. Obviously, there are bronze cards who have a higher goal ratio than CR7 simply because bronze squads mostly are used against other bronze squads through friendlies or bronze tournaments. Hence, one can’t assume that the average level of opposition is the same for in-forms as for non in-forms, unless they have the same rating.
Further, please notice that stat upgrades of regular TOTW in-forms usually are minor. Neymar’s total stats are upgraded by 2 %, so quite obviously we shouldn’t expect him to score 10 % more goals. With such small stat upgrades, there is a good chance that the other factors mentioned above will overrule the actual performance upgrade. And on top of that, the statistical inaccuracy of up to 8 % will make it hard to measure a 2 % performance upgrade.
To conclude from the thoughts above, we cannot simply assume that all performance differences – good or bad – are caused by the card stats. Further, we cannot assume that the lack of a (statistically significant) performance difference proves that there isn’t a performance difference. This however doesn’t imply that we can’t use card comparison to tell anything about the impact of stats at all.
Comparing some actual cards
In the table below, I have inserted the performance stats for all in-form strikers (and their regular version) issued until and including TOTW 3. I have decided to present the G/M ratios as a span covering the lowest and highest value from three observations (made within 2 days) to give you a hands-on idea about the level of statistical inaccuracy.
|Level||G / M||G / M|
|Cristiano Ronaldo (LW)||92->93||0,88 – 0,95||0,86 – 0,95|
|Neymar (LW)||86->87||0,64 – 0,65||0,65 – 0,67|
|Diego Costa||85->86||0,97 – 1,02||0,97 – 1,01|
|Salomon Rondon||80->81||0,87- 0,89||0,88 – 0,90|
|Mauro Icardi||76->78||0,82 – 0,84||0,82 – 0,86|
|Andre-Pierre Gignac||78->80||0,79 – 0,81||0,75 – 0,90|
|Bradley Wright-Phillips||72->73||0,93 – 0,98||0,87 – 1,03|
|Vidar Orn Kjartansson||69->72||1,10 – 1,14||1,09 – 1,14|
|Mana Dembele||66->70||0,92 – 0,94||0,91 – 0,95|
|Jamie Vardy||69->72||0,77 – 0,78||0,77 – 0,79|
G/M = goals per match, Black = in-form, Blue = non in-form
As you will notice, the performance changes between regular and in-form card generally aren’t statistically significant, as the performance spans of the in-form versions and the regular versions are overlapping in most cases.
However, there are observations to be made in this table. For instance, please look at the G/M difference between the two left wings, Neymar and Ronaldo: Ronaldo scores 40 % more goals, which definitely is statistically significant.
I believe we can rule out that this is an effect of Ronaldo-owners getting weaker opponents, as Ronaldo is rated 6 points higher than Neymar. In fact, swapping Neymar with Ronaldo will increase your squad rating by 1 point. Actually, Ronaldo-owners should be facing tougher opposition on average.
This leaves us with two options: Either the players owning a Ronaldo are far more skilled than the players owning a Neymar, or we are in fact seeing the results of Ronaldo’s 22 % higher shot power, 12 % better finishing, 39 % better heading accuracy, 3 % better pace, work rates and so on. Or perhaps both.
Since both cards are pretty expensive (Neymar: 302.000 / 915.000, Ronaldo: 2.275.000 / 2.625.000), I would expect that we will find most experienced gamers than casuals in both cases. Thus, my guess would be that differences in skill play a minor role in relation to the scoring performance gap.
If we look at all the gold strikers above, it generally appears as if there is a good link between stats and goals-per-match. However, it’s harder to isolate the effects of the other factors when we compare expensive players with inexpensive players and so on.
Regarding the silver players, I’m extremely hesitant to conclude anything because the cards are extremely rare. If only 10 cards are for sale (as was the case with Kjartansson at some point today), it’s likely that individual players will bias the G/M ratio heavily.
A few notes on the handicapping dispute
As mentioned in the introduction, it’s a widespread belief that teams with in-forms, TOTY, TOTS and star players in general are handicapped. In our recent handicapping survey, some 2/3 of the respondents answered that this claim was likely. To exemplify what this belief is about, I have put in a few quotes from the comments sections to some of the previous articles we have had on this subject:
- “I think it exists I had a complete tots bundesliga team and lost a substantial amount of games more than the standard gold team that consisted of the exact same players.”
- “I’ve seen tots players play like bronze players when facing weaker teams.”
- “I experience SLUGGISH players, when playing with TOTS-teams. Sluggish in FIFA: It feels like players balance, agility and reaction are very low.”
The consensus appears to be that star players actually underperform in direct comparison to other players.
When looking at the comparison table above, it’s immediately clear that there is no indication of an underperformance when comparing in-forms with regular cards.
Although we only have data concerning goal scoring, it goes without saying that you can’t maintain the same goal ratio if your players act sluggish in relation to turning, passing and maintain their balance. So, although we don’t measure these aspects directly, it doesn’t appear likely that there is an actual difference between in-forms and non in-forms with respect to these traits either.
Of course, we are just looking at regular, TOTW in-forms here, not TOTS as they haven’t come out yet. On the other hand, if there was something sinister going on, it ought to leave traces in Ronaldo’s performance when compared to other players, given that he is higher rated than the majority of the TOTS and TOTY players we will see later.
Below you see all gold left-wingers in La Liga and all gold strikers from the Bundesliga with pace >=77. I have taken the liberty to sort the players by price rather than rating, as I believe the prices reflect the quality of the player better than the ratings.
Even when considering the effects of human skill, which definitely has an impact here, everything suggest that better stats equals better performance.
If there was some mechanism handicapping squads with star players, then why does Ronaldo outperform Neymar (and all other La Liga LW’s) and why does Lewandowski score more goals than all other strikers in the Bundesliga?
Are in-forms worth the money? None of the 10 informs in our comparison were significantly better than their non-inform, so the basic idea of swapping a non-inform with the inform version of the same player most likely won’t make a lot of difference. What may make sense though is to invest in repositioned informs.
Are in-forms subject to handicapping? Even though there is a lot of statistical inaccuracy involved here, it’s still notable that none of the 10 informs in the comparison did perform much different than the regular version of the same player.
Feel free to share your opinions in the comments section below! Do you think scripting exists?