Nightengale
Member
Plenty of games are announced/shown with stuff that aren't in-game footage.
-
Hey, guest user. Hope you're enjoying NeoGAF! Have you considered registering for an account? Come join us and add your take to the daily discourse.
From Stage to Screen: How fast do conference games release? (PS4, Xbox One)
- Thread starter Liabe Brave
- Start date
Plenty of games are announced/shown with stuff that aren't in-game footage.
Just a question on the methods directed towards the OP. Reading through them you will see that if a title was flashed on a screen or mentioned by a speaker, he didn't count that as when the clock started. He also didn't count sizzle reels.
So I'm personally just curious how he treated a situation like the one I mentioned.
LiquidSolid
Member
I take it games like Resogun, Alienation, Ori, Bound, etc. aren't included because they're considered "indie games" even though they're not indie at all and are just as much first party as any other game on that list?
Another question for the OP.
Unless reading this incorrectly you have Deep Down as the longest shown to screen in development at 55 months.
It appears you included the Last Guardian in your list from a previous post. Would that be the longest as it was announced in 2009 initially?
Unless reading this incorrectly you have Deep Down as the longest shown to screen in development at 55 months.
It appears you included the Last Guardian in your list from a previous post. Would that be the longest as it was announced in 2009 initially?
Wouldn't Nioh also be included
yep, that's interesting.
Though, I can understand why – budget + not enough gamedev experience.
CG announcements are included
These number are interesting, but these raw numbers doesn't take into account the scale of the game, or how important they are to the respective platform.
Big first party titles like Scalebound, Fable Legends, Phantom Dust are a bigger deal than smaller third party/indie games.
Also, I just got around to playing it and Sony probably should have cancelled The Last Guardian.
Edit: Did we hear anything official on Deep Down recently?
Big first party titles like Scalebound, Fable Legends, Phantom Dust are a bigger deal than smaller third party/indie games.
Also, I just got around to playing it and Sony probably should have cancelled The Last Guardian.
Edit: Did we hear anything official on Deep Down recently?
nod.gif, it's just GAF's #5 game of the year
I've gotten really tired of seeing all the huge game reveals that aren't coming out within, like, the next two years. Sony's E3 a few years ago was huge, what with FF7 and Shenmue and The Last Guardian, but two of those still don't even have a release date yet. Stuff's getting announced earlier and earlier.
To be faire for Sony and Shenmue, Sony only announced the opening of the game's kickstarter on their stage. Never did they announce the game being halfway through development.
As for Square Enix, well it is FF7... they need to hype it a bit longer dince they will separate the game in episodes/parts.
ForsakenLotus
Member
The Last Guardian and Final Fantasy Versus XIII skew public perception heavily. As does Kingdom Hearts III and Final Fantasy VII. So they have a few really high profile games that either had a significant delay between reveal and release or currently have games that are suffering from that with the expectation that it will be similar (though maybe not as drastic).
panda-zebra
Member
Some effort that, good work. I long suspected the state of play wasn't quite as weighted to one side or the other as GAF suggested, especially recently, it just didn't feel right.
There's a lot of 2nd/3rd party games in there, unless you count 1st party = published by Sony/MS.
Also I can't read that abload picture in 1st page, don't know if it's my problem but it's unreadable after I zoom
Great work btw
FinaruDensetsu
Member
FFXV and KH3 and likely FF7R aren't PS exclusives though so they would affect both equally.
That really worked out great for the Saturn didn't it?
DriftingSpirit
Member
Yeah, there isn't really a "second party". If it is published by them, they consider it first. If it isn't, third.There's a lot of 2nd/3rd party games in there, unless you count 1st party = published by Sony/MS.
Also I can't read that abload picture in 1st page, don't know if it's my problem but it's unreadable after I zoom
Great work btw
Liabe Brave
Member
"Indie" is being used in the sense of production value scale, not whether self-published or not. This is a common usage, and not just in video games (e.g. when people refer to "indie films" they almost never mean things like Star Wars: The Phantom Menace). I deployed the distinction this way because the people I was responding to seemed to be talking about big projects, exclusives and so forth.
In any case, including all platform-owner-published titles regardless of scale would not flip the results. In fact, it only deepens them, because in that case the Sony average interval would be 12.63, and the Microsoft one 15.07 months.
My counts are only for PS4/Xbox One versions of games, so for example GTA V is not tracked starting from its original announcement, neither is Ryse, etc. However, I did address this same question earlier in the thread. Even if we decided to add in the incredibly long transition times for Last Guardian, Final Fantasy XV, Until Dawn, and Nioh (over 10 years for that one alone!), Sony's average interval (14.12 months) would still be below Microsoft's.
I also posted numbers for big first-party titles alone, and the pattern still holds there too.
I didn't explicitly disqualify CG trailers from "footage shown", so yes I did count them. And here's another aspect where the data may be surprising. In fact, few games have initially been shown this way. Microsoft has done this 21 times, and Sony 20 times--that's only 16.2% and 7.3% of their total output respectively. And the usage went way down in 2016. Here's the yearly breakdown of CG trailers for each platform.
Code:
Microsoft Sony
2013 7 5
2014 5 4
2015 7 9
2016 2 2This was already addressed in the OP. Read the full thing.
Liabe Brave
Member
This is a necessary assumption, if we're not willing to wait until after the generation finishes before looking at the data. As mentioned, I actually discuss this effect in the OP. It artificially lowers the intervals, as you said, since recently-shown games are not actually out yet. And since Sony presents more games at conferences, that skew applies more strongly to them. Yet removing it is insufficient to reverse the general trend of the results. If we cut off the last two years, and only allow games shown at E3 2015 or earlier, the average intervals become about 15 months for each platform, with Sony still marginally lower.
I also thought it might be interesting to search for differences between the conferences themselves. Do the separate events have individual character, distinct from the others? On initial review, the answer appears to be affirmative.
("Other" here is primarily PlayStation Experience, though it also includes the two console reveal events, and the time Sony presented at Paris Games Week instead of Gamescom in Europe.)
Looking at the whole, it's no surprise E3 has featured the most games, since it's the only event both platforms present at every year. Apart from that, perhaps we can see a slight surplus of games stuck in development, but the relative flatness of mean, median, and mode is pretty suggestive of a normal distribution (with a long right tail, as all the conferences have).
Gamescom is notable in that it's got a high percentage of released games...but the ones that haven't released seem to have immense dev intervals. This makes sense, given that indies proportionally have taken up more stage slots. (As discussed upthread, indies--in the sense of scale--generally take longer to release than major studio games.)
Finally, we can see a notably truncated release interval for games presented at Tokyo Game Show. The median gap approaches 40% less than other events, and even the average is significantly reduced. Alongside the highest proportion of released titles, that means few examples languish in dev for extended periods. The maintenance of this rapid cycle of met deadlines with few blunders definitely is a strong contributor to Sony's overall statistics.
Humdinger
Member
Appreciate you compiling all that data. Must've been a lot of work. Thank you.
I do have a question, though.
Why would it be necessary to make any assumptions like that (that a game not out yet is considered "released" at the time of the data analysis)? Why not just restrict your analysis to the games that actually have been released, rather than making assumptions about the ones that haven't?
I do have a question, though.
Why would it be necessary to make any assumptions like that (that a game not out yet is considered "released" at the time of the data analysis)? Why not just restrict your analysis to the games that actually have been released, rather than making assumptions about the ones that haven't?
LukasTaves
Member
Try making it about all games which were published or they had some exclusivity deal temporary or otherwise.I have made changes to the Sony numbers posted, as there were some hidden rows of duplicate games that were mistakenly being included. I apologize for any confusion, but removing these 7 titles hasn't materially affected any of the conclusions posted so far.
Here's what the comparison looks like, when only considering first-party, non-indie games.
Here is the total list of games included.
Code:Bloodborne Crackdown 3 Days Gone Dead Rising 3 Death Stranding Detroit: Become Human Disney Fantasia Music Evolved Dreams Driveclub Fable Legends Farpoint Forza Horizon 2 Forza Horizon 3 Forza Motorsport 5 Forza Motorsport 6 Gears of War 4 God of War Gran Turismo Sport Gravity Rush 2 Gravity Rush Remastered Halo 5: Guardians Halo Wars 2 Halo: TMCC Horizon: Zero Dawn inFamous: First Light inFamous: Second Son Killer Instinct Killzone: Shadow Fall Knack Knack 2 Little Big Planet 3 Marvel vs. Capcom Infinite MLB 15: The Show MLB The Show 16 MLB The Show 17 New Everybody's Golf No Man's Sky Phantom Dust Project Spark Quantum Break Rare Replay Ratchet & Clank ReCore Rez Infinite RIGS: Mechanized Combat League Ryse Scalebound Screamride Sea of Thieves Shangheist Spiderman Starblood Arena State of Decay 2 Street Fighter V Sunset Overdrive Tearaway Unfolded The Last Guardian The Last of Us Part II The Last of Us Remastered The Order: 1886 Ultra Street Fighter IV Uncharted 4 Uncharted: The Lost Legacy Until Dawn WiLD Wipeout Omega Collection
This was already addressed in an earlier post. Sony has shown many more remasters (24 versus Microsoft's 3), but removing them doesn't change the overall result. Sony's average interval would rise to about 14 months, still half a month shorter than Microsoft's.
Keep in mind that the three months after E3 aren't a very high-selling period in general. So you would have the challenge of whipping up excitement during a relative lull, in addition to the effort to cause enough hype in a short period.
That way it would include all games they are involved somehow and remove games that they just marketed on their presser.
Liabe Brave
Member
That's just asking a slightly different question than I was interested in. I was trying to answer, "If you see a game for the first time in a press conference, what's likely to happen with it? What are the chances it'll be canceled? How long will it probably take to come out, based on the platform, game scale, exclusivity, event, etc.?"
What you're looking for is an answer to "If a shown game has been released or canceled already, how long did that take?" This query does eventually converge to mine, but only after every game of the generation comes out. Before then, it explicitly ignores games like Below, God of War, Crackdown 3, and Last of Us Part II...but obviously those titles are part of the dataset we're trying to characterize.
Your query isn't invalid, but it's essentially attempting to predict the whole generation, no matter how long it ends up lasting, based on a sample we're not sure is representative. Whereas my version also contains distorting assumptions, but thereby provides a solid floor which only alters as new games are shown. For this reason I prefer it. But with all the proper caveats in mind, why don't we pull your answer?
Here's what the results look like if we only count games that have been released or canceled.
I answered this question from someone else in passing earlier in the thread. If we use the list of all games published by Microsoft and Sony (in exchange for exclusivity periods, under hire, etc.), the average intervals are then 15.07 and 12.63 months respectively. I didn't bother to calculate medians, etc. because the change obviously doesn't affect the overall pattern.
Humdinger
Member
That's just asking a slightly different question than I was interested in. I was trying to answer, "If you see a game for the first time in a press conference, what's likely to happen with it? What are the chances it'll be canceled? How long will it probably take to come out, based on the platform, game scale, exclusivity, event, etc.?"
What you're looking for is an answer to "If a shown game has been released or canceled already, how long did that take?" This query does eventually converge to mine, but only after every game of the generation comes out. Before then, it explicitly ignores games like Below, God of War, Crackdown 3, and Last of Us Part II...but obviously those titles are part of the dataset we're trying to characterize.
Your query isn't invalid, but it's essentially attempting to predict the whole generation, no matter how long it ends up lasting, based on a sample we're not sure is representative. Whereas my version also contains distorting assumptions, but thereby provides a solid floor which only alters as new games are shown. For this reason I prefer it. But with all the proper caveats in mind, why don't we pull your answer?
Here's what the results look like if we only count games that have been released or canceled.
I answered this question from someone else in passing earlier in the thread. If we use the list of all games published by Microsoft and Sony (in exchange for exclusivity periods, under hire, etc.), the average intervals are then 15.07 and 12.63 months respectively. I didn't bother to calculate medians, etc. because the change obviously doesn't affect the overall pattern.
Thanks for running the analysis that way. That's what I was interested in (and if I may be presumptuous, what a lot of people are interested in) -- "Does Sony take longer to release games after they announce them than MS does?" I don't think the original model was fair in assessing that question, because there were assumptions being made about games that haven't released yet. I think to get an answer to the question, you have to stick to only games that have actually released.
Interesting to see that Sony is still about a month quicker. I'm surprised. I figured your assumptions favored Sony, and a "clean" analysis (restricted only to games that had actually released) would end up even or maybe favoring MS.
p.s. Just to clarify, you stated, "Your query isn't invalid, but it's essentially attempting to predict the whole generation." No, not really. I'm not trying to predict anything. I'm just interested in what the data show at this point.
Stumpokapow
listen to the mad man
I doubt a t-tested difference in means would be significant.
Melchiah
Member
Agreed. Thanks to the OP for compiling and presenting the data, which must have been quite an effort. It's nice to see gut feelings based on few outlier titles to be abolished by hard facts, including CG trailers being more common than they actually are.
I also found it interesting how games shown at TGS are more likely to be released in timely fashion. I'd be interested to see how open world games fare against more linear ones.
Liabe Brave
Member
The p-value derived from a t-test of the Microsoft and Sony interval sets (assuming equivalent variance) is .1655. So no, definitely not significant.
Liabe Brave
Member
Sorry, but I likely won't get around to splitting the games into types. To do it accurately would probably take as much research as all the rest of what I've done put together.
But let's recap what we have looked at. To start, here was the way I characterized the general expectations I think people had:
- Some games seem to be revealed too early, as evidenced by CG trailers when gameplay isn't ready
- Games get shown for multiple years before they actually come out
- The AAA blockbusters are especially prone to lengthy dev time
- Sony is worse about announcing early than Microsoft or Nintendo
Some games seem to be revealed too early, as evidenced by CG trailers when gameplay isn't ready
Only 10% of games for the whole generation (21 Microsoft, 20 Sony) have been shown as CG trailers, so this isn't rampant. And generally, games released after CG showings don't take a lot longer to come out than titles shown with gameplay first. However, there are issues on both sides. For Sony, in some instances CG does seem to have covered for overly early announcements. Deep Down, Rime, Dead Island 2, and Shadow of the Beast do seem to have been unready for public showing (especially since a couple of these games aren't guaranteed to come out at all). For Microsoft, development is pretty uniformly normal in length...but 4 such games have been canceled before release. That's a whopping 19% of their CG-shown titles canceled, as opposed to less than 2% of other titles.
Games get shown for multiple years before they actually come out
This isn't true in general. As my results show, both platforms average under 15 months to release, and over half the games on each come out before a year is up. (Though it must be acknowledged that these are slight underestimations, for reasons explained in the OP.) Disregarding indies--in the sense of games with relatively lower production values--drops these results even lower, as does looking just at first-party efforts, or looking just at games that've already released. The impression of overall lengthier times must stem from concentration on specific titles. Of all games shown, 109 (27.0%) aged less than six months; 112 more ( 27.7%) less than twelve months; 124 more (30.7%) less than two years; 51 (12.6%) less than three years; and only 8 (2.0%) took or are taking longer. That's over 85% of games younger than two years.
The AAA blockbusters are especially prone to lengthy dev time
This point is pretty straightforwardly untrue. Only 22 AAA games have aged more than two years. That's 11% of all AAA, as opposed to 19.1% of non-AAA. As mentioned upthread with more numbers, indies are significantly more likely to have extended development cycles.
Sony is worse about announcing early than Microsoft or Nintendo
I regrettably have no comparisons for Nintendo (though as explained in the OP their business has been different enough that I'm not sure direct juxtaposition would even make sense). But as explicated more fully in the rest of the thread, this conclusion doesn't hold versus Microsoft. Looking at all games, Microsoft's mean/median/modal intervals are longer. Looking at just AAA games, they're longer. Looking at just first-party AAA games, they're longer. Looking at just games announced through E3 2015, they're longer. Looking at just games that have been released, they're longer. Sony shows a lot of remasters/remakes, and those "easy" titles could be distorting the results...but removing them still leaves Microsoft with longer intervals. Some of Sony's games started development years earlier on different platforms, so maybe we should include those stretches...but even if we do (and even if we don't do the same for Microsoft), then Microsoft's numbers are still higher.
In short, many different counting schemes and adjustments have been proposed, but none of them is sufficient to reverse the situation seen in the wide count of all titles. Because the gap isn't enormous, there exist some multiplex combinations of sampling which will flip the order. (Though I doubt any reasonable protocol could get it heavily reversed.) But this fact runs up against a critical lesson that was brought home to me by SalesGAF discussions. If you're saying, "If you ignore [x], [y], and [z], then Sony would take longer to release games", that's exactly the same statement as "Sony do not take longer to release games, and [x], [y], and [z] are the reasons why."
Of course, all the discussion above is about properties of sets. There's still the possibility that variability differs between the platforms, which could lead to interval quartiles with divergent traits. For example, of the 22 AAA games which have aged more than two years, how many were first shown by Sony? The answer is 16, versus 6 from Microsoft, which seems to support the conjecture. But, Sony shows more games in total, at a ratio of about 2.6:1. So that spread is actually just what you should expect.
And this pushes us toward an explanation of why public perception may be skewed as it is. There are simply more long-delayed games on the Sony side, so it's easier to recall examples. Meanwhile, the fact that there's also more undelayed games doesn't register as a factor. (For instance, Sony is the only platform of the two that has released games less than a month after their first showing.) I believe this alone is sufficient to explain much of the usual narrative. That delayed Sony games spring easily to mind is quite compelling, and the associated reasoning is plausible. That's why until I ran the actual numbers I expected to find all this stuff to be easily true. It's exciting that the results were surprising!
D
Deleted member 465307
Unconfirmed Member
EDIT: Nevermind. Just read the conclusions above.
Very cool work, OP. The time and effort that must have taken is very impressive.
Very cool work, OP. The time and effort that must have taken is very impressive.
Banjo-Kazooie
Member
I realize nothing is perfect, but games always get missed in counts or not counted for reason "x". Like I'm not sure why Recore and Screamride is listed but not Zoo Tycoon when they all released at the same $40 price
Liabe Brave
Member
As per the title, this thread is about games that have been presented onstage at a press conference. Zoo Tycoon--along with tons of other games for both platforms--did not ever get that. I thought this was pretty clear, from the fact that I explicitly state it in the OP. Also, I'd think it'd be an obvious clue when only 404 games are tallied in the OP; that cannot possibly represent the entirety of both platforms' combined libraries.I realize nothing is perfect, but games always get missed in counts or not counted for reason "x". Like I'm not sure why Recore and Screamride is listed but not Zoo Tycoon when they all released at the same $40 price
Humdinger
Member
Nice work. Interesting analysis. I agree with your conclusions about why the perception may be the way it is. I always enjoy it when data contradicts popular conceptions, and this is a nice example of it.
It does only cover E3 2013 to 2016, though, so I suppose Xbox guys would be justified in wondering what the results would be, if we included all of the 360/PS3 era.
It'll be interesting to see how things look in another few years, too.
It does only cover E3 2013 to 2016, though, so I suppose Xbox guys would be justified in wondering what the results would be, if we included all of the 360/PS3 era.
It'll be interesting to see how things look in another few years, too.
Kain-Nosgoth
Member
wow, huge work, you're amazing OP!
Really interesting thread, would be cool if someone did the same for nintendo!
Really interesting thread, would be cool if someone did the same for nintendo!
Nightengale
Member
I've run my own numbers, looking solely at first-party games, with filters of games that are retail exclusives, digital exclusives, undated, unreleased, etc to filter the numbers and the numbers don't run far from Brave's.
The long and short of it:
Both companies have roughly the same amount of games that are in loong announcement cycles like Crackdown, Everybody's Golf, Wild and Quantum Break.
The games that people think are in long-dev cycles/announcements are only approximately 3-4 months longer than games like Halo 5 or Gears 4, which the only difference is that those games don't get a delay to spring.
Both companies have games with short release cycles too.
The long and short of it:
Both companies have roughly the same amount of games that are in loong announcement cycles like Crackdown, Everybody's Golf, Wild and Quantum Break.
The games that people think are in long-dev cycles/announcements are only approximately 3-4 months longer than games like Halo 5 or Gears 4, which the only difference is that those games don't get a delay to spring.
Both companies have games with short release cycles too.
Nightengale
Member
Not as comprehensive, but here:
https://docs.google.com/spreadsheets/d/1TEGrjyzTLtIQI6fkFSwImXH9CNk2G9yZJdcU51kBSqc/edit?usp=sharing
https://docs.google.com/spreadsheets/d/1TEGrjyzTLtIQI6fkFSwImXH9CNk2G9yZJdcU51kBSqc/edit?usp=sharing
Justified in wondering how things were in the past, perhaps, but not at all justified in wishing it was included in this analysis. It doesn't address any of the OPs major points.