Big Data + Hollywood = ?

William Goldman, a two time Oscar-winning screenwriter, famously said, “Nobody knows anything…Not one person in the entire motion picture field knows for a certainty what’s going to work. Every time out it’s a guess and, if you’re lucky, an educated one.”

However, due to the explosion of data and analytics tools, we now have the ability to analyse patterns such as viewing behaviour and user feedback (through social media) real-time. Over time, this data could allow creators to predict the most effective casting and plot lines. For example, when the Times Group used insights from a data analytics app, Parse.ly to reorder their gallery stories, their page views went up by 70%. The editors now use the tool to shape the direction of stories, using real-time data of audience engagement.

Hollywood is already redefining its relationship between the creator and consumer: various tools are allowing production houses to incorporate audience reactions into how content gets created, marketed and distributed. IBM offers social sentiment analysis to gauge the emotional response to films through Twitter and Facebook responses. These tools can not only highlight the responses, but attempt to explain the reasons for these responses.

This of, course, is limiting: if you use historical data to predict future behaviour, you are unlikely to support potentially game-changing scripts or talent. Moreover, key Hollywood executives such as Richard Plepler and Nina Jacobson, have publicly stated that they believe that data science can only be used to an extent in content creation: in the end, the fundamentals- such as the quality of story, talent and production value- matter more.

Additionally, the literature on what makes a ‘hit’ TV show or film highlights that while there are certain formulas you can follow to create appealing content, ultimately even the Silicon Valley players such as Netflix and Facebook agree that data analytics are better utilised in 1) content distribution and audience targeting/ personalisation of contentrecommendations and 2) understanding audience engagement.

And yet, the content production process itself needs to adapt, given the 1) influx of capital into original programming, as tech players look to compete for high quality content and 2) changing audience behaviours and tastes and 3) the rapid globalisation of content distribution. In 2017 itself, Apple and Facebook both budgeted over $1 billion for original programming, and Netflix announced that it planned to spend $8 billion on content. This year, there are over 500 scripted TV shows being made- which is 2x the number in 2012.

The amount of content being made, the variety of content being made, and the locations at which the content is being created is increasing rapidly, while the supply of talent, and audience attentions spans stays constant. The industry has already been disrupted: it needs the tools to adapt. I believe data science can be used to disrupt the content production process itself.

Given that the most time consuming parts of the process can be 1) storyboarding and shot list generation 2) schedule optimisation and 3 budgeting: data science could be used to automate the more tedious processes, so that content creators can focus on the storytelling + other more creative parts of the process.

Project management apps are already helping production teams collaborate across teams and locations. However, these apps face severe challenges when it comes to large-scale adoption, given that production teams are often freelance teams that work together for a period of time, and come from a non tech savvy culture. When Netflix introduced an ecosystem of apps to improve efficiency, it found encouraging the adoption of this new technology difficult at the start.

However, using AI for ‘agile’ content creation could both reduce script rewriting time and improve production planning by running ‘what if’ scenarios to test script variations, and removing certain elements of production to reduce costs. This would also not require wholesale adoption across production teams, as is required in the case of production management apps.

Netflix is already pioneering this approach. As highlighted in it’s Medium Blog, this is a necessity, given the scale at which it is producing content:

“Each production is a mountain of operational and logistical challenges that consumes and produces tremendous amounts of data. At Netflix’s scale, this is further amplified to levels seldom encountered before in the history of entertainment. This has created opportunities to organize, analyze and model this data that are equally singular in history. This is where data science can aid the art of producing entertainment.”

To provide a few examples, the company models cost estimations of how much a production will cost, that can deal with data sparsity. It also generates schedules using mathematical optimisation. As the company expands globally, it is using visualisation to analyse resource dependency and anticipate production delivery patterns. Lastly it uses prediction algorithms to sequence and scale its content localisation slating across markets.

Clearly, the exploration of applying data science to pre and post content production is a nascent field. As more tech platforms begin to invest in content at a large-scale globally, the use of data science will become a necessity, both in terms of efficiency of content production and content economics. Given this approach plays into these companies’ strengths, I think a large-scale change in culture is likely afoot. I’m excited to see whether AI can successfully be applied to analysing scripts and storylines over the next few years; several ventures are already attempting to tackle this complex challenge.

A blue ocean space is using data science to identify new talent. The stars seem to be aligning in this area. The need for new, diverse voices in Hollywood- across production, acting and writing- is resoundingly clear. The appetite for this new crop of talent has already been validated at the box office. On the other hand, there is an entire generation of social media stars that has built up a following, and talent on Youtube that is waiting to be discovered. We’re already seeing companies attempting to use AI to identify future leaders: how long will it be before this is applied to future stars and influencers?

Big Data + Hollywood = ? was originally published in Hacker Noon on Medium, where people are continuing the conversation by highlighting and responding to this story.