IT training catchpoint ebook OReilly devops for media khotailieu

Co
m
pl
im
en
ts
of

DevOps for Media
and Entertainment
Accelerate Your Digital Transformation Through
Automation, Monitoring, and the Cloud

Zubin Irani & Brandon Cipes



DevOps for Media
and Entertainment

Accelerate Your Digital Transformation
through Automation, Monitoring,
and the Cloud

Zubin Irani and Brandon Cipes

Beijing

Boston Farnham Sebastopol

Tokyo

DevOps for Media and Entertainment
by Zubin Irani and Brandon Cipes
Copyright © 2017 O’Reilly Media, Inc. All rights reserved.
Printed in the United States of America.
Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA
95472.
O’Reilly books may be purchased for educational, business, or sales promotional use.
Online editions are also available for most titles ( For more
information, contact our corporate/institutional sales department: 800-998-9938 or


Editors: Brian Anderson and
Virginia Wilson

Production Editor: Colleen Cole
July 2017:

Copyeditor: Octal Publishing, Inc.
Interior Designer: David Futato
Cover Designer: Karen Montgomery

First Edition

Revision History for the First Edition
2017-07-10:

First Release

The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. DevOps for Media
and Entertainment, the cover image, and related trade dress are trademarks of
O’Reilly Media, Inc.
While the publisher and the authors have used good faith efforts to ensure that the
information and instructions contained in this work are accurate, the publisher and
the authors disclaim all responsibility for errors or omissions, including without
limitation responsibility for damages resulting from the use of or reliance on this
work. Use of the information and instructions contained in this work is at your own
risk. If any code samples or other technology this work contains or describes is sub‐
ject to open source licenses or the intellectual property rights of others, it is your
responsibility to ensure that your use thereof complies with such licenses and/or
rights.

978-1-491-98745-2
[LSI]


Table of Contents

DevOps in Media and Entertainment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
New Media Was First in DevOps
Digital Disruption in Media and Entertainment
DevOps Defined: Six Key Competencies
Advancing Beyond Agile
Benefits and Limits of DevOps
CI/CD
Dynamic Cloud Infrastructure
Automated Testing
Security Automation/DevSecOps
Seven DevSecOps Imperatives

Monitoring
Beginning Your DevOps Journey
DevOps and the Next Media and Entertainment Disruptor

2
4
6
7
8
10
14
17
18
20
22
24
26

iii



DevOps in Media and
Entertainment

Traditional media and entertainment businesses are struggling with
digital transformation on a more dramatic scale than those in any
other industry. Media and entertainment executives who have been
around long enough have experienced the move from running
entirely on analog data and devices to becoming wholly digital com‐

panies. More recently, they’ve made the rapid leap from embracing
software products as a tool to realizing that software—operating in
the cloud like a living, breathing organism needing constant care
and feeding—is their product. And they’ve seen the fundamental
definition of what a media and entertainment business is expand as
methods of content creation and consumption have exploded.
And the changes are still coming. More than 70 percent of media
and entertainment industry executives who responded to a poll last
year said they anticipated “moderate or massive digital disruption”
in the next 12 months. Whatever their sector—movies, music, gam‐
ing, television, radio, or publishing—the leaders of these businesses
know that speed and agility are crucial to keeping up with the pace
of technological change, consumer expectations, and their competi‐
tors.
We believe that the key to enabling this level of agility is a highly col‐
laborative and communicative relationship between Development
and IT Operations. The merged term DevOps symbolizes the inte‐
gration that needs to occur between the two departments. New
media leaders such as Facebook and Google are built upon DevOps.
And among traditional media and entertainment businesses, the
most progressive have already adopted DevOps. In this report, we
1


focus on a few of them as case studies, including Netflix, Disney, and
Sony.
Across all industries, 80 percent of “global 1,000” organizations are
expected to implement DevOps by 2019. And yet, as many as 60
percent of leaders in organizations of between 500 and 10,000
employees don’t know what DevOps is or are unsure if their compa‐

nies practice it.
In our view, the first step to “doing” DevOps is to understand the
disciplines that comprise a DevOps journey. In this report, we aim
to enlighten media and entertainment executives about the compo‐
nents of a DevOps transformation and to show the ways that adopt‐
ing DevOps practices will enable their businesses to evolve with
their industry’s ever-changing landscape. We also aim to convince
readers who don’t already agree with us on this point: media and
entertainment enterprises that don’t implement a DevOps approach
will inevitably fall far behind their industry’s leaders.

New Media Was First in DevOps
It’s no surprise that the earliest example of DevOps in action came
from a new media business that was challenged to house and serve
massive banks of digital files.
In 2009, Flickr stored more than 3 billion photos that its users
accessed at a rate of 40,000 per second. And that year its operations
manager and a software engineer made history giving the first
known public talk about the importance of bridging the divide
between Development and Operations. The evidence for their argu‐
ment wowed the audience at O’Reilly’s Velocity 2009 conference:
Flickr was deploying new features to its website 10 times a day.
Back then, that rate of deployment was a big deal. O’Reilly’s confer‐
ence program noted:
Flickr takes the idea of “release early, release often” to an extreme—
on a normal day there are 10 full deployments of the site to our
servers. This session discusses why this rate of change works so
well, and the culture and technology needed to make it possible.

In their presentation, “10+ Deploys Per Day: Dev and Ops Coopera‐

tion at Flickr”, colleagues John Allspaw and Paul Hammond argued
that communications and cooperation between the traditionally
adversarial development and operations teams was the key to their
2

|

DevOps in Media and Entertainment


business’s ability to deploy new code, plan projects, and manage cri‐
ses quickly and effectively.
Allspaw and Hammond agreed at the time that only frequent, con‐
tinual improvements could keep a business like theirs from being
overtaken by the likes of Facebook and Twitter. But change wasn’t
the norm in their profession at the time. In fact, many viewed every
change as a site outage risk. The archetypal “ops person” would
argue against frequent software changes in favor of site stability, so
Allspaw and Hammond built tools and a culture of cooperation
between their two departments that allowed change to happen as
often as possible. DevOps and continuous delivery/continuous
deployment took the risk out of change by making it normal.
“As web infrastructures grow,” they said, “the line between systems
and software become quite blurred. Operations and development
are disciplines that historically have been limited to a predictable list
of responsibilities and have suffered from a culture of fingerpointiness.”1 To overcome that, they recommended simply that the
humans in development and the humans in operations try to learn
to think more like each other.
At the time, few traditional media and entertainment business lead‐
ers would have thought their development and IT operations teams

had something to learn from a five-year-old new-media startup. But,
thanks to its DevOps approach, Flickr has scaled to store 13 billion
digital photos and videos, and process millions per day for its online
customers, while the structure and function of traditional media and
entertainment businesses have evolved to have much in common
with them. To be sure, even today, releases as frequent as what Flickr
cited in 2009 are not the norm in most industries. Few application
development and IT operations managers claim to release applica‐
tions more frequently than once per month. But as the digital revo‐
lution has forced all varieties of content providers to reinvent
themselves as technology businesses, Flickr’s approach to keeping
pace with consumer demand for device-agnostic online media
access has become a model for media businesses—old and new.

1 “10+ Deploys Per Day: Dev and Ops Cooperation at Flickr”.

New Media Was First in DevOps

|

3


Digital Disruption in Media and Entertainment
It’s no secret that top DevOps performers are taking the lead in the
media and entertainment industry. Netflix, which reported earlier
this year that its DevOps team facilitates 4,000 software deployments
per day, now considers its biggest competition not another video
streaming platform or traditional broadcaster, but its customers’
time asleep.2, 3

A scan of the job openings at publishers including The New York
Times, Dow Jones, and even 94-year-old book publisher W.W. Nor‐
ton reveal that DevOps engineers are also in demand in old media,
where print has given way to electronic content and websites are
competing with mobile apps for eyeballs. Digital delivery has also
completely transformed the businesses of film and music producers
and distributors. Mobile technology has revolutionized gaming.
And AI, machine learning, virtual and augmented reality, Internet
of Things, and Living Services are poised to transform them all
again.4, 5, 6, 7, 8
Meanwhile, accessible technologies have lowered barriers to entry in
all areas of the industry, so that content and competition has explo‐
ded, and innovative new entrants present serious threats to longtime leaders. Every sector of the industry is shapeshifting: Print
media looks more like broadcast media, video streaming platforms
are venturing into film and television production, and social media
platforms have morphed into providers of news and video. The
entire sector is melting into a single category of competitors.
The media & entertainment businesses that are thriving are also rebuilding their infrastructures—namely, they’re applying the practi‐
ces, methods, and cultural principles of DevOps to empower their

2 “Making multi-cloud deployment a reality at Netflix with Spinnaker”.
3 “Netflix’s Biggest Competition Is Sleep, Says CEO Reed Hastings”.
4 “Artificial Intelligence Will Revolutionize the Media Business. Here’s How.”
5 “Leveraging cloud-based predictive analytics to strengthen audience engagement”.
6 “What a venture capitalist sees in the virtual and augmented reality market”.
7 “Four digital trends reshaping the media industry”.
8 “Living Services: The next Wave in the Digitization of Everything”.

4

|

DevOps in Media and Entertainment


people, streamline their processes, and deliver best-in-class technol‐
ogy rapidly and effectively.
What digital accessibility has made possible for customers of every
industry rings particularly true for media and entertainment busi‐
nesses: Consumers glued to their screens expect consistent and reli‐
able experiences on whichever device they choose to use, whenever
they choose to use it. To stay vital, media and entertainment busi‐
nesses must deliver digital products that consistently satisfy their
customers’ expectations.
A report this year on digital transformation in media and entertain‐
ment from Accenture expresses the urgency:
To compete in this new ecosystem, new digital-based business
strategies and associated capabilities are essential. Broadcasters, tel‐
cos, and cable companies that hesitate now may spend years trying
to catch up.

The good news for media and entertainment companies is that
they’re better positioned than those in many other industries to ben‐
efit from the power of DevOps. The tools, methods, and processes of
DevOps are ideal for building and operating platforms that deliver
content to viewers, readers, listeners, and players wherever they are,
on any device, at any time. And it’s the proven best practice for
organizations tasked with distributing consumer-facing content
over the web.
A cost–benefit analysis might not be enough to persuade decision

makers in high-risk industries such as finance, healthcare, or medi‐
cal device technologies to undertake a DevOps transformation. But
for media and entertainment organizations, DevOps is a no-brainer.
Done right, it’s proven to reduce long-term costs and risk and
improve speed and nimbleness.
To be sure, the most established enterprises will struggle the most
with DevOps adoption. A DevOps transformation is a massive
change-management undertaking. Media and entertainment busi‐
nesses must take radical steps to reorganize their teams, reinvent the
ways they protect their content through digital rights management,
ensure compliance with the Digital Millennium Copyright Act, and
reboot their innovation process. The more years a culture and its
processes have existed, the greater the challenge. And an organiza‐
tion embarking on a total overhaul is likely to be overwhelmed by
the choices in the crowded DevOps tools market.
Digital Disruption in Media and Entertainment

|

5


Still, some of these companies are bravely (or desperately) hurling
themselves into the twenty-first century.

DevOps Defined: Six Key Competencies
DevOps is a practice, a methodology, and a culture centered on
empowering IT teams to deliver and support more robust software
services more quickly. Through new tools and the heavy use of auto‐
mation, a DevOps organization brings cross-functional teams

together to provide value faster than before. Like an Agile organiza‐
tion, a DevOps one also relies on incremental, iterative work pro‐
cesses to gather and respond quickly to feedback.
But, as our earlier Flickr example demonstrates, and as the parable
of Bill, the under-the-gun IT manager in the veritable DevOps bible,
The Phoenix Project (IT Revolution), shows, success is as much
about human cooperation as it is about technology. Doing DevOps
demands that you change your culture, get people to work together
toward common deliverables, and realign who’s in charge of what.
We like to say, “show me your organizational chart and I’ll tell you
how big a problem you have.” The more conventional and hierarchi‐
cal your chart, the bigger the problem. Flattening every silo might
not be a prerequisite to doing DevOps, but building skyways
between them is.
We approach DevOps transformation as a holistic process that
addresses the People, the Process, and the Technology across six dis‐
tinct competencies simultaneously. We dive deeper into each of
these in the sections that follow, and we share case studies that
demonstrate each one, but here’s how we define the six competen‐
cies necessary for a healthy DevOps practice:
Continuous integration (CI)
CI is the habit of regularly merging your code and validating the
results through unit tests to set your organization up for contin‐
uous delivery. Result: Speedier software development.
Continuous delivery (CD)
CD is a method by which teams work in very short cycles to
automate code building and testing. Result: Functional software
gets to where it’s needed, when it’s needed.

6

|

DevOps in Media and Entertainment


Dynamic cloud infrastructure
When CI and CD accelerate the organization’s velocity, a
dynamic, software-defined Infrastructure as a Service (IaaS) will
enable teams to create precisely configured environments on
demand. Result: Shed the most traditional bottleneck of software
development while saving money.
Test automation
This competency is about creating a process to automate repeti‐
tive and time-consuming formalized manual testing efforts,
especially when CI/CD methods have accelerated developers’
output. Result: Software operates as intended and teams are freed
up to discover issues in the code as it is being built.
Security automation
Also known as Development Security Operations (DevSecOps),
this competency infuses a security mindset into the execution of
a nimble and automated DevOps practice. It automates security
testing so that all known security concerns are being proactively
audited as part of the software delivery process. Result: Simpli‐
fied security audits and peace of mind.
Monitoring
When CI and CD and IaaS combine to speed up your team’s
activity and spread it out over many servers, monitoring is the
way to ensure that everything is being tracked and to measure
whether systems are operating as expected. Result: Your soft‐

ware relays the most critical information about your operations to
enable you to act on feedback and respond to problems quickly.
In a high-functioning DevOps organization, teams across testing,
development, infrastructure, security, and support will all live and
breathe these six competencies. Such a big-picture execution is not
easy—and it is vastly more complex than an Agile transformation.

Advancing Beyond Agile
Speaking of Agile, as Agile consultants, we’re about to say something
controversial. Agile was oversold, but DevOps will deliver. Sure,
Agile has done a great job helping people organize their work, and
we believe in the benefits of being an Agile operation before you
become a DevOps one. In fact, we see going Agile as the gateway to
your DevOps transformation.
Advancing Beyond Agile

|

7


When it comes to the digital requirements of competing in media
and entertainment, only through DevOps will you achieve true effi‐
ciencies in terms of speed to market, scalability, and the ability to
innovate better and faster.
Here’s why we say that. By adopting Agile practices, many organiza‐
tions improved on speed and agility in planning their application
development process. But Agile doesn’t speak to the technical tools
and processes that systems administrators, quality assurance, and
engineering departments follow, technical execution became a bot‐

tleneck. Changing how those teams work to fit the Agile approach
demands a major DevOps undertaking.
For instance, when one organization we worked with began working
in two-week sprints, their project management teams were sailing.
But making changes to the software applications they managed
required coordinating between the software and operations team,
which could take months. Even though the software teams were
moving fast, the company wasn’t.
It’s little wonder that Agile practices and the move to the cloud are
the top factors driving interest in DevOps. Just as an Agile approach
breaks up large tasks into iterative steps, DevOps organizes teams
and system architectures into smaller ones. It replaces the plodding
pace of building monolithic applications with microservices-based
application architecture. Instead of putting an entire application on
one server, for example, DevOps excels when apps are broken into
smaller, focused segments that reside on separate servers that can
communicate with one another. And, it introduces automation to
enable deployment daily instead of quarterly.
These DevOps best practices have been shown to reduce errors, fric‐
tion, and frustration and speed up delivery when businesses auto‐
mate environment management and deployments and integrate
continuously. And layering DevOps over Agile is proven to improve
new business growth and operational efficiency.

Benefits and Limits of DevOps
The advantages of DevOps done right go far beyond accelerating the
benefits of Agile, of course. The many and various results businesses
attribute to the practice make clear what media and entertainment
organizations stand to gain from undergoing a DevOps transforma‐

8

|

DevOps in Media and Entertainment


tion. We call the benefits “itys”—velocity, stability, and agility, which
all contribute to profitability.
In fact, high-performing DevOps-enabled organizations have
reported deploying 200 times more often, spending 22 percent less
on unplanned work, recovering from deployment failures 24 times
faster, spending 29 percent more time on new work, and failing onethird as often.9
How? DevOps empowers small teams to test and deploy their own
code to the customer; it saves teams from waiting in line for testing
or to coordinate with other teams or get on the release schedule; and
it better enables teams to respond to customer insights.
But not all businesses that claim to be doing DevOps are high per‐
formers or really “doing DevOps” in a legitimate way. Because it’s
not as clear cut as Agile, DevOps can create confusion for organiza‐
tions that don’t understand its nuances. Many companies taking
their first steps toward a DevOps transformation struggle with how
to get started, budgeting for the change, finding and integrating the
right tools, security concerns, a lack of expertise, inflexible project
management processes, resistant employees, and a lack of support
from top management. Even for teams that easily overcome those
hurdles, the DevOps journey is likely to be a long one.
The software company Atlassian developed a model to measure the
DevOps maturity of respondents to a survey who said they were
doing DevOps at some level. Based on their answers, Atlassian

ranked more than half at the “beginner” or “intermediate” level; 15
percent ranked at the “basic” level—less mature than beginners. In
the same study, Atlassian discovered that companies consider ach‐
ieving the DevOps cultural mindset—sharing tools and knowledge
between the development and operations teams—to be the easy
part. Implementing CD, automated testing, and monitoring is diffi‐
cult, they believe.
In our practical experience, however, the biggest challenge is getting
formerly independent departments to collaborate with one another.
And even there, many businesses that say they have a DevOps cul‐
ture are only part way there, for instance, only sharing information

9 “DevSecOps Transformation: The New DNA of Agile Business”.

Benefits and Limits of DevOps

|

9


when it’s asked for instead of in a consistently open format. That’s
not very DevOpsy.
One challenge to doing DevOps specific to the media and entertain‐
ment industry is constant change in the organizational chart. More
than half of media and entertainment industry executives who
responded to a recent survey said they expect to pursue acquisitions
in the next 12 months to advance their digital innovation capabili‐
ties. Change in leadership can be another obstacle on the path of a
DevOps journey if an executive team that bought in to its value and

understood the commitment is replaced by one that doesn’t.
Let’s talk now about those six competencies we consider crucial to
any DevOps practice:
• CI
• CD
• Dynamic cloud infrastructure
• Test automation
• Security automation/DevSecOps
• Monitoring

CI/CD
When it comes to delivering product or fixing problems, speed wins
in the competitive and fast-paced media and entertainment busi‐
ness. Keeping a pace is what CI and CD are about.
At the heart of CI and CD are the following questions:
• Why accumulate developed code while waiting to release it once
a quarter if you can release it in small pieces every day?
• And, after code is released, why not put it into use?
For many companies, this is a radical idea.
The CI/CD approach has three clear benefits:
• It allows the organization to isolate and correct problems faster.
• It saves developers from waiting for manual processes to get
working code into needed environments as quickly as possible.
10

|

DevOps in Media and Entertainment

• It provides the control framework for using automated testing
to ensure software is working as expected and ready to go at any
time.
In practicing CI, a developer tackles small pieces of code and merges
changes or new features with other developers’ code regularly—
whether it’s a few times a week, at a regular time each day, or even
automatically every time the code is committed to a shared reposi‐
tory. The practice minimizes complicated merges, exposes integra‐
tion issues, and sets the organization up for CD.
CD is the practice of working in short cycles (“sprints” in Agilespeak) to automate the rebuilding of the code set as well as testing,
packaging, and placing code on a server. (CD is a similar concept
that speaks more specifically to automating the deployment of the
code to production environments.)
This iterative process allows QA teams to isolate problems earlier,
developers to make fixes more quickly, and functioning software to
be shipped to needed environments at any time. Instead of checking
for problems at the end of a process, CD ensures quality at each
stage. By leaning on CI to deliver code changes more often, this
downstream automation speeds up development and gives develop‐
ers the feedback loop they need to make improvements. When, for
instance, a code change creates a bug, referring to the recent itera‐
tive tests lets you quickly locate the source of the problem.
Developers at organizations that do CI/CD well reportedly spend
less time on unplanned work, more time on new features or code,
and less time correcting errors.10
Businesses new to the idea of CI/CD often need to scale some diffi‐
cult hurdles: Choosing from among the CI-enabling tools that are
available is one step, modifying them to fit your needs is another,
and releasing code every day is a big psychological stretch for devel‐
opers and operators. After they make the leap, however, and their

teams become nimble enough through CI/CD to accelerate develop‐
ment, the next hurdle will pop up: The servers and infrastructure
they release to will need to be nimble and automatically configured,
too.

10 “2016 State of DevOps Report Key Findings”.

CI/CD

|

11


In other words, to get to true DevOps levels of efficiency and flexi‐
bility, the CI/CD approach must be paired with a dynamic cloud
infrastructure.

CI/CD Case Study: Disney
As at most enterprises, development and operations teams at Dis‐
ney were once as separate as church and state. Developers handed
over code and deployment documentation to operations without
even interacting. Jason Cox, now director of systems engineering,
says that when he joined Disney in 2005, a year went by before he
met a developer.
But when leaders of Disney business units—from ABC to ESPN to
its movie studios—began pushing for speedier deliveries, a DevOps
journey began. First, to gain more respect from developers, the
operations team changed its name to Systems Engineering (it
worked), then its members embedded with product groups world‐

wide, joining the teams that developed software and performed
quality assurance and testing. The reorganization sparked continu‐
ous collaboration that hadn’t before been possible.
Cox, who explained Disney’s transformation at the DevOps Sum‐
mits in 2016 and 2015 in a talk called “Disney DevOps: The Enter‐
prise Awakens”, said those newly integrated groups next adopted
tools for CI and configuration management (Puppet Enterprise and
Chef). The process drove improvements across business units. For
instance, deployments of Disney Studios apps that formerly took
several hours could be done in a few minutes; time to buildout Dis‐
neyID apps for the Guest Data Services unit dropped from one day
to one minute; and deployment times for new employee tools for
the Corporate Social Business unit were reduced from days to 5 to
10 minutes. Today, Cox and his colleagues are known as DevOps
Jedi.

12

|

DevOps in Media and Entertainment


CI/CD Case Study: Associated Press
When the Associated Press (AP) began thinking about a DevOps
transformation in 2014, software development director Chad
Schorr said the goals were simple: deliver new features to the plat‐
form faster and improve the ability to make changes.11 Achieving
those would increase customer conversions and expand the busi‐
ness.

Schorr explained, “Adding a new way to help customers discover
content on one of our Web platforms could take months to turn
around in our search engine. Improving our user’s ability to find
exactly what they need within a huge archive of content directly
affects our bottom line.”
Changes weren’t frequent enough to demand daily deployments
like the super DevOps performers. They just wanted to shorten the
time it took to deliver a search change from months to days.
Schorr says those goals led a cross-department team to the decision
to pursue CD. “Continuous delivery reduces turnaround times
through heavy automation and increases the visibility of change via
deployment pipelines,” he wrote. “Any change to a system, whether
it’s functional or infrastructure, kicks off an automated pipeline that
fully provisions your solution and runs an automated test suite to
validate the change. If all tests pass, your change is ready to be
released to production.”
One year into the organization’s CD adoption, Schorr attested that
the benefits of DevOps and continuous delivery he had been read‐
ing about are real. “I’ve seen some dramatic improvements in my
team’s ability to reliably turn around changes using the automation
we’ve invested in.” But, he warned, “The journey toward realizing
the returns from DevOps is longer and more painful than most
anticipate...You must define a vision, commit an upfront invest‐
ment, and work through the organizational growing pains to realize
the benefits of DevOps.”

11 “How I Learned To Tune Out The DevOps Buzz”.

CI/CD

|

13


Dynamic Cloud Infrastructure
Every production, test, and development “instance” of software
needs to reside somewhere. To build and configure new server envi‐
ronments, IT organizations have long relied on golden copies, oneoff scripts, and manual configuration, which has led to slow and
error-prone deployments.
When your teams accelerate development through CI and CD,
you’ll be faced with major bottlenecks if you don’t expand your
server space and power. DevOps organizations see multiple develop‐
ers, be they 5 or 500, releasing and testing on the same day. To keep
pace without bankrupting the business, dynamic cloud infrastruc‐
ture is the DevOps answer.
Dynamic cloud infrastructure is the key DevOps-enabling tool.
Especially in the media and entertainment industry, for which pro‐
duction traffic ebbs and flows throughout the day and in response to
a variety of predictable and unpredictable triggers, the DevOps con‐
cept wasn’t practical until the Amazon Web Services public cloud
came along. Even Netflix, with its 4,000 daily deploys, relies on
AWS. The company’s video streaming usage swells to occupy twothirds of all internet traffic in the United States at night, which is
only possibly with AWS.
In fact, deciding whether to rely on the public cloud or set up a pri‐
vate one is a dilemma for many companies. The fact is, on its own
servers—a private cloud—no company can possibly scale to the level
that one on a public cloud can. We won’t say that you can’t do
DevOps on a private cloud, but the truth is, you can’t do as much.
And for media and entertainment companies whose work is predi‐

cated on massive data and files, private cloud is rarely the best
choice.
One exception is the giant Disney. The company invested heavily to
build a private cloud for its Disney Interactive video gaming and
web properties.12 Then, two years ago it began “making history as
the first major combo broadcaster and cable programmer to begin
to build a global distribution company entirely on...its own global
broadcast cloud”. Disney is big enough to afford it. But the capital

12 “Disney Builds Private Cloud for Videogame Empire”.

14

| DevOps in Media and Entertainment


expenditure might look questionable in comparison to features
available in five years from cloud providers like Amazon, Google, or
Microsoft.
With IaaS, DevOps teams can configure the environments they
need, when they need them. Your team needs additional servers for
deploying software? They’re available. When they don’t, you’re not
paying for those unused servers.
Conversely, the desire to move to the cloud can be a back door to
DevOps. Asked what the secret to cloud success is, CloudExpo 2016
attendees overwhelmingly responded with a mix of Agile and
DevOps. Indeed, DevOps deployment gets the most from the cloud.
Most organizations that deliver applications monthly or more fre‐
quently rely on cloud-based environments. Amazon itself reports:
“Media and entertainment companies are increasingly taking advan‐

tage of the scalability, elasticity, and security of the AWS Cloud to
enable their businesses in new ways.” Still, plenty of media and
entertainment companies don’t. It’s a tough move for businesses that
already have spent tens of millions of dollars on in-house hardware.

Dynamic Cloud Case Study: Sony Digital Media Group
Within Sony Pictures Technologies, the Digital Media Group
(DMG) was developing a system to manage entertainment assets.
But it was hamstrung by manual processes, and software delivery
was taking too long. Months could go by before the group shipped
a piece of completely developed software to users.
To enable speedier delivery and simultaneously reduce infrastruc‐
ture and delivery costs, Sony DMG created an automated cloud
delivery system running the Amazon Web Services infrastructure.
With outside help, the group stood up a complete CD system on
seven AWS components for managing and provisioning resources,
managing application stacks, securely isolating cloud resources,
compute instances, storage, scalable DNS, and secure control of
access to AWS services and resources.
DMG executive director Charles Cole said that the integrated
approach enabled his group to be “more adaptive and responsive to
customers by releasing new features and changes based on business
needs.”
Other reported outcomes included “happier customers, happier
developers, and a significant cost reduction as releasing [became] a
Dynamic Cloud Infrastructure

|

15

‘non-event’ that happens with the click of a button.” And instead of
months to ship software to users, now it takes minutes.

Dynamic Cloud Case Study: Discovery Communications
With 155 worldwide television networks including Discovery
Channel, TLC, and Animal Planet, Discovery Communications has
1.8 billion cumulative subscribers in 218 countries and territories.
When the company wanted to upgrade and consolidate its website
infrastructure and gain the ability to scale quickly to meet demand,
but avoid a costly upfront one-time expense for updating its hard‐
ware, it turned to Amazon Web Services. Between January 2012
and June 2013, Discovery migrated more than 40 sites to the cloud.
Now it hosts all digital media there and expands or shrinks its
infrastructure as business requirements change by automatically
scaling each part of the stacks horizontally.
Discovery IT executives say that relying on the cloud infrastructure
enables them to focus on business initiatives without having to
manage hardware and infrastructure, and, meanwhile, administra‐
tors in the Digital Media division have grown their skills to be sys‐
tem engineers, which provided more benefit to the company.
“Without using the AWS API and services, we wouldn’t be able to
provide our staff with the tools we do. Our entire continuous deliv‐
ery system and our development platform are built around using
[them],” says senior systems engineer/architect Shawn Stratton.13

Dynamic Cloud Case Study: NewsCorp
The media conglomerate NewsCorp, with 400 million global view‐
ers and multiple lines of business, set some ambitious goals for a

cloud transformation it launched in FY2014 at its largest unit, Dow
Jones: Save $100 million annually by moving 75 percent of the com‐
pany’s computational power to the cloud and consolidating 50
global datacenters to just 6.
In a recent webcast, Dow Jones CIO Shaown Nandi says that might
seem normal in 2017, but in 2013, “people thought it was crazy.”

13 “AWS Case Study: Discovery Communications”.

16

|

DevOps in Media and Entertainment


The a 130-year-old public company was facing the challenges that
many large enterprises do on their DevOps journeys. Organiza‐
tional silos stood in the way of forming a cohesive cloud deploy‐
ment strategy; legacy apps required manual deployment;
compliance bottlenecks in production slowed releases, and slow
release cycles slowed time to market; and people thought they
should be working on security but didn’t know how.
In 2016, with implementation help from an outside team, the
DevOps transformation began delivering results. In 18 months, the
company jettisoned 2,500 physical servers, and dropped power con‐
sumption to less than its 2005 rates. Instead of randomly, develop‐
ers began committing code hundreds of times a day. Integration
rates jumped from monthly to three times a day, deployment to QA
went from monthly to every other week, and deployment to pro‐

duction went from quarterly to every other week.
As for security, after developers no longer needed to request access
to go into production to debug, they could fix and redeploy soft‐
ware on their own. Security became much less of an issue, Nandi
says.

Automated Testing
In the world of DevOps and highly integrated systems, where doz‐
ens or hundreds of applications are talking to each other, automated
testing is crucial. And it’s not just that automation itself is difficult,
but there’s so much more to be automated. The sheer volume
increase in applications and tools contributes to a heavier QA work‐
load. Whether it’s end-user or black-box testing, unit testing, inte‐
gration testing, regression testing, or performance testing, doing it
manually is no longer an option for keeping up with code across a
dynamically changing infrastructure in a DevOps practice. Auto‐
mating those tasks is the only way to maintain the accelerated
release pace in a DevOps environment.
To automate some types of tests is more straightforward than others,
but an automation strategy that covers every type of test is critical to
your team’s ability to validate the quality and functionality of your
software produced by CI/CD methods. A team that accelerates
CI/CD without ramping up testing simultaneously is asking for
trouble.

Automated Testing

|

17

DevOps practice executes tests as part of a software build. Incorpo‐
rating testing into CI/CD lets your developers improve applications
as they work. It also enables teams to more punctually discover
problems brought on by separate build and test processes, it ensures
that critical tests are always run, and it provides better test coverage.
Testing whole networked systems is also essential to make sure that
properly functioning independent systems also work correctly in
concert.
Another benefit of automated testing is the ability to segment test
cases. Automated testing can set your servers spinning for 36 hours
running a full test, especially for businesses that have larger net‐
works of systems and upward of 50,000 test cases to run.
By segmenting test cases, organizations can stay on the “release early
and often” track. Of course, understanding what kinds of changes
necessitate which levels of testing is crucial to avoid paralysis. If you
fail to get in front of testing, you’ll be buried dealing with bugs that
must be addressed immediately.
DevOps philosophy emphasizes that testing is as much an organiza‐
tional effort as it is technical. It touches so many moving parts that it
can’t be just the job of code developers or Q/A. You need a test case
management suite that can capture the entire ecosystem of test
cases, results, and roles—the stronger your testing, the more robust
your CI/CD.
The biggest obstacles to implementing automated testing are time,
budget, and culture. Companies don’t like to hear that integrating
these steps into the process will make it 20 percent more expensive
or take that much longer. But DevOps is a continuous cycle of ideat‐
ing, building, operating, and changing software that isn’t sustainable

without integrating this kind of testing into your software builds,
where Dev and Ops have mutual responsibility for testing.

Security Automation/DevSecOps
Just as automated functional testing is critical to ensuring that soft‐
ware operates as intended, automated security testing ensures that
all known security concerns are being checked as part of the soft‐
ware delivery process. Ultimately, by building automated security
tests into your development process, you will produce higherquality applications.

18

|

DevOps in Media and Entertainment


To be sure, most security teams report having a tough time keeping
up with the rapid pace of feature releases that DevOps teams pro‐
duce.14 Making matters more complicated is the fact that in a fastpaced development environment, typical applications are assembled
largely from open-source components and frameworks.15 Security
flaws seep into code unnoticed. The Heartbleed bug, for instance,
that enabled users to expose private security keys in some situations,
was coded into 70 percent of publicly available applications. Adding
automated scanning and other security testing to CI/CD will catch
more vulnerabilities earlier. It also means that you can trace and
audit what tests were done and when.
In a DevOps environment, security cannot be an afterthought
addressed at audit stage. As systems have become more complex,
elaborate, and technical, audits have become expensive and inade‐

quate. DevSecOps offers the advantage of checking for common vul‐
nerabilities as software is being built—not after it’s deployed into
production. By baking basic security checks into the build process,
you’re checking systems before they go online; you’re preventing
problems from even being allowed to exist. Open ports, user access,
unpatched libraries are all secured. For instance, if users can get into
an open port they could get into your system, and an unpatched
library could allow malware into your environment.
Media and entertainment DevOps teams are tasked with responsi‐
bility for protecting customer privacy, intellectual property, and dig‐
ital rights management/Digital Millennium Copyright Act
compliance, as well as compliance with Payment Card Industry Data
Security Standard. Organizations want to know, “Is all of our con‐
tent encrypted in a format that can’t be read by anyone not using my
iTunes player?” Or, “If we have a license to sell content in the US,
but not in China, have we built in regional protections?” Adopting
DevSecOps will also minimize the pain of audits: the method pro‐
duces version-controlled lists of all security checks performed with
every deployment. By making security part of the build process
instead of enforcing checks after-the-fact, DevSecOps drastically
reduces the time team members spend remediating issues. The
approach ensures that safe practices are being followed with every

14 “DevSecOps Transformation: The New DNA of Agile Business”.
15 “2017 DevSecOps Community Survey”.

Security Automation/DevSecOps

|

19