The Future of Content Navigation:
Building the Byte Generation’s Guide to Life the Universe and Everything
As consumers peer into the ever-widening chasm of choice, how can they be sure they’re not
missing out? This applies to many things in our society. But with the ever-widening supply of
video and entertainment options, it’s increasingly about locating a favorite a program, some
vital piece of information or haphazardly stumbling upon some gem not usually considered in
one’s normal line up of favorites. In this increasingly complex scenario of information and
entertainment options, guides are key. In this article we focus on program guides to help
people find and tune to the huge supply of video content in their usual (pay) TV packages and
on the Internet.
It used to be simple to find out what was ‘on’. With a quick check of the relatively few
channels listed in the daily newspaper or the local weekly TV guide, printed guides provided
the titles and details of programs of scheduled programs in a simple to read, column format,
organised by time. Additional editorial commentary, and a photo, flagged noteworthy
<sample from old newspaper>
As the number of channels began to increase, ‘grid’ format TV listings were introduced, and by
mid 1980s, they became the preferred and ultimately dominant format. Grids even formed the
basis for the first known ‘electronic’ guide. Broadcast by a New York cable system, printed
grids were attached to a drum, filmed while the drum was rotated, and then broadcast to
deliver the world’s first on-screen guide.
Grid-format TV listings are still widely published and their layout has provided the main
structural building block for most existing of today’s Electronic Program Guides (EPGs). But
with so many channels, not enough space, and no way to really convey much useful detail
about the programs, today’s printed guides have just about as much charm and readability of
an Excel spreadsheet!
<example of grid format listings>
<example of grid style epg>
The Evolving Electronic Program Guide
With the proliferation of more and more specialised channels (catering to the interests of a
growing number of niche audiences) Electronic Program Guides have become the essential tool
for navigating content in the mass market TV landscape. EPGs were created to ease and
improve the navigation of viewing options and are also considered an important marketing and
revenue generation tool by the pay-tv operators who provide their closed networks of content
to paying subscribers. In recent years, other, primarily internet-based tools have become
commonplace for locating content and more more beyond the television set. While we once
needed to find out what was ‘on’ - in a scheduled based environment, now, we’ve migrated
now to navigating what’s always ‘on’ or always available.
<text box – unless some other reference can be found>:
“A household with 300 cable or satellite channels has access to 7,000 hours of programming a
day, almost 3 million per year. That’s a lot, but it’s only a fraction of the 31 million hours of
total annual programming. Every major cable company is making investments to allow TV to
be distributed over the Internet, giving you access to each one of those 31 million hours. And
then there’s this year’s  36-fold explosion in consumer-generated video on the
internet.” article, Yahoo!: The Super Network, by Josh McHugh, Wired, 09/2005.
There is an undeniable need to navigate. But in the context of convergence, are Electronic
Program Guides, in their current forms, sustainable? How might they evolve? Will ‘TV’ or the
‘Internet’ be the driving force? And how might consumers eventually be able to search, locate
and select from the vast range of content now at their disposal? Considering the scenario of an
always-on, on-demand world, what is at the core of making all these choices accessible?
The World that High-Tech Forgot: Metadata
Most people probably haven’t thought much about the effort it takes for specialized TV listings
companies and press agencies – those organizations that serve up the TV data requirement for
major press outlets, online and electronic program guides – to obtain the program schedules,
or ‘listings’ data, from broadcasters and channels.
Back in the days before online and email, program schedules were sent out by the channels by
mail and fax. This of course necessitated endless re-typing at the recipient’s end, and
impossible challenges to keep the schedule correct with latest program schedule updates!
Often taken for granted, the data acquisition, cleaning, formatting and distribution process,
amounts to rather a lot of effort, which is rather a big deal these days, because ‘metadata’ is –
simply - at the heart of content navigation.
As the number of channels and types of TV guides increased, specialised companies began
catering to their needs. In Europe, it wasn’t until the the 1980s that the growth of satellite
television led to the emergence of specialised TV listings companies. Channels now found that
they had to provide their listings across multiple international territories, in different
languages and in different time zones. The ‘TV listings company’ acted as the necessary
middlemen between the messy formats provided by the channels, and the more standardised
and streamlined requirements of the TV guide publishers.
<Diagram of the data flow, pre and post metadata aggregators>
It’s useful to think of these TV listings in terms of how you might import names and addresses
into your contact management software: the field names have to compatible, and the field
characteristics have to match (eg, you can’t import characters into a ‘numeric’ field). The task
could have been much simpler if the files provided by the channels had, at least, consistent
tags like <title> prior to the title of the program. But no such luck. The data – still today – is
delivered in much the same format as the words you are reading right now. No tags, no
structure, just linear narrative on a page – with no possibility to easily import this data into
anything like a database. Even where database files are available from individual channels,
there remains no consistent structure across all channels to enable easy importatation into an
EPG (database) application, which has the need for a single format of incoming data format for
all channels. Hence, TV listings companies (aka ‘metadata aggregators’) typically act as the
necessary clearing house for the provision of programming information to Electronic Program
<illustrate: genre example>
(is the Simpsons a sitcom, vs, comedy, series a cartoon or an animation)
<Caption: Data for a complete and consistent EPG application has to meet both qualitative
and quantitative requirements. Additionally, different protocols are imposed by each TV guide
publisher, whether it is print or electronic, for the delivery and updating mechanisms they
require to match their varying editorial and production processes. >
EPGs provide the primary communication mechanism by which consumers regularly interact
with their pay-tv provider’s services and associated content offerings. Indeed, as choice has
increased, additional mechanisms were built into the EPG to provide a better user experience
and ease of use.
<<SOMETHING HERE ABOUT EXISTING EPGS in traditional pay-tv and new iptv platforms –
And a pictures : eg: <Examples: Search by title, by channels, by genre, etc>
The introduction personal Digital Video Recorders (DVRs), with in-built hard disks for time-
shifting programs, basically provided the mechanism to convert linear TV into Video on
Demand. This has enabled readily available viewing options to expand considerably and has
added an interesting twist to what the EPG needed to offer. Now, what was on yesterday or
last week is just as important as “what’s on” tonight or tomorrow, moving viewers into a world
where they must scan and select from a greater library of available programs that can be
recorded and later watched at anytime.
DVRs additionally have the technological ability to provide opt-in, targeted marekting. A
robust EPG environment would seem to be necessary for this to function: providing a platform
for viewers to selected what they want to opt-in to (with agents delivering relevant content);
and for advertisers to be charged on a ‘pay-per-click’ model (similar to the internet).
<<EXAMPLES – SAY – ABOUT WHAT THEY LOOK LIKE AND HOW THEY ARE MIGRATING TO
SUPPORT NEW DEMANDS>>
<<Some pictures? A few words, to segway into....>>
Beyond EPGs and Search
What will the future look like for ‘electronic program guides”, or more aptly, the future of
content navigation and search, encompassing everything a consumer might want to know,
beyond finding a movie or sports event to watch?
In contrast to and beyond traditional pay-tv entertainment services, the internet offers access
to unimaginable quantities of information and entertainment. Such a vast world of content
provokes consideration of what a truly converged guide might look like, going even further
than managing just video entertainment in the living room, regardless of the delivery
mechanism: traditional cable, satellite, TV over internet, IPTV or perhaps even one day it will
be something simpler, like moonbeams.
A new breed “EPG”, perhaps better called an “E-Service Guide”, might seamlessly integrate all
available (free and subscribed to) content offerings (web pages, audio, video, newsfeeds,
newsgroups, etc) of interest and relevance. It should provide built-in serendipity
(randomness) and also include the option to choose and access trusted mainstream linear
‘channels’. This all encompassing guide might not look all that different from the kind of
existing personalised ‘home pages’ that can be set up on the web with RSS readers like
my.yahoo (and many others).
Without having to go in search of, download and learn a multitude of softwares and widgets,
the integrated elements of such a solution would integrate many now-common Web 2.0
features such as:
- social bookmarking / tagging
- the sharing of user preferences amongst small user groups
- addition of user-defined editorial opinion (“write a review of this”)
- collaborative filtering / recommendation engines / personal agents (“if you liked that,
you might like this”)
- option to access traditional mainstream sources (ie, Time Out’s TV section)
- search functionality
Imagine a consumer-defined personal agent tracking information interests during the course
of a single day. The agent should be invisible, knowing your preferences without having to
work hard at defining them (much the same as cookies help to deliver you to the right spot
and/or know what you want when you visit a website). The agent would deliver content to a
consumer-defined ‘home page’ customized to his information needs / tastes. The agent might
be instructed thus: The default is “I’m interested in X subject(s) today’; show me, later on, Mr
Agent, some stuff that you find on this topic, prioritise it by seeing if any of my trusted friends
like it too, and then go off and search all other English language content you happen to find),
As far as search goes, this will undoubtedly be a feature of future EPGs. But as is the case
with any search query today, you have to know what you’re looking for before you can find it!
A handy ‘search term’ harvesting gadget, perhaps attached to your mobile phone, would soon
prove its value as one goes about their day mentally noting the subjects or keywords to use
for more in-depth research once in front of the computer.
Video search companies like Silicon Valley start-up, Blinkx, and others, have had to take a
different approach, filling in with very advanced technology what’s missing in the form of
consistent metadata. Their approach emphasizes speech recognition in order to deliver its
video search results, and it tries to glean ‘understanding’ from what is said and their sounds,
rather than the images themselves. In this sense, they may not go far enough, since speech is
pernickety and what might be said in a program may not have anything to do with what the
program is about at all!
Google Video has also made fledgling efforts to enable video search, but just because Google
has not yet been successful, doesn’t mean they won’t be, especially considering their recent
purchase of YouTube. Indeed, there are a slew of such companies on the market, target the
three main sub-sets of the video search market: those who spider and trawl the open internet
(eg, Google Video, Truveo); those which index shared / uploaded content (eg, You Tube, Flurl)
and those that employ other methods including those using Artificial Intelligence (AI) and / or
One promising start-up is Shadow TV, based in New York City. Its founder Joachim Kim
received his BA from Columbia University in Computer Science with graduate studies in
Artificial Intelligence. At Citibank, one of his first jobs, he was responsible for the creation of
the world’s first Automated Teller Machine (ATM). Shadow TV is both speech recognition and
AI-based, but as you would expect, there’s more in the secret sauce.
Shadow TV claims to offer a solid proposition for tradition media companies and content
owners, offering a way to monetise archived content, provide additional ad inventory and gain
additional revenue from advertisers, strengthen brands, aggregate data on consumer behavior
and extend the reach of a fragmenting viewer base.
Founded in 2001, Shadow TV is a on-going concern with with $1.5m in revenues from its
professional subscription video monitoring technology alone. With blue-chip customers
including The White House, the New York Stock Exchange and NASDAQ, it is now seeking
funding to take the business direct to the consumer market. Shadow TV works by taking
existing programming from national, local, and independent sources, they then archive and
index the content, making it keyword searchable, and embed an ad. Then, consumers (or third
parties on behalf of consumers) can search for video which can easily be shared with friends,
family, and their communities. Shadow TV’s technology is now driving the video search feature
on the CNBC.com, and the search results are claimed to be delivered at a speed twice that of
That the video search technology market is crowded with innovative brainiacs is not in doubt.
Video search will only get better. But coherent and consistent metadata is a hugely
complementary element in improving ‘search’ to the extent that the results are really useful
One of the greatest challenges of putting together an integrated E-Service Guide is the data
collection and aggregation about available ‘content’ on the internet as well as from traditional
broadcasters, as remarked earlier.
- genre harmonisation (both type and mood)
- the international harmonization of ‘content ratings’ (PG, R, etc).
- language issues.
- harvesting descriptive metadata to web content (work of the w3c.org)
After all, people have more things in their daily lives to think about than what movie or series
they want to catch up on.
Schedule-based programming once meant that consumers had to adjust to the broadcasters’
schedules, and that was that. But no more. When it comes to ‘content navigation’ let’s be
clear. It’s not just about accessing video, it’s about all the tools that help us manage all the
other things we do in life too. As the demands on a consumer’s time are endless, what kind of
guide will they ultimately find most useful?
Metadata will remain a key driver of search engine results. But it may be more likely, as an
extension of the social-networking focused Web 2.0, that consuemrs are harnessed to help
with this job.