СИСТЕМА ОЦЕНКИ УЯЗВИМОСТЕЙ CVSS 4.0 / CVSS v4.0 [RU].pdf
Дмитрий Рыжков_WiDi Android Intel
1. Использование Presentation API
и Intel® Wireless Display
в приложения для Android
Дмитрий Рыжков
Инженер по программному обеспечению
Intel Corporation
2. Содержание
•
•
•
•
•
•
Miracast* и Intel® Wireless Display
Совместимые устройства
Режимы работы и модели использования
Демонстрация
Android* Presentation API
Резюме
Software and Services Group
3. Содержание
•
•
•
•
•
•
Miracast* и Intel® Wireless Display
Совместимые устройства
Режимы работы и модели использования
Демонстрация
Android* Presentation API
Резюме
Software and Services Group
4. Что такое Miracast*?
• Miracast* стандарт беспроводной передачи
мультимедийного сигнала, принятый и
утверждённый объединением Wi-Fi Alliance
• Беспроводная альтернатива HDMI*/MHL*
• Построен на базе существующих стандартов
– H.264 (кодирование) и WiFi Direct (транспорт)
• Основан на спецификации Intel® Wi-Fi Display
• Является открытым стандартом
• Имеет поддержку в Android* 4.2
Это не DLNA или ChromeCast
Software and Services Group
5. Что такое Intel® Wireless Display (WiDi)
• Сертифицирован WFA Miracast*
– Совместим с сертифицированными Miracast* устройствами.
• Поддерживает HD видео (1080p/60 + 5.1 аудио)
• Защита передаваемого контента (HDCP2.1)
• Два режима работы
– Редим клонирования (с помощью Android*
media framework)
– Расширенный режим (отдельные
представления для каждого экрана)
• Доступен на большинстве устройств с
Intel® Atom™ SOCs
Intel® WiDi совместим WFA Miracast*
Software and Services Group
6. Как работает Miracast*
Audio/Video Payload
Content
Composition of
Audio/Video
frames, graphic
s and text
Encoding
Decoding
Rendering
Wi-Fi Direct*
Device capability
negotiation and
session maintenance
Control
Device capability
negotiation and
session maintenance
Software and Services Group
7. Содержание
•
•
•
•
•
•
Miracast* и Intel® Wireless Display
Совместимые устройства
Режимы работы и модели использования
Демонстрация
Android* Presentation API
Резюме
Software and Services Group
8. Сертифицированные устройства
Адаптеры
• Actiontec* ScreenBeam Pro
• Netgear* (ex: PTV3000)
• Viewsonic (WPG-370)
• Acer*
• Best Buy* Rocketfish*
• Lenovo*
•…
Проекторы
•
•
•
•
•
Intel® WiDi Certified
Телевизоры
• Samsung*
Dell*
• Toshiba*
Seiko Epson *
• LG*
• TCL*
Ricoh*
• Sharp*
LG*
• Philips*
…
•…
Полный список устройств доступен на
http://www.wi-fi.org/wi-fi-certified-miracast
10. Совместимые устройства на Android* 4.2.x с
поддержкой Presentation API
Устройство
Сертифицировано
Miracast*
HDMI*, MHL*
или SlimPort*
Samsung* Galaxy S4
Да
Да
Samsung Galaxy Tab3 10.1
Нет
Да
Sony* Xperia Z
Да
Да
LG* Optimus G
Да
Да
Nexus* 4
Да
Да
HTC* One
Да
Да
Lenovo* K900
Да
Нет
Asus* MeMO Pad FHD10
Да
Да
11. Подключение Wireless Display в Android*
Sony* Xperia Z
Stock/Intel®
Samsung* Galaxy S4
Software and Services Group
12. Содержание
•
•
•
•
•
•
Miracast* и Intel® Wireless Display
Совместимые устройства
Режимы работы и модели использования
Демонстрация
Android* Presentation API
Резюме
Software and Services Group
13. Режим клонирования (по умолчанию)
• После подключения пользователь видит экран
мобильного устроства на телевизоре
• Используется разрешение мобильного устройства
• От разработчика не требуется никаких изменений
Software and Services Group
14. Режим презентации
• Внешний экран отображает контент
• Экран мобильного устройства
используется для управления
• Достигается использованием Android*
Presentation API
Software and Services Group
15. Расширенный режим
(Доступен только для Intel® WiDi)
• Активируется атвоматически, если приложение использует
Android* Media Player API (VideoView)
• Контен отображается в оригинальном разрешении
• Опции управления доступны на мобильном устройстве
• С целью экономии заряда батареи контент не отображается
на мобильном устройстве
Software and Services Group
16. Модели использования
• Видео
– Отображение видео на внешнем экране, мобильное
устройство используется как пульт ДУ
• Игры
– Использование мобильного устройства как игрового
контроллера
• Продуктивность
– Отображение презентации на внешнем экране, мобильное
устройство используется как кликер
• IVI
– Отображение навигационного приложения на экране IVI
Software and Services Group
23. Содержание
•
•
•
•
•
•
Miracast* и Intel® Wireless Display
Совместимые устройства
Режимы работы и модели использования
Демонстрация
Android* Presentation API
Резюме
Software and Services Group
24. Android* Presentation API
Класс Presentation в Android* 4.2 (API Level
17), позволяет:
• Использовать второй экран не заботясь о способе подключения
• Использовать MHL*, HDMI*, SlimPort* & Miracast* совместимые
устройства
• Контролировать контент, отображаемый на внешнем экране
Software and Services Group
25. Объект Presentation
• Представление должно наследовать Presentation
public class DemoPresentation extends Presentation {
• В момент создания ассоциируется с объектом Display
• Presentation наследуется от Dialog, так же как для Dialog жизненный цикл
зависит от Activity
Dialog
Presentation
Activity должно контролировать представление
(Presentation) в моменты изменения текущего
статуса (onPause, onStop, onResume)
26. Использование Presentation API
Сначала необходимо получить объект Display, на котором будет отображаться
представление:
1. MediaRouter API (в API 16): автоматический выбор наиболее
подходящего
// Get the media router service.
MediaRouter mMediaRouter =
(MediaRouter)getSystemService(Context.MEDIA_ROUTER_SERVICE);
// Use the MediaRouter that supports live video
MediaRouter.RouteInfo mRouteInfo =
mMediaRouter.getSelectedRoute(MediaRouter.ROUTE_TYPE_LIVE_VIDEO);
Display presentationDisplay = mRouteInfo.getPresentationDisplay();
2.
Display Manager API (в API 17): Перечисление доступных
// Get the display manger service.
DisplayManager mDisplayManager =
(DisplayManger)getSystemService(Context.DISPLAY_SERVICE);
// enumerate the displays
Display[] displays =
mDisplayManager.getDisplays(DisplayManger.DISPLAY_CATEGORY_PRESENTATION);
27. Presentation API
How to get a Presentation displayed:
MediaRouter.
getSelectedRoute
(ROUTE_TYPE_LIVE_VIDEO)
MediaRouter.
routeInfo
new Presentation
(activityContext, display)
.show()
getPresentationDisplay()
Используя MediaRouter.addCallback, необходимо отслеживать и обрабатывать:
• onRouteUnselected
• onRouteSelected
• onRoutePresentationDisplayChanged
Внутри Activity, создавшей представление:
• onResume
• onPause
28. Тестирование приложений
Варианты:
• На устройствах с наличием возможности подключения (Miracast, HDMI, и т.д.)
• В режиме эмуляции – новая возможность в Android* 4.2 (Инструменты
разработчика). Позволяет выбрать разрешение и DPI, после активации
отображается поверх экрана.
Software and Services Group
29. Добавление MediaRouteButton
res/menu/default.xml:
<item
android:title="Media Route Settings"
android:actionProviderClass="android.app.MediaRouteActionProvider"
android:showAsAction="always" />
//Sets Media Route Button to second screen mode
mediaRouteActionProvider.setRouteTypes
(MediaRouter.ROUTE_TYPE_LIVE_VIDEO);
//Launches the Wireless display setting intent
startActivity(new Intent
("android.settings.WIFI_DISPLAY_SETTINGS"));
30. Реализация интерфейса Presentation
Используется тот же стиль, что и для Dialog.
Разрешение и DPI:
TV setting
Разрешение
Density
Screen Density
Display Resolution
Screen size
identifier
720p
1280 x 720 px
tvdpi
213 dp
960 x 540 dp
large
1080p
1920 x 1080 px
xhdpi
320 dp
960 x 540 dp
large
source: https://developers.google.com/tv/android/docs/gtv_android_patterns
Density телевизора и смартфона сопоставимы, больший
размер экрана компенсируется расстоянием.
Тема по умолчанию Holo Dark: светлый текст на темном
фоне легче читается на телевизоре.
Всегда ландшафтный режим ориентации
31. Wireless Display APIs в Android* 4.2
• Диалог настроек беспроводного дисплея можно
вызвать используя
android.settings.WIFI_DISPLAY_SETTINGS, так же
возможны варианты
– Samsung*:
com.samsung.wfd.LAUNCH_WFD_PICKER_DLG
– HTC*:
com.htc.wifidisplay.CONFIGURE_MODE_NORMAL
Software and Services Group
32. Содержание
•
•
•
•
•
•
Miracast* и Intel® Wireless Display
Совместимые устройства
Режимы работы и модели использования
Демонстрация
Android* Presentation API
Резюме
Software and Services Group
33. Резюме
Miracast* и Intel® WiDi :
• Позволяют отображать контент на внешнем устройстве
используя беспроводную передачу
• Intel® WiDi является полностью совместимым с Miracast* и
предоставляет дополнительные функции, такие как Dolby
5.1, защита контента (HDCP 2.1), низкая задержка
Android* Presentation API
• Добавлены в Android* API Level 17, позволяют использовать
внешние устройства для отображения контента, вне
зависимости от способа подключения
34. Дополнительная информация
Официальная документация:
• http://developer.android.com/reference/android/app/MediaRouteActionProvider.
html
• http://developer.android.com/reference/android/hardware/display/DisplayMana
ger.html
• http://developer.android.com/reference/android/app/Presentation.html
37. Legal Disclaimer
Software Source Code Disclaimer: Any software source code reprinted in this document is furnished under a software
license and may only be used or copied in accordance with the terms of that license.
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated
documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons
to whom the Software is furnished to do so, subject to the following conditions:
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT
NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Software and Services Group
38. Risk Factors
The above statements and any others in this document that refer to plans and expectations for the third quarter, the year and the future are forward-looking
statements that involve a number of risks and uncertainties. Words such as “anticipates,” “expects,” “intends,” “plans,” “believes,” “seeks,” “estimates,” “may,”
“will,” “should” and their variations identify forward-looking statements. Statements that refer to or are based on projections, uncertain events or assumptions
also identify forward-looking statements. Many factors could affect Intel’s actual results, and variances from Intel’s current expectations regarding such factors
could cause actual results to differ materially from those expressed in these forward-looking statements. Intel presently considers the following to be the
important factors that could cause actual results to differ materially from the company’s expectations. Demand could be different from Intel's expectations due to
factors including changes in business and economic conditions; customer acceptance of Intel’s and competitors’ products; supply constraints and other disruptions
affecting customers; changes in customer order patterns including order cancellations; and changes in the level of inventory at customers. Uncertainty in global
economic and financial conditions poses a risk that consumers and businesses may defer purchases in response to negative financial events, which could
negatively affect product demand and other related matters. Intel operates in intensely competitive industries that are characterized by a high percentage of
costs that are fixed or difficult to reduce in the short term and product demand that is highly variable and difficult to forecast. Revenue and the gross margin
percentage are affected by the timing of Intel product introductions and the demand for and market acceptance of Intel's products; actions taken by Intel's
competitors, including product offerings and introductions, marketing programs and pricing pressures and Intel’s response to such actions; and Intel’s ability to
respond quickly to technological developments and to incorporate new features into its products. The gross margin percentage could vary significantly from
expectations based on capacity utilization; variations in inventory valuation, including variations related to the timing of qualifying products for sale; changes in
revenue levels; segment product mix; the timing and execution of the manufacturing ramp and associated costs; start-up costs; excess or obsolete inventory;
changes in unit costs; defects or disruptions in the supply of materials or resources; product manufacturing quality/yields; and impairments of long-lived
assets, including manufacturing, assembly/test and intangible assets. Intel's results could be affected by adverse economic, social, political and
physical/infrastructure conditions in countries where Intel, its customers or its suppliers operate, including military conflict and other security risks, natural
disasters, infrastructure disruptions, health concerns and fluctuations in currency exchange rates. Expenses, particularly certain marketing and compensation
expenses, as well as restructuring and asset impairment charges, vary depending on the level of demand for Intel's products and the level of revenue and profits.
Intel’s results could be affected by the timing of closing of acquisitions and divestitures. Intel's results could be affected by adverse effects associated with product
defects and errata (deviations from published specifications), and by litigation or regulatory matters involving intellectual
property, stockholder, consumer, antitrust, disclosure and other issues, such as the litigation and regulatory matters described in Intel's SEC reports. An
unfavorable ruling could include monetary damages or an injunction prohibiting Intel from manufacturing or selling one or more products, precluding particular
business practices, impacting Intel’s ability to design its products, or requiring other remedies such as compulsory licensing of intellectual property. A detailed
discussion of these and other factors that could affect Intel’s results is included in Intel’s SEC filings, including the company’s most recent reports on Form 10Q, Form 10-K and earnings release.
Rev. 7/17/13
40. Miracast* Concept
Content
Composition of
Audio/Video
frames,
graphics and
text
Audio/Video Payload
Encoding
Decoding
Rendering
Wi-Fi Direct*
Source device
Device capability
negotiation and
session maintenance
Control
Device capability
negotiation and
session maintenance
Sink
device
Software and Services Group
41. Miracast* Architecture (Source side)
Display Application & Manager
Miracast
Control
UIBC
(opt)
Video
Encode
Audio Encode
PES
PES
HDCP 2.0/2.1 (opt)
MPEG2-TS / AV MUX
RTSP
RTP
TCP (socket)
UDP (socket)
Wi-Fi Direct* (P2P) / TDLS & Wi-Fi Protected Setup
WLAN Device
Software and Services Group
42. Как работает Miracast*
Источник
Видео
Аудио
Приемник
Видео
Аудио
Video Encode
Audio Encode
Video Decode
Audio Decode
Packetize
Packetize
De-Packetize
De-Packetize
Link Content Protection Encrypt
(Optional)
Link Content Protection Decrypt
(Optional)
AV Mux
AV DeMux
Transport
Transport
LLC
LLC
WI-Fi MAC (Direct Link)
WI-Fi MAC (Direct Link)
Wi-Fi PHY
Wi-Fi PHY
Software and Services Group
43. Miracast vs Intel® WiDi Receiver Spec
Features
Miracast
Intel® WiDi (4th Gen)
Resolution
480p/30fps
720p Optional
1080p/30fps
1080p/60fps Optional
HDCP
Optional
2.1
Audio
Stereo Sound
Stereo Sound & 5.1 Surround
Sound
S3D
Optional
Yes
Speed Mode
Optional
Low Latency
Fast Cursor
No
Yes
Software and Services Group
Editor's Notes
Our presentation will include providing an overview of Miracast and Intel Wireless display technologies, a quick look at available phone and tablets and display devices in the market today supporting second screen usage, some second-screen use cases and idea, followed by demo of an in-house developed 2-screen media player application. Then in the final part of the presentation we will explain how to develop with the Android secondary-screen APIs complemented with a live coding.
- Allows content streaming from one Miracast certified device to another- Wireless replacement of a wireless HDMI or HML cable between a Source device (i.e. smartphone or tablets) to a sink or Display device (i.e. TV, projector, etc..).- Built upon exiting standards such as H.264 (MPEG-4 AVC) for video compression, and WiFi Direct …- Based on Intel’s Wireless Display (aka WiDi) specification.- Supports HD (1080p) at 30FPS (or 720p at 60FPS for smoother playback)
So, what is Intel WiDi:-It Miracast certified, so any Miracast certified adapter should work. With Intel Wdi it is preferred to use a Intel Certified adapter - we will lokk at these later…Supports 1080p At 30FPS or 720p/60 fps for smoother motion video Display protected content wirelessly with HDCP (High bandwidth Digital content protection) 2.1Can be used with Android 4.2 Presentation API for dual screen usage, also support 2 addition modes- And available on most current ATOM SOCs.
Miracast Concept at a high-level [1] Before any content can be sent a link or session must be established in prep for content streaming (this is done by the phone/tablet scanning for the available displays (& capabilities)**, and asks the user to select one) typically via through the display settings[2] When the connection is, the source performs composition of the Audio and Video and encodes the content it into a mpeg2 bit-stream and the transmits it over WiFi Direct to the Display device. Once the Display device receives the content it decodes the renders it to the Display.Session Management takes care of: connection setup, capability negotiation, DRM Support (mandatory for Intel WiDi), Session establishment and streaming, **payload control/handshaking** and finally display session teardown.Supported formats for video: H.264 (aka Advanced Video Coding) and Audio: Linear PCM (LPCM) 16 bits, 48KHz) 2 channels. Optional audio codes are based on LPCM 16 bits, 44/1KHz sampling, 2 channels, Advanced Audio Coding (AAC) and Dolby Advanced Codec 3 (AC3)The next slide will go a little deep in to the Miracast architecture …
For full list of Wi-Fi Miracast certified devices (that even includes TV built in refrigerators) go to www.wi-fi.orgFor full list of Miracast™ enabled source and display devices checkout the WiFi Alliance web site: http://www.wi-fi.org/wi-fi-certified-miracast%E2%84%A2http://certifications.wi-fi.org/search_products.php?search=1&advanced=1&selected_certifications%5B%5D=69
Low latency gaming- 50ms to 60ms (depends on conditions) vs. competition is 100 Easy pairing: - Instead of typing a 10 digit code, you can use QR codesfull 1080p- always supports content up to 1080p, other Miracast devices might not be able to support especially with the low latency.link health indicator- helps you know your signal strength so you know if you are too far away or if interferencebest channel selection- Picks the fastest of the ABGN (supports both 2.4G and 5G Hz)Other differentiation1) Extended Video mode2) Multi-Tasking -Only with selected OEMS - we add support for multiple video stream playbacks simultaneously, remotely playing video while locally doing anything e.g. play a game, play a video, phone call etc.
Info from www.gsmarena.comAny more Intel phones??This slide shows some of the devices that run or upgradable to Android 4.2.2 that are either Miracast™ compliant or support HDMI either via MHL or micro-HDMI that are currently on the Market or will be very soon. For full list of devices you can search on gsmarena.com. I lot of my testing has been with the Lenovo K900 and Asus memo pad with netgear PTV3000 and Actiontech sunbeam – and I can say it works very well.
In Android 4.2, there is no a wireless display app in the IA phone’s menu.The connection item is under SettingDisplayWireless Display.You should also search first, the click to connect without inputting 8 digital number
Clone Mode duplicates the phone display on the remote display. The resolution of the frames sent to the adapter matches the resolution of the local display.In this mode, both local and remote displays are turned on and show the same content. The frames sent to the adapter are captured from the graphics engine, that is from LCD’ frame buffer. You can see in the right photo, In Clone mode,the phone and the TV show the same picture.
Extended Video Mode is activated when the user plays video through Android’s Media Player application.In this mode, the original resolution of the video content is sent to the adapter. While in this mode, the local display remains turned off and the video content appears only on the remote display alone. The user is allowed to touch the local display to bring up the slider controls and control the playback. The frames being sent to the adapter are captured from the fullscreen video decoder.Shown as the right photo, Video is playing on TV while there is only title bar on the phone’s screen. Now widi can support 1080p HD video at 30fps.TODO: check…
In fps: can use orientation of the device to start shooting.Screenshot taken from Open-Source (GPL) game Nexuiz.
The Android Second-screen API (Presentation)was introduced in Android 4.2, the Presentation API allows you to …
is associated with the target Display at creation time and configures its context and resource configuration according to the display's metrics.
Add getSystemService line ?Display Manager: class that holds all displays connected. Requires either developer or end user to choose a Display.Media Router: The MediaRouter class holds all possible routes for audio and video to go (e.g. speakers and displays) but differs from Display Manager as it will automatically select the best Route (or Display in our case)
Your route can change, your presentation display can be disconnected, your activity can be paused, etc. Once you created your presentation, you need to care about all this.A presentation is automatically canceled when its display is removed. But the activity must take care of pausing its content when the activity itself is paused, and test if the display still exists (so the presentation) while resuming, to resume or restart it.
If you don’t have a WiDi adapter, Android 4.2 includes an Developer options that allows you to simulate the second display, you can choose res and dpi and it will then create an overlay of that display.If developer options don’t show up, tap on Build number within About phone around 5 times to bring it up.
+ setRouteTypes(MediaRouter.ROUTE_TYPE_LIVE_VIDEO) on it.Using setRouteTypes with ROUTE_TYPE_LIVE_VIDEO flag (API level 17) will allow mirrored version of the built in screen or a customized version (using presentation class) to be send to second screen.ActionBarActivity Base class for activities that use the support library action bar features. MediaRouteActionProvider The media route action provider displays a media route button in the application's ActionBar to allow the user to select routes and to control the currently selected route. MediaRouteButton The media route button allows the user to select routes and to control the currently selected
TODO: add link to sample
Miracast Concept at a high-level [1] Before any content can be sent a link or session must be established in prep for content streaming (this is done by the phone/tablet scanning for the available displays (& capabilities)**, and asks the user to select one) typically via through the display settings[2] When the connection is, the source performs composition of the Audio and Video and encodes the content it into a mpeg2 bit-stream and the transmits it over WiFi Direct to the Display device. Once the Display device receives the content it decodes the renders it to the Display.Session Management takes care of: connection setup, capability negotiation, DRM Support (mandatory for Intel WiDi), Session establishment and streaming, **payload control/handshaking** and finally display session teardown.Supported formats for video: H.264 (aka Advanced Video Coding) and Audio: Linear PCM (LPCM) 16 bits, 48KHz) 2 channels. Optional audio codes are based on LPCM 16 bits, 44/1KHz sampling, 2 channels, Advanced Audio Coding (AAC) and Dolby Advanced Codec 3 (AC3)The next slide will go a little deep in to the Miracast architecture …
-This slide shows the Miracast architecture (source side) in more detail:Session Management takes care of: connection setup, capability negotiation, DRM Support (mandatory for Intel WiDi), Session establishment and streaming,UIBC, payload control/handshaking and finally display session teardown.once connection is made and the capability negotiation is completed, the source and display devices setup a Miracast session, then the content can be streamed.Before streaming, the content (from display buffer) gets encoded and packetized, and if it DRM protected the content is encrypted then transcoded to an MPEG2-Bit Stream, AV mux’ed, then streamed using the WiFi network stack to the display device.Talk about Sink side /…---STOP here---RTSP: Controls the media stream at application level HDCP: (opt in Miracast but mandatory in Intel WiDi) allows DRM protected content to be played.WiFi Direct: manages the layer 2 connection (data link layer) between devicesWLAN Device – Layer 1 or Physical layer for direct device to device comms (radio link)PES – Packetizes Elementary stream
Video: h264WiFi Direct communication, AV encoding on the source, AV decoding on the sink.11n WPA2 WiFi with Wi-Fi Protected SetupRTP, UDP
Use Android 4.2 Presentation API for dual screen usage. Wireless Display/ Miracast™ CompliantWireless streaming and playback from Intel devices to enabled devices.1080p/30 or 720p/60 fps for smoother motion video Display protected content wirelessly with HDCP 2.1WirelessReplacement for MHL/HDMIMiracast™ is standard for wireless peer-to-peer screencasting, created by the WiFi alliance.WiDi is an Intel based technology for wireless streaming of audio and video between compatible devices.As of version 3.5+, WiDi is now Miracast™ certified, meaning WiDi now works with other Miracast™ certified devices.