All These Sophisticated Attacks, Can We Really Detect Them - PDF
HTML5 and Video Streaming Vol.1
1. This slide by mzyy94 and Icons by Font Awesome are licensed under a Creative Commons Attribution 4.0 International License.
HTML5 and Video Streaming
1
Vol.1
2. This slide by mzyy94 and Icons by Font Awesome are licensed under a Creative Commons Attribution 4.0 International License.2
×
3. This slide by mzyy94 is licensed under a Creative Commons Attribution 4.0 International License.3
<video src="BigBuckBunny_320x180.mp4"></video>
4. This slide by mzyy94 and Icons by Font Awesome are licensed under a Creative Commons Attribution 4.0 International License. Big Buck Bunny is licensed under a Creative Commons Attribution 3.0 Unported License.
5. This slide by mzyy94 is licensed under a Creative Commons Attribution 4.0 International License.5
😆
6. This slide by mzyy94 and Icons by Font Awesome are licensed under a Creative Commons Attribution 4.0 International License.6
MP4
7. This slide by mzyy94 and Icons by Font Awesome are licensed under a Creative Commons Attribution 4.0 International License.7
MP4
8. This slide by mzyy94 and Icons by Font Awesome are licensed under a Creative Commons Attribution 4.0 International License.8
HTML5
Media Source Extension
Encrypted Media Extension
Clear Key
Widevine
ISO-Base Media File Format
Fragmented MP4
HTMLMediaElement
MPEG-4 Part 10 Advanced Video Coding
MPEG-DASHContent Decryption Module
MediaKeySession JSON Web Key
WebCrypto
Autoplay Policy
Media Engagement IndexTimed Text Markup Language
Secure Context
PlayReady
Common Encryption
HTTP Live Streaming
Adaptive Bitrate
Digital Rights Management
ECMAScript 2015
Persistence License
KeySystem
SourceBuffer
HEVC
FairPlay Streaming 2.0
emsg
ID3
WebVTT
MPEG-TS
TextTrack
9. This slide by mzyy94 and Icons by Font Awesome are licensed under a Creative Commons Attribution 4.0 International License.
• Adaptive Bitrate
• Content Decryption Module
• MediaKeySession
• Encrypted Media Extension
• Common Encryption
• Source Buffer
• Media Source Extension
• MPEG-DASH
• HTTP Live Streaming
9
MP4
10. This slide by mzyy94 and Icons by Font Awesome are licensed under a Creative Commons Attribution 4.0 International License.10
HTML5
Media Source Extension
Encrypted Media Extension
Clear Key
Widevine
ISO-Base Media File Format
Fragmented MP4
HTMLMediaElement
MPEG-4 Part 10 Advanced Video Coding
MPEG-DASHContent Decryption Module
MediaKeySession JSON Web Key
WebCrypto
Autoplay Policy
Media Engagement IndexTimed Text Markup Language
PlayReady
Common Encryption
HTTP Live Streaming
Adaptive Bitrate
Digital Rights Management
ECMAScript 2015
Persistence License
KeySystem
SourceBuffer
HEVC
FairPlay Streaming 2.0
emsg
ID3
WebVTT
MPEG-TS
Secure Context TextTrack
11. This slide by mzyy94 is licensed under a Creative Commons Attribution 4.0 International License.
History of Movie File
11
And Browser Plug-in
12. This slide by mzyy94 is licensed under a Creative Commons Attribution 4.0 International License.
FLV
12
1990 2000 20202010
AVI
WMV
MJPEG
MP4
MOV
SWF
???
3GP
13. This slide by mzyy94 is licensed under a Creative Commons Attribution 4.0 International License.This slide by mzyy94 and HTML5 icon by Font Awesome are licensed under a Creative Commons Attribution 4.0 International License. Other icons, don’t meet the threshold of originality, are in the public domain.
1990 2000 20202010
AVI
WMV
MJPEG
MP4
SWF
???
Windows
Media Player
FLV
3GP
Silverlight ™
Microsoft®
MOV
2014
14. This slide by mzyy94 is licensed under a Creative Commons Attribution 4.0 International License.
An MP4 Video
14
15. This slide by mzyy94 and Icons by Font Awesome are licensed under a Creative Commons Attribution 4.0 International License.15
MP4
16. This slide by mzyy94 and Icons by Font Awesome are licensed under a Creative Commons Attribution 4.0 International License.
MP4 File Format
• A/V Container Format
• De facto standard on the Web
• Based on ISO BMFF
• ISO/IEC 14496-14 Standard
• Formed as a series of Boxes
MP4
16
17. This slide by mzyy94 and Icons by Font Awesome are licensed under a Creative Commons Attribution 4.0 International License.
A/V Container Format
17
MP4
H.264 + AACH.264 + AAC
H.264 + MP3
H.265 + AAC
AAC
18. This slide by mzyy94 and Icons by Font Awesome are licensed under a Creative Commons Attribution 4.0 International License.
De facto standard on the Web
18
All Modern Browsers may Play MP4
19. This slide by mzyy94 and Icons by Font Awesome are licensed under a Creative Commons Attribution 4.0 International License.
Based on ISO BMFF
19
MOV MP4 3GP
ISO Base Media File Format
Common
+Extension
21. This slide by mzyy94 is licensed under a Creative Commons Attribution 4.0 International License.
ISO/IEC 14496-12:2015(E)
4 Object-structured File Organization
4.1 File Structure
Files are formed as a series of objects, called boxes in this
specification. All data is contained in boxes; there is no other data
within the file. This includes any initial signature required by the
specific file format.
– c068960_ISO_IEC_14496-12_2015.pdf p.6
21
22. This slide by mzyy94 is licensed under a Creative Commons Attribution 4.0 International License.
“Type a quote here.”
22
ISO/IEC 14496-12:2015(E)
Figure 1 — Simple interchange file
5.1.3 Content Creation
During content creation, a number of areas of the format can be exercised to useful effect, particularly:
the ability to store each elementary stream separately (not interleaved), possibly in separate
– c068960_ISO_IEC_14496-12_2015.pdf p.9
25. This slide by mzyy94 and Icons by Font Awesome are licensed under a Creative Commons Attribution 4.0 International License.
ftyp moov
mvhd
trak
tkhd
mdia mdhd minf
trak
tkhd
mdia mdhd minf
mdat
Resolution: 1920x1080
Sample Rate: 48000Hz
Channel: 2
This slide and block shape by mzyy94 are licensed under a Creative Commons Attribution 4.0 International License. Big Buck Bunny is licensed under a Creative Commons Attribution 3.0 Unported License.
Metadata A/V content
25
Duration: 9min57sec
26. This slide by mzyy94 is licensed under a Creative Commons Attribution 4.0 International License.
Streaming Playback
26
27. This slide by mzyy94 is licensed under a Creative Commons Attribution 4.0 International License.27
<video src="BigBuckBunny_320x180.mp4"></video>
Single MP4 File
• Fixed Duration
• Static Bitrate
• More Buffering
28. This slide by mzyy94 is licensed under a Creative Commons Attribution 4.0 International License.28
• Flexible Duration
• Adaptive Bitrate
• Less Buffering
<video src="BigBuckBunny_320x180.mp4"></video>
Single MP4 File
• Fixed Duration
• Static Bitrate
• More Buffering
29. This slide by mzyy94 is licensed under a Creative Commons Attribution 4.0 International License.29
mpeg-DASH
HTTP Live Streaming
30. This slide by mzyy94 is licensed under a Creative Commons Attribution 4.0 International License.
HTTP Live Streaming
• a.k.a. HLS
• Developed by Apple
• Specified in IETF’s RFC 8216
• Safari and Edge support natively
30
───────┼──────────────────────────────────────────────────
1 │ #EXTM3U
2 │ #EXT-X-VERSION:4
3 │ #EXT-X-INDEPENDENT-SEGMENTS
4 │
5 │ #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME=
6 │ #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME=
7 │ #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME=
8 │ #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME=
9 │ #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME=
10 │
11 │
12 │
13 │ #EXT-X-STREAM-INF:BANDWIDTH=1808281,AVERAGE-BANDW
14 │ https: //p-events-delivery.akamaized.net/3004qzusa
15 │ #EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=851340,AVERAG
16 │
17 │ #EXT-X-STREAM-INF:BANDWIDTH=7673072,AVERAGE-BANDW
18 │ https: //p-events-delivery.akamaized.net/3004qzusa
19 │ #EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=3783736,AVERA
20 │
21 │ #EXT-X-STREAM-INF:BANDWIDTH=5781204,AVERAGE-BANDW
22 │ https: //p-events-delivery.akamaized.net/3004qzusa
23 │ #EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=2837802,AVERA
24 │
25 │ #EXT-X-STREAM-INF:BANDWIDTH=4362301,AVERAGE-BANDW
26 │ https: //p-events-delivery.akamaized.net/3004qzusa
27 │ #EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=2128350,AVERA
28 │
29 │ #EXT-X-STREAM-INF:BANDWIDTH=3416367,AVERAGE-BANDW
30 │ https: //p-events-delivery.akamaized.net/3004qzusa
31. This slide by mzyy94 is licensed under a Creative Commons Attribution 4.0 International License.
• Dynamic Adaptive Streaming over HTTP
• Developed under MPEG
• Specified in ISO/IEC 23009-1:2014
• Edge supports playback natively
31
mpeg-DASH
───────┼──────────────────────────────────────────────────
1 │ <?xml version="1.0"?>
2 │ <!-- MPD file Generated with GPAC version 0.5.1-D
3 │ <MPD xmlns="urn:mpeg:dash:schema:mpd:2011" minBuf
4 │ <ProgramInformation moreInformationURL="http: //g
5 │ <Title>Spring_3000k_dash.mpd generated by GPAC
6 │ </ProgramInformation>
7 │
8 │ <Period id="" duration="PT0H2M45.76S">
9 │ <AdaptationSet segmentAlignment="true" maxWidth
10 │ <Representation id="1" mimeType="video/mp4" co
11 │ <BaseURL>Spring_3000k_track1_dashinit.mp4 </Ba
12 │ <SegmentBase indexRangeExact="true" indexRang
13 │ </Representation>
14 │ <Representation id="2" mimeType="video/mp4" co
15 │ <BaseURL>Spring_5000k_track1_dashinit.mp4 </Ba
16 │ <SegmentBase indexRangeExact="true" indexRang
17 │ </Representation>
18 │ <Representation id="3" mimeType="video/mp4" co
19 │ <BaseURL>Spring_7500k_track1_dashinit.mp4 </Ba
20 │ <SegmentBase indexRangeExact="true" indexRang
21 │ </Representation>
22 │ <Representation id="4" mimeType="video/mp4" co
23 │ <BaseURL>Spring_10000k_track1_dashinit.mp4 </B
24 │ <SegmentBase indexRangeExact="true" indexRang
25 │ </Representation>
26 │ <Representation id="5" mimeType="video/mp4" co
27 │ <BaseURL>Spring_15000k_track1_dashinit.mp4 </B
28 │ <SegmentBase indexRangeExact="true" indexRang
29 │ </Representation>
30 │ <Representation id="6" mimeType="video/mp4" co
32. This slide by mzyy94 and Icons by Font Awesome are licensed under a Creative Commons Attribution 4.0 International License.32
mpeg-DASH HTTP Live Streaming
MPD M3U8
Manifest
• MP4
• WEBM
• MPEG-TS
• MP4
• MPEG-TS
33. This slide by mzyy94 and Icons by Font Awesome are licensed under a Creative Commons Attribution 4.0 International License. Big Buck Bunny is licensed under a Creative Commons Attribution 3.0 Unported License.
<video src="blob:http://example.com/ac8bef67-c5c4-4a1f-9b3f-7394dbe0ed75"></video>
init.mp4
seg-01.m4s
seg-02.m4s
seg-03.m4s
seg-04.m4s
seg-05.m4s
・
・
・
Manifest
<video src=“blob:http://example.com/ac8bef67-c5c4-4a1f-9b3f-7394dbe0ed75”></video>
34. This slide by mzyy94 and Icons by Font Awesome are licensed under a Creative Commons Attribution 4.0 International License.
Fragmented MP4
34
init.mp4
seg-01.m4s
seg-02.m4s
seg-03.m4s
seg-04.m4s
seg-05.m4s
・
・
・
35. This slide by mzyy94 and Icons by Font Awesome are licensed under a Creative Commons Attribution 4.0 International License. Big Buck Bunny is licensed under a Creative Commons Attribution 3.0 Unported License.
Normal MP4
ftyp moov
mvhd
trak
tkhd
mdia mdhd minf
trak
tkhd
mdia mdhd minf
mdat
Metadata A/V content
ftyp moov
mvhd
trak
tkhd
mdia mdhd minf
trak
tkhd
mdia mdhd minf
moof mdat
moof mdat
moof mdat
moof mdat
moof mdat
moof mdat
Fragmented MP4
36. This slide by mzyy94 and Icons by Font Awesome are licensed under a Creative Commons Attribution 4.0 International License. Big Buck Bunny is licensed under a Creative Commons Attribution 3.0 Unported License.
ftyp moov
mvhd
trak
tkhd
mdia mdhd minf
trak
tkhd
mdia mdhd minf
moof mdat
moof mdat
moof mdat
moof mdat
moof mdat
moof mdat
Fragmented MP4
https://w3c.github.io/media-source/isobmff-byte-stream-format.html
BigBuckBunny_320x180.mp4
37. This slide by mzyy94 and Icons by Font Awesome are licensed under a Creative Commons Attribution 4.0 International License. Big Buck Bunny is licensed under a Creative Commons Attribution 3.0 Unported License.
ftyp moov
mvhd
trak
tkhd
mdia mdhd minf
trak
tkhd
mdia mdhd minf
moof mdat
moof mdat
moof mdat
moof mdat
moof mdat
moof mdat
Fragmented MP4
init.mp4
seg-01.m4s
seg-02.m4s
seg-03.m4s
seg-04.m4s
seg-05.m4s
seg-06.m4s
38. This slide by mzyy94 and Icons by Font Awesome are licensed under a Creative Commons Attribution 4.0 International License.38
init.mp4
seg-01.m4s
seg-02.m4s
seg-03.m4s
seg-04.m4s
seg-05.m4s
seg-06.m4s
How to Play?
Media Source Extensions
39. This slide by mzyy94 is licensed under a Creative Commons Attribution 4.0 International License.
Media Source Extensions
39
40. This slide by mzyy94 and Icons by Font Awesome are licensed under a Creative Commons Attribution 4.0 International License.40
MP4
MP4
MP4
41. This slide by mzyy94 and Icons by Font Awesome are licensed under a Creative Commons Attribution 4.0 International License.
Media Source Extensions
41
<video src=“blob:http://example.com/ac8bef67-c5c4-4a1f-9b3f-7394dbe0ed75”></video>
new MediaSource()mediaSource
URL.createObjectURL(mediaSource)Binding
sourceBuffer
mediaSource.addSourceBuffer(mimeType)Create Source
sourceBuffer.appendBuffer(buf)Appending
42. This slide by mzyy94 and Icons by Font Awesome are licensed under a Creative Commons Attribution 4.0 International License.
• Adaptive Bitrate
• Content Decryption Module
• MediaKeySession
• Encrypted Media Extension
• Common Encryption
• Source Buffer
• Media Source Extension
• MPEG-DASH
• HTTP Live Streaming
42
MP4
Streaming Protocol
Media Source Extensions
DRM
43. This slide by mzyy94 is licensed under a Creative Commons Attribution 4.0 International License.43
• Flexible Duration
• Adaptive Bitrate
• Less Buffering
• Fixed Duration
• Static Bitrate
• More Buffering
<video src="BigBuckBunny_320x180.mp4"></video>
Single MP4 File
44. This slide by mzyy94 and Icons by Font Awesome are licensed under a Creative Commons Attribution 4.0 International License.
Flexible Duration
44
init.mp4
seg-01.m4s
seg-02.m4s
seg-86.m4s
seg-87.m4s
Duration: 9min57sec init.mp4
seg-01.m4s
seg-02.m4s
seg-03.m4s
seg-04.m4s
Duration: 0
・
・
・
・
・
・
Infinite !
45. This slide by mzyy94 and Icons by Font Awesome are licensed under a Creative Commons Attribution 4.0 International License.
Adaptive Bitrate
45
240p
360p
540p
720p
1080p
2160p
init 01 02 03 04 05 06 07
Source Buffer
46. This slide by mzyy94 and Icons by Font Awesome are licensed under a Creative Commons Attribution 4.0 International License.
Less Buffering
46
Waiting… Done! Can Play!
47. This slide by mzyy94 and Icons by Font Awesome are licensed under a Creative Commons Attribution 4.0 International License.
Upcoming
• Learn All APIs so as to play Web Streaming
• How to create Streaming Media
• DRM Handcrafting
• Deep Dive into Adaptive Bitrate
• Browser Video Feature Implementation Comparison
47
And more..