These slides cover a workshop called "Having fun with Janus and WebRTC" at the virtual edition of OpenSIPS 2021. The workshop guided viewers to how they could use different features in Janus to build a WebRTC Social TV application, including how to write a new plugin in JavaScript to build a virtual remote.

1. Having fun with Janus and WebRTC!
Lorenzo Miniero
@elminiero
OpenSIPS Summit Distributed
September 13th 2021,

2. Who am I?
Lorenzo Miniero
• Ph.D @ UniNA
• Chairman @ Meetecho
• Main author of Janus
Contacts and info
• lorenzo@meetecho.com
• https://twitter.com/elminiero
• https://www.slideshare.net/LorenzoMiniero
• https://soundcloud.com/lminiero

3. Just a few words on Meetecho
• Co-founded in 2009 as an academic spin-off
• University research efforts brought to the market
• Completely independent from the University
• Focus on real-time multimedia applications
• Strong perspective on standardization and open source
• Several activities
• Consulting services
• Commercial support and Janus licenses
• Streaming of live events (IETF, ACM, etc.)
• Proudly brewed in sunny Napoli(*), Italy

4. ((*)
I’m legally obliged to show this)

5. A quick reminder on what Janus is!
Janus
General purpose, open source WebRTC server
• https://github.com/meetecho/janus-gateway
• Demos and documentation: https://janus.conf.meetecho.com
• Community: https://groups.google.com/forum/#!forum/meetecho-janus

6. Modular architecture
• The core only implements the WebRTC stack
• JSEP/SDP, ICE, DTLS-SRTP, Data Channels, Simulcast, VP9-SVC, ...
• Plugins expose Janus API over different “transports”
• Currently HTTP / WebSockets / RabbitMQ / Unix Sockets / MQTT / Nanomsg
• “Application” logic implemented in plugins too
• Users attach to plugins via the Janus core
• The core handles the WebRTC stuff
• Plugins route/manipulate the media/data
• Plugins can be combined on client side as “bricks”
• Video SFU, Audio MCU, SIP gatewaying, broadcasting, etc.

7. Modular architecture
• The core only implements the WebRTC stack
• JSEP/SDP, ICE, DTLS-SRTP, Data Channels, Simulcast, VP9-SVC, ...
• Plugins expose Janus API over different “transports”
• Currently HTTP / WebSockets / RabbitMQ / Unix Sockets / MQTT / Nanomsg
• “Application” logic implemented in plugins too
• Users attach to plugins via the Janus core
• The core handles the WebRTC stuff
• Plugins route/manipulate the media/data
• Plugins can be combined on client side as “bricks”
• Video SFU, Audio MCU, SIP gatewaying, broadcasting, etc.

8. Modular architecture
• The core only implements the WebRTC stack
• JSEP/SDP, ICE, DTLS-SRTP, Data Channels, Simulcast, VP9-SVC, ...
• Plugins expose Janus API over different “transports”
• Currently HTTP / WebSockets / RabbitMQ / Unix Sockets / MQTT / Nanomsg
• “Application” logic implemented in plugins too
• Users attach to plugins via the Janus core
• The core handles the WebRTC stuff
• Plugins route/manipulate the media/data
• Plugins can be combined on client side as “bricks”
• Video SFU, Audio MCU, SIP gatewaying, broadcasting, etc.

9. Modular architecture
• The core only implements the WebRTC stack
• JSEP/SDP, ICE, DTLS-SRTP, Data Channels, Simulcast, VP9-SVC, ...
• Plugins expose Janus API over different “transports”
• Currently HTTP / WebSockets / RabbitMQ / Unix Sockets / MQTT / Nanomsg
• “Application” logic implemented in plugins too
• Users attach to plugins via the Janus core
• The core handles the WebRTC stuff
• Plugins route/manipulate the media/data
• Plugins can be combined on client side as “bricks”
• Video SFU, Audio MCU, SIP gatewaying, broadcasting, etc.

10. A ton of scenarios done today with Janus!
• SIP and RTSP gatewaying
• WebRTC-based call/contact centers
• Conferencing & collaboration
• E-learning & webinars
• Cloud platforms
• Media production
• Broadcasting & Gaming
• Identity verification
• Internet of Things
• Augmented/Virtual Reality
• ...and more!

11. How do you talk to Janus?
https://janus.conf.meetecho.com/docs/rest

12. How do you talk to Janus?
https://janus.conf.meetecho.com/docs/rest

13. Session, Handles and PeerConnections
https://janus.conf.meetecho.com/docs/rest

14. Session, Handles and PeerConnections
https://janus.conf.meetecho.com/docs/rest

15. Session, Handles and PeerConnections
https://janus.conf.meetecho.com/docs/rest

16. Our practical use case for today: SocialTV!

17. A few requirements for a SocialTV demo
• An interesting scenario
• A few friends connect to the same web page to watch TV
• They can talk to each other while watching
• Any of them can change channel at any time
• Effective example of combining different plugins
• We’ll need something for the TV broadcasting...
• ... something to let them mingle ...
• ... and something for the interactive features (e.g., channel surfing)
• This workshop will guide you through the process
• Choosing plugins, combining handles, etc.

18. A few requirements for a SocialTV demo
• An interesting scenario
• A few friends connect to the same web page to watch TV
• They can talk to each other while watching
• Any of them can change channel at any time
• Effective example of combining different plugins
• We’ll need something for the TV broadcasting...
• ... something to let them mingle ...
• ... and something for the interactive features (e.g., channel surfing)
• This workshop will guide you through the process
• Choosing plugins, combining handles, etc.

19. A few requirements for a SocialTV demo
• An interesting scenario
• A few friends connect to the same web page to watch TV
• They can talk to each other while watching
• Any of them can change channel at any time
• Effective example of combining different plugins
• We’ll need something for the TV broadcasting...
• ... something to let them mingle ...
• ... and something for the interactive features (e.g., channel surfing)
• This workshop will guide you through the process
• Choosing plugins, combining handles, etc.

20. First obvious choice: VideoRoom plugin!
https://janus.conf.meetecho.com/docs/videoroom

21. Chatting with friends using the VideoRoom
• The VideoRoom plugin is an SFU (Selective Forwarding Unit)
• Publish/Subscribe approach
• Participants can send their media and subscribe to others
• Very widely used in conferencing scenarios
• Obvious choice for the friends mingling feature
• They’re basically in a “conference” with each other
• We may want to constrain it a bit compared to a traditional conference, though
• Maybe a limited number of participants? (e.g., 4)
• Capping the bandwidth is a good idea too (they’ll be thumbnails)

22. Chatting with friends using the VideoRoom
• The VideoRoom plugin is an SFU (Selective Forwarding Unit)
• Publish/Subscribe approach
• Participants can send their media and subscribe to others
• Very widely used in conferencing scenarios
• Obvious choice for the friends mingling feature
• They’re basically in a “conference” with each other
• We may want to constrain it a bit compared to a traditional conference, though
• Maybe a limited number of participants? (e.g., 4)
• Capping the bandwidth is a good idea too (they’ll be thumbnails)

23. Chatting with friends using the VideoRoom
• The VideoRoom plugin is an SFU (Selective Forwarding Unit)
• Publish/Subscribe approach
• Participants can send their media and subscribe to others
• Very widely used in conferencing scenarios
• Obvious choice for the friends mingling feature
• They’re basically in a “conference” with each other
• We may want to constrain it a bit compared to a traditional conference, though
• Maybe a limited number of participants? (e.g., 4)
• Capping the bandwidth is a good idea too (they’ll be thumbnails)

24. Sample configuration for the room
room-1000: {
description = "Social TV Room"
secret = "verysecret"
pin = "zapp"
require_pvtid = true
publishers = 4
bitrate = 200000
fir_freq = 10
audiocodec = "opus"
videocodec = "vp8"
transport_wide_cc_ext = true
}

25. Broadcasting with the Streaming plugin
https://janus.conf.meetecho.com/docs/streaming

26. Using the Streaming plugin for TV channels
• The Streaming plugin is an effective RTP-to-WebRTC broadcaster
• It works by creating so called “mountpoints” to identify a stream
• A “mountpoint” can receive RTP from any source (FFmpeg/GStreamer/VLC/others)
• Multiple participants can subscribe to receive the same stream via WebRTC
• Easy to generate WebRTC streams using non-WebRTC tools
• Tools only need to understand RTP and support the right codecs
• Janus “wraps” RTP in WebRTC coat, but won’t transcode
• Browsers need to be able to decode the media
• Only need a single copy of the stream
• The Streaming plugin duplicates it to interested subscribers

27. Using the Streaming plugin for TV channels
• The Streaming plugin is an effective RTP-to-WebRTC broadcaster
• It works by creating so called “mountpoints” to identify a stream
• A “mountpoint” can receive RTP from any source (FFmpeg/GStreamer/VLC/others)
• Multiple participants can subscribe to receive the same stream via WebRTC
• Easy to generate WebRTC streams using non-WebRTC tools
• Tools only need to understand RTP and support the right codecs
• Janus “wraps” RTP in WebRTC coat, but won’t transcode
• Browsers need to be able to decode the media
• Only need a single copy of the stream
• The Streaming plugin duplicates it to interested subscribers

28. Using the Streaming plugin for TV channels
• The Streaming plugin is an effective RTP-to-WebRTC broadcaster
• It works by creating so called “mountpoints” to identify a stream
• A “mountpoint” can receive RTP from any source (FFmpeg/GStreamer/VLC/others)
• Multiple participants can subscribe to receive the same stream via WebRTC
• Easy to generate WebRTC streams using non-WebRTC tools
• Tools only need to understand RTP and support the right codecs
• Janus “wraps” RTP in WebRTC coat, but won’t transcode
• Browsers need to be able to decode the media
• Only need a single copy of the stream
• The Streaming plugin duplicates it to interested subscribers

29. Using the Streaming plugin for TV channels
• The Streaming plugin is an effective RTP-to-WebRTC broadcaster
• It works by creating so called “mountpoints” to identify a stream
• A “mountpoint” can receive RTP from any source (FFmpeg/GStreamer/VLC/others)
• Multiple participants can subscribe to receive the same stream via WebRTC
• Easy to generate WebRTC streams using non-WebRTC tools
• Tools only need to understand RTP and support the right codecs
• Janus “wraps” RTP in WebRTC coat, but won’t transcode
• Browsers need to be able to decode the media
• Only need a single copy of the stream
• The Streaming plugin duplicates it to interested subscribers

30. Different ways of implementing TV channels
• One mountpoint per room?
• RTP source of the mountpoint must change when surfing TV
• Requires server side management of the media
• No need for channel synchronization among participants, but...
• ... can’t re-use mountpoints/channels for different rooms
• One mountpoint per channel?
• RTP source of the mountpoint never changes
• Participants change mountpoint they receive when surfing TV
• Does require channel synchronization among participants, but...
• ... allows channels re-use for different rooms

31. Different ways of implementing TV channels
• One mountpoint per room?
• RTP source of the mountpoint must change when surfing TV
• Requires server side management of the media
• No need for channel synchronization among participants, but...
• ... can’t re-use mountpoints/channels for different rooms
• One mountpoint per channel?
• RTP source of the mountpoint never changes
• Participants change mountpoint they receive when surfing TV
• Does require channel synchronization among participants, but...
• ... allows channels re-use for different rooms

32. Different ways of implementing TV channels
• One mountpoint per room?
• RTP source of the mountpoint must change when surfing TV
• Requires server side management of the media
• No need for channel synchronization among participants, but...
• ... can’t re-use mountpoints/channels for different rooms
• One mountpoint per channel?
• RTP source of the mountpoint never changes
• Participants change mountpoint they receive when surfing TV
• Does require channel synchronization among participants, but...
• ... allows channels re-use for different rooms

33. Different ways of implementing TV channels
• One mountpoint per room?
• RTP source of the mountpoint must change when surfing TV
• Requires server side management of the media
• No need for channel synchronization among participants, but...
• ... can’t re-use mountpoints/channels for different rooms
• One mountpoint per channel?
• RTP source of the mountpoint never changes
• Participants change mountpoint they receive when surfing TV
• Does require channel synchronization among participants, but...
• ... allows channels re-use for different rooms

34. Different ways of implementing TV channels
• One mountpoint per room?
• RTP source of the mountpoint must change when surfing TV
• Requires server side management of the media
• No need for channel synchronization among participants, but...
• ... can’t re-use mountpoints/channels for different rooms
• One mountpoint per channel?
• RTP source of the mountpoint never changes
• Participants change mountpoint they receive when surfing TV
• Does require channel synchronization among participants, but...
• ... allows channels re-use for different rooms

35. Different ways of implementing TV channels
• One mountpoint per room?
• RTP source of the mountpoint must change when surfing TV
• Requires server side management of the media
• No need for channel synchronization among participants, but...
• ... can’t re-use mountpoints/channels for different rooms
• One mountpoint per channel?
• RTP source of the mountpoint never changes
• Participants change mountpoint they receive when surfing TV
• Does require channel synchronization among participants, but...
• ... allows channels re-use for different rooms

36. Different ways of implementing TV channels
• One mountpoint per room?
• RTP source of the mountpoint must change when surfing TV
• Requires server side management of the media
• No need for channel synchronization among participants, but...
• ... can’t re-use mountpoints/channels for different rooms
• One mountpoint per channel?
• RTP source of the mountpoint never changes
• Participants change mountpoint they receive when surfing TV
• Does require channel synchronization among participants, but...
• ... allows channels re-use for different rooms

37. Different ways of implementing TV channels
• One mountpoint per room?
• RTP source of the mountpoint must change when surfing TV
• Requires server side management of the media
• No need for channel synchronization among participants, but...
• ... can’t re-use mountpoints/channels for different rooms
• One mountpoint per channel?
• RTP source of the mountpoint never changes
• Participants change mountpoint they receive when surfing TV
• Does require channel synchronization among participants, but...
• ... allows channels re-use for different rooms

38. Mapping TV channels to mountpoints
• Each channel can be mapped to a separate mountpoint
• Independent channels and separate RTP media sources
• A cool feature called mountpoint “switching”
• If you’re watching mountpoint #1, you can “switch” to #2
• No need to create a new PeerConnection for the purpose, it’s the source that changes
• Only works if the two mountpoints were configured with the same codecs, of course
• For our SocialTV, different mountpoints can be different TV channels
• Participants start with a channel and can change dynamically
• Logic used to change channels can be delegated to a different plugin

39. Mapping TV channels to mountpoints
• Each channel can be mapped to a separate mountpoint
• Independent channels and separate RTP media sources
• A cool feature called mountpoint “switching”
• If you’re watching mountpoint #1, you can “switch” to #2
• No need to create a new PeerConnection for the purpose, it’s the source that changes
• Only works if the two mountpoints were configured with the same codecs, of course
• For our SocialTV, different mountpoints can be different TV channels
• Participants start with a channel and can change dynamically
• Logic used to change channels can be delegated to a different plugin

40. Mapping TV channels to mountpoints
• Each channel can be mapped to a separate mountpoint
• Independent channels and separate RTP media sources
• A cool feature called mountpoint “switching”
• If you’re watching mountpoint #1, you can “switch” to #2
• No need to create a new PeerConnection for the purpose, it’s the source that changes
• Only works if the two mountpoints were configured with the same codecs, of course
• For our SocialTV, different mountpoints can be different TV channels
• Participants start with a channel and can change dynamically
• Logic used to change channels can be delegated to a different plugin

41. Sample configuration for a mountpoint
rtp-sample-12: {
type = "rtp"
id = 12
description = "Sports"
audio = true
video = true
audioport = 6002
audiopt = 111
audiortpmap = "opus/48000/2"
videoport = 6004
videortcpport = 6005
videopt = 100
videortpmap = "VP8/90000"
secret = "verysecret"
}

42. Sample configuration for another mountpoint
rtp-sample-13: {
type = "rtp"
id = 13
description = "Comedy"
audio = true
video = true
audioport = 7002
audiopt = 111
audiortpmap = "opus/48000/2"
videoport = 7004
videortcpport = 7005
videopt = 100
videortpmap = "VP8/90000"
secret = "verysecret"
}

43. Implementing a shared TV remote
• The Streaming plugin can switch from channel to channel dynamically
• ... but this only applies to an individual participant
• How should we synchronize this among all of them?
• A ton of different ways to implement this
• Maybe an HTTP/WS based web application (e.g., via node.js)
• Could leverage an existing framework (e.g., Firebase)
• Why not an instant messaging protocol? (e.g., XMPP)
• WebRTC datachannels are an option too, of course!

44. Implementing a shared TV remote
• The Streaming plugin can switch from channel to channel dynamically
• ... but this only applies to an individual participant
• How should we synchronize this among all of them?
• A ton of different ways to implement this
• Maybe an HTTP/WS based web application (e.g., via node.js)
• Could leverage an existing framework (e.g., Firebase)
• Why not an instant messaging protocol? (e.g., XMPP)
• WebRTC datachannels are an option too, of course!

45. Using datachannels for our TV remote
• When looking at datachannels, many options available
• Peer-to-peer connections, without going through Janus?
• Piggybacking existing VideoRoom streams (audio/video AND data)
• Using the TextRoom instant messaging functionality
• Writing a new ad-hoc plugin (in C, Lua or JavaScript)
For the sake of this workshop, let’s write a new plugin!
• Writing one in JavaScript (Duktape plugin) is trivial and informative
• Exchanging datachannel messages is a simple logic to implement
• We can keep some server-side state too (list of channels, current channels, etc.)

46. Using datachannels for our TV remote
• When looking at datachannels, many options available
• Peer-to-peer connections, without going through Janus?
• Piggybacking existing VideoRoom streams (audio/video AND data)
• Using the TextRoom instant messaging functionality
• Writing a new ad-hoc plugin (in C, Lua or JavaScript)
For the sake of this workshop, let’s write a new plugin!
• Writing one in JavaScript (Duktape plugin) is trivial and informative
• Exchanging datachannel messages is a simple logic to implement
• We can keep some server-side state too (list of channels, current channels, etc.)

47. Writing a Janus plugin in JavaScript
https://janus.conf.meetecho.com/docs/duktape

48. Sequence diagram for the interactions

49. Event on existing channels and active one
{
"channels": {
"1": {
"name": "Tech",
"title": "Meetecho Spot"
},
"12": {
"name": "Sports",
"title": "Napoli Milan 4-2"
},
"13": {
"name": "Comedy",
"title": "SNL - Cold Commercial"
},
"14": {
"name": "Music",
"title": "Iron Maiden - Be quick or be dead"
}
},
"currentChannel": 1
}

50. Changing TV channels
Request (sent by Bob):
{
"request": "change",
"channel": 12
}
Response:
{
"changed": {
"name": "Bob",
"channel": 12
}
}

51. Putting it all together in a web application
• Three different Janus plugins used at the same time
• Custom Duktape plugin (one handle, datachannel PeerConnection)
• Streaming plugin (one handle, recvonly audio/video PeerConnection)
• VideoRoom plugin (multiple handles, sendonly publisher + N recvonly subscribers)
• Potential workflow for the UI
1 Prompt participant for a display name
2 Use the display name to join as a VideoRoom publisher
3 Create a datachannel with the custom Duktape plugin
4 Create subscribers for other VideoRoom participants in the same room
5 Update UI with list of channels received via datachannels
6 Subscribe to the mountpoint associated with the current channel
7 In case of channel changes, switch the mountpoint source

52. Putting it all together in a web application
• Three different Janus plugins used at the same time
• Custom Duktape plugin (one handle, datachannel PeerConnection)
• Streaming plugin (one handle, recvonly audio/video PeerConnection)
• VideoRoom plugin (multiple handles, sendonly publisher + N recvonly subscribers)
• Potential workflow for the UI
1 Prompt participant for a display name
2 Use the display name to join as a VideoRoom publisher
3 Create a datachannel with the custom Duktape plugin
4 Create subscribers for other VideoRoom participants in the same room
5 Update UI with list of channels received via datachannels
6 Subscribe to the mountpoint associated with the current channel
7 In case of channel changes, switch the mountpoint source

53. Putting it all together in a web application
• Three different Janus plugins used at the same time
• Custom Duktape plugin (one handle, datachannel PeerConnection)
• Streaming plugin (one handle, recvonly audio/video PeerConnection)
• VideoRoom plugin (multiple handles, sendonly publisher + N recvonly subscribers)
• Potential workflow for the UI
1 Prompt participant for a display name
2 Use the display name to join as a VideoRoom publisher
3 Create a datachannel with the custom Duktape plugin
4 Create subscribers for other VideoRoom participants in the same room
5 Update UI with list of channels received via datachannels
6 Subscribe to the mountpoint associated with the current channel
7 In case of channel changes, switch the mountpoint source

54. Putting it all together in a web application
• Three different Janus plugins used at the same time
• Custom Duktape plugin (one handle, datachannel PeerConnection)
• Streaming plugin (one handle, recvonly audio/video PeerConnection)
• VideoRoom plugin (multiple handles, sendonly publisher + N recvonly subscribers)
• Potential workflow for the UI
1 Prompt participant for a display name
2 Use the display name to join as a VideoRoom publisher
3 Create a datachannel with the custom Duktape plugin
4 Create subscribers for other VideoRoom participants in the same room
5 Update UI with list of channels received via datachannels
6 Subscribe to the mountpoint associated with the current channel
7 In case of channel changes, switch the mountpoint source

55. Putting it all together in a web application
• Three different Janus plugins used at the same time
• Custom Duktape plugin (one handle, datachannel PeerConnection)
• Streaming plugin (one handle, recvonly audio/video PeerConnection)
• VideoRoom plugin (multiple handles, sendonly publisher + N recvonly subscribers)
• Potential workflow for the UI
1 Prompt participant for a display name
2 Use the display name to join as a VideoRoom publisher
3 Create a datachannel with the custom Duktape plugin
4 Create subscribers for other VideoRoom participants in the same room
5 Update UI with list of channels received via datachannels
6 Subscribe to the mountpoint associated with the current channel
7 In case of channel changes, switch the mountpoint source

56. Putting it all together in a web application
• Three different Janus plugins used at the same time
• Custom Duktape plugin (one handle, datachannel PeerConnection)
• Streaming plugin (one handle, recvonly audio/video PeerConnection)
• VideoRoom plugin (multiple handles, sendonly publisher + N recvonly subscribers)
• Potential workflow for the UI
1 Prompt participant for a display name
2 Use the display name to join as a VideoRoom publisher
3 Create a datachannel with the custom Duktape plugin
4 Create subscribers for other VideoRoom participants in the same room
5 Update UI with list of channels received via datachannels
6 Subscribe to the mountpoint associated with the current channel
7 In case of channel changes, switch the mountpoint source

57. Putting it all together in a web application
• Three different Janus plugins used at the same time
• Custom Duktape plugin (one handle, datachannel PeerConnection)
• Streaming plugin (one handle, recvonly audio/video PeerConnection)
• VideoRoom plugin (multiple handles, sendonly publisher + N recvonly subscribers)
• Potential workflow for the UI
1 Prompt participant for a display name
2 Use the display name to join as a VideoRoom publisher
3 Create a datachannel with the custom Duktape plugin
4 Create subscribers for other VideoRoom participants in the same room
5 Update UI with list of channels received via datachannels
6 Subscribe to the mountpoint associated with the current channel
7 In case of channel changes, switch the mountpoint source

58. Putting it all together in a web application
• Three different Janus plugins used at the same time
• Custom Duktape plugin (one handle, datachannel PeerConnection)
• Streaming plugin (one handle, recvonly audio/video PeerConnection)
• VideoRoom plugin (multiple handles, sendonly publisher + N recvonly subscribers)
• Potential workflow for the UI
1 Prompt participant for a display name
2 Use the display name to join as a VideoRoom publisher
3 Create a datachannel with the custom Duktape plugin
4 Create subscribers for other VideoRoom participants in the same room
5 Update UI with list of channels received via datachannels
6 Subscribe to the mountpoint associated with the current channel
7 In case of channel changes, switch the mountpoint source

59. Handles/PeerConnections per participant

60. Time to see the demo running!

61. Thanks! Questions? Comments?
Get in touch!
• https://twitter.com/elminiero
• https://twitter.com/meetecho
• http://www.meetecho.com