Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Eyeball Messenger SDK WebRTC Developer Reference Guide

9,219 views

Published on

Messenger SDK WebRTC is a set of libraries which enable WebRTC-compatible voice calling, video chat, and P2P file sharing with no plugins! With Messenger SDK WebRTC, you can build WebRTC applications for all major platforms including Android, Internet Explorer, iOS, OS X, Safari, Windows, Windows RT, and more.

Messenger SDK WebRTC includes STUN/TURN/ICE for guaranteed connections on any fixed or mobile network, through any NAT or firewall, and on any device. Messenger SDK WebRTC works with any third-party signaling (e.g. Jingle, SIP, WebSync) and STUN/TURN server or service.

Published in: Software
  • Be the first to comment

Eyeball Messenger SDK WebRTC Developer Reference Guide

  1. 1. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. Eyeball Messenger Interoperable with WebRTC Developer Reference Guide Last Modified: February 2014 Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved.
  2. 2. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. Introduction Thanks for using Messenger! We hope you find Messenger to be both powerful and easy to use. If you have any questions or encounter any difficulties, please get in touch with us. What is Messenger? Messenger is a collection of libraries that enable you to create reliable UDP media streams between peers, regardless of the peer's network configuration and environment. Messenger traverses every possible firewall/NAT combination to guarantee connection establishment. It uses IETF/IANA standards to provide the broadest compatiblity with third-party components. What platforms does Messenger support? Messenger has libraries for JavaScript, iOS, Java, Mac, Mono, .NET, .NET Compact, Unity, Windows Phone, Windows 8, Xamarin.Android and Xamarin.iOS. If you have a particular platform need, please let us know. Does Messenger support WebRTC? Yes! Messenger's network tunneling strategies are identical to the ones recommended by the WebRTC specification. The WebRTC extension bundled with Messenger provides a full audio/video/data-channel stack that is fully interoperable with modern WebRTC implementations. What about older web browsers? Messenger comes bundled with a Java applet that includes a complete WebRTC implementation. It is integrated with the JavaScript SDK tightly so that the browser will automatically fall-back to use the applet if native WebRTC functionality is unavailable. This graceful degradation is completely seamless. You never have to touch a single line of Java code. What are you waiting for? Start building your P2P app!
  3. 3. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. Signaling Before a P2P link can be established, peers must exchange some information - a description of the streams and formats that will be used, called the offer/answer, and a set of network "candidates". (The initiating peer creates what is called the offer, and the other peer creates an answer in response. Both peers create candidates.) A well-known protocol called SDP (Session Description Protocol) is used to define both the offer/answer and candidates. Messenger will generate the offers/answers and candidates, but it needs some external messaging system to send them from one peer to another. We use WebSync in all our examples. Other options include SIP, XMPP, and Google's Channel API. The process itself is simple. Messenger starts out with a Conference. Whenever Messenger generates an OfferAnswer or Candidate, the Conference fires an event (OnOfferAnswer or OnCandidate). Your responsibility as the developer is to ensure that: 1. Whenever an OfferAnswer or Candidate is raised, the SDP message in the event arguments is sent to the other peer. 2. Whenever an SDP message is received from a peer, either ReceiveOfferAnswer or ReceiveCandidate is called. P2P links are often encrypted, and for this reason, we highly recommend using a secure channel to send SDP offers/answers and candidates between peers. If you are going to use WebSync, make sure you have a valid SSL certificate installed on your server and you use HTTPS to connect your clients.
  4. 4. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. Conferences Conferences are the core of Messenger. Whether you are creating a link to one peer or several, you start by creating a Conference. Conferences are associated with an Messenger server and a number of stream descriptions. The Messenger server is used when connecting to a new peer to facilitate network discovery (and possibly relay data for highly restrictive firewalls). The stream descriptions are used to create the offer/answer that will be used to describe the types of data/media that will be exchanged. Conferences make sure that links between peers are created seamlessly with a consistent eventing structure. Each link that is created goes through a simple event cycle: 1. LinkInit is fired when a new link is about to start. The initiating peer triggers this event explicitly by calling Conference.Link, perhaps as the result of dialing a number or double-clicking on a contact. The other peer triggers this event implicitly by calling Conference.ReceiveOfferAnswer when an SDP offer/answer arrives on your signalling connection. 2. LinkUp is fired when a link has been successfully established. If this link fires, it does so after LinkInit has been fired. This event provides a reference to the negotiated stream formats that both peers had in common. 3. LinkDown is fired when either (a) a link could not be established or (b) a link was established, but went down. The WasUp property lets you differentiate between the two. Each event will fire at most once for each unique link. After a link goes down, it is discarded. A reconnection will result in the creation of a new link.
  5. 5. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. WebRTC Where possible, Messenger providers a WebRTC extension that allows you to create WebRTC- compatible audio/video/data streams between peers, including web browsers. What is WebRTC? In a nutshell, it's an open project backed by Google, Mozilla, Ericsson, and other industry leaders that is attempting to standardize real-time P2P audio/video/data communications and make them accessible in a web browser without the use of plug-ins. Messenger's network traversal strategy is identical to the one recommended by WebRTC. Building on this, the WebRTC extension for Messenger provides a complete audio/video/data-channel stack that lets you send/receive P2P data between a web browser and a desktop client, all without plugins. We do all the heavy lifting:  Stream formatting.  RTP packet processing.  RTCP packet processing.  Audio encoding and decoding (G.711u, G.711a).  Video encoding and decoding (VP8, Motion-JPEG).  Audio capturing from device microphone.  Video capturing from device camera.  Audio rendering to device speakers.  Video rendering to screen container. For audio/video feeds, first get a reference to the device's local media stream by calling UserMedia.GetMedia. If successful, create a WebRTC AudioStream, VideoStream, and/or DataChannelStream and use those stream definitions when creating your Conference. The rest is automatic! We even include a LayoutManager for each platform that helps you position your video feed controls in your interface.
  6. 6. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. Creating a Conference: iOS and Mac The iOS/Mac SDK provides a complete peer-to-peer (P2P) streaming solution, including a complete WebRTC stack for rapidly creating audio, video, and data-channel streams that can be consumed natively in modern web browsers. To get started, you'll need to add some static libraries and headers from the SDK to your project (Xcode):  libEyeball.a + Eyeball.h (supporting code)  libEyeballMessenger.a + EyeballMessenger.h (Messenger core)  libEyeballMessengerWebRTC.a + EyeballMessengerWebRTC.h (WebRTC stack, optional) Also add a few Apple framework dependencies:  libz.dylib  Security.framework  CFNetwork.framework (iOS only) For WebRTC only, some additional dependencies are required:  AudioToolbox.framework  AudioUnit.framework (Mac only)  AVFoundation.framework  CoreAudio.framework  CoreGraphics.framework  CoreMedia.framework  CoreVideo.framework  GLKit.framework (iOS only)  OpenGLES.framework (iOS only) Finally, add "-all_load" to "Other Linker Flags" under the target's build settings. Now let's write some code! (See iOS/Mac.Client, iOS/Mac.Client.WebRTC, and iOS/Mac.Client.DataChannels in the SDK Examples folder.) Part 1: Stream Descriptions The first step is to describe the streams we want to use. Stream descriptions are used to define the types of media we are going to be sending back and forth between peers in a conference. Messenger allows us to createcustom streams with any number of possible data formats. The WebRTC extension allows us to
  7. 7. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. create audio, video, and data-channel streams. You can mix and match custom and WebRTC streams, but only WebRTC streams can be consumed natively by web browsers. Part 1a: Custom Stream Descriptions Custom stream descriptions require a "stream type" and one or more "stream formats". The stream type is used in the SDP offer/answer to help differentiate between multiple streams (audio, video, application/binary, etc.). The stream format is used in the SDP offer/answer to help differentiate between multiple data formats supported for a given stream (i.e. G.711, G.722, Opus, etc. for audio). // Create a custom stream description that indicates text // data will be sent using a format labelled "utf8". EyeballMessengerStream *customStream = [EyeballMessengerStream streamWithType:EyeballMessengerStreamTypeText format:[EyeballMessengerStreamFormat streamFormatWithEncodingName:@"utf8"]]; [customStream addOnLinkReceiveRTPWithValueBlock:^(EyeballMessengerStreamLinkReceiveRTPArgs *e) { NSLog(@"%@", [[EyeballEncoding utf8] getStringWithBytes:e.packet.payload]); }]; // To send custom stream data, use EyeballMessengerConference.sendRTPWithFormat:packet: // once you have created a conference (see Part 2): // <code> // [conference sendRTPWithFormat:customStream.format packet:[EyeballMessengerRTPPacket rtpPacketWithPayload:[EyeballEncoding.utf8 getBytesWithS:@"Hello, world!"]]]; // </code> // TO BE CONTINUED... Part 1b: WebRTC Data-Channel Stream Description
  8. 8. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. WebRTC data-channel streams are easy. Describe one or more data channels (name + message handler) and pass them into the stream description. // Create a WebRTC data channel description, including a // handler for processing received messages. EyeballMessengerWebRTCDataChannelInfo *dataChannelInfo = [EyeballMessengerWebRTCDataChannelInfo dataChannelInfoWithLabel:@"mydatachannel"]; [dataChannelInfo setOnReceiveBlock:^(EyeballMessengerWebRTCDataChannelReceiveArgs *e) { NSLog(@"%@", e.data); }]; // Create a WebRTC data channel stream description using // our data channel. EyeballMessengerWebRTCDataChannelStream *dataChannelStream = [EyeballMessengerWebRTCDataChannelStream dataChannelStreamWithChannelInfo:dataChannelInfo]; // To send data-channel stream data, use EyeballMessengerConference.sendDataWithChannelInfo:data: // once you have created a conference (see Part 2): // <code> // [conference sendDataWithChannelInfo:dataChannelInfo data:@"Hello, world!"]; // </code> // TO BE CONTINUED... Part 1c: WebRTC Audio/Video Stream Descriptions
  9. 9. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. WebRTC audio and video stream descriptions require a reference to the local media (microphone, camera, or both), obtained by calling [EyeballMessengerWebRTCUserMedia getMediaWithGetMediaArgs:]. // WebRTC has chosen VP8 as its mandatory video codec. // Since video encoding is best done using native code, // reference the video codec at the application-level. // This is required when using a WebRTC video stream. [EyeballMessengerWebRTCVideoStream registerCodecWithEncodingName:@"VP8" createCodecBlock:^() { return [[[Vp8Codec alloc] init] autorelease]; } preferred:YES]; // WebRTC audio and video streams require us to first get access to // the local media (microphone, camera, or both). EyeballMessengerWebRTCGetMediaArgs *getMediaArgs = [EyeballMessengerWebRTCGetMediaArgs getMediaArgsWithAudio:YES video:YES]; [getMediaArgs setVideoWidth:320]; // optional [getMediaArgs setVideoHeight:240]; // optional [getMediaArgs setVideoFrameRate:30]; // optional [getMediaArgs setOnFailureBlock:^(EyeballMessengerWebRTCGetMediaFailureArgs *e) { [self alert:@"Could not get media. %@", e.exception.message]; }]; [getMediaArgs setOnSuccessBlock:^(EyeballMessengerWebRTCGetMediaSuccessArgs *e) { // We have successfully acquired access to the local // audio/video device! Grab a reference to the media.
  10. 10. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. // Internally, it maintains access to the local audio // and video feeds coming from the device hardware. EyeballMessengerWebRTCLocalMediaStream *localMedia = e.localStream; // This is our local video control, a UIView (iOS) or // and NSView (Mac). It is constantly updated with our // live video feed since we requested video above. Add // it directly to the UI or use the Messenger layout manager, // which we do below. NSObject *localVideoControl = e.localVideoControl; // Create an Messenger layout manager, which makes the task // of arranging video controls easy. Give it a reference // to a UIView (iOS) or NSView (Mac) that can be filled // with video feeds. EyeballMessengerWebRTCLayoutManager *layoutManager = [EyeballMessengerWebRTCLayoutManager layoutManagerWithContainer:container]; // Position and display the local video control on-screen // by passing it to the layout manager created above. [layoutManager setLocalVideoControlWithLocalVideoControl:localVideoControl]; // Create a WebRTC audio stream description (requires a // reference to the local audio feed). EyeballMessengerWebRTCAudioStream *audioStream = [EyeballMessengerWebRTCAudioStream audioStreamWithLocalStream:localMedia]; // Create a WebRTC video stream description (requires a
  11. 11. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. // reference to the local video feed). Whenever a P2P link // initializes using this description, position and display // the remote video control on-screen by passing it to the // layout manager created above. Whenever a P2P link goes // down, remove it. EyeballMessengerWebRTCVideoStream *videoStream = [EyeballMessengerWebRTCVideoStream videoStreamWithLocalStream:localMedia]; [videoStream addOnLinkInitWithValueBlock:^(EyeballMessengerStreamLinkInitArgs *e) { NSObject *remoteVideoControl = [e.link getRemoteVideoControl]; [layoutManager addRemoteVideoControlWithPeerId:e.peerId remoteVideoControl:remoteVideoControl]; }]; [videoStream addOnLinkDownWithValueBlock:^(EyeballMessengerStreamLinkDownArgs *e) { [layoutManager removeRemoteVideoControlWithPeerId:e.peerId]; }]; // TO BE CONTINUED... Part 2: The Conference Now that we have some stream descriptions, we are ready to create a conference - the hub that manages your P2P links. Conferences require the address/port of a STUN/TURN server. STUN provides a way for peers to discover their public IP addresses, and TURN provides a way to relay data through highly restrictive firewalls. Messenger comes bundled with a STUN/TURN server implementation, so you can run your own if you have access to a computer that is directly exposed to the public Internet. Alternatively, there are several public STUN servers that are free-to-use. Free TURN servers are more difficult to find, since relaying with TURN can consume significant server bandwidth.
  12. 12. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. // CONTINUED FROM ABOVE... // Create a conference using our stream descriptions. Replace // 127.0.0.1 : 3478 with your STUN/TURN (Messenger) server address. NSMutableArray *streams = [NSMutableArray arrayWithObjects:audioStream, videoStream, dataChannelStream, nil]; EyeballMessengerConference *conference = [EyeballMessengerConference conferenceWithServerAddress:@"127.0.0.1" serverPort:3478 streams:streams]; // Supply TURN relay credentials in case we are behind a // highly restrictive firewall. These credentials will be // verified by the TURN server. [conference setRelayUsername:@"test"]; [conference setRelayPassword:@"pa55w0rd!"]; // Add a few event handlers to the conference so we can see // when a new P2P link is created or changes state. [conference addOnLinkInitWithValueBlock:^(EyeballMessengerLinkInitArgs *e) { NSLog(@"Link to peer initializing..."); }]; [conference addOnLinkUpWithValueBlock:^(EyeballMessengerLinkUpArgs *e) { NSLog(@"Link to peer is UP."); }];
  13. 13. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. [conference addOnLinkDownWithValueBlock:^(EyeballMessengerLinkDownArgs *e) { NSLog([NSString stringWithFormat:@"Link to peer is DOWN. %@", e.exception.message]); }]; // TO BE CONTINUED... Part 3: Signalling With our conference in hand, we are almost done! Just one more thing to set up. Before we can create a P2P link, the peers have to exchange some information - specifically descriptions of the streams (called the offer/answer) and some local network addresses (called candidates). Messengergenerates this information automatically, but you are responsible for distributing it to the peer as quickly as possible. This is called "signalling". We're going to use WebSync here, but other popular options include SIP and XMPP - any real-time messaging system will do. We use WebSync since it uses HTTP (WebSockets/long-polling) and therefore has no issues with firewalls or connecting from JavaScript-based web applications. // CONTINUED FROM ABOVE... // Create a WebSync client and establish a persistent // connection to the server. Replace localhost with your // WebSync server address. EyeballWebSyncClient *client = [EyeballWebSyncClient clientWithRequestUrl:@"http://localhost/websync.ashx"]; EyeballWebSyncConnectArgs *connectArgs = [EyeballWebSyncConnectArgs connectArgs]; [connectArgs setOnFailureBlock:^(EyeballWebSyncConnectFailureArgs *e) { NSLog(@"Could not connect to WebSync. %@", e.exception.message);
  14. 14. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. }]; [client connectWithConnectArgs:connectArgs]; // Add a couple event handlers to the conference to send // generated offers/answers and candidates to a peer. // The peer ID is something we define later. In this case, // it represents the remote WebSync client ID. WebSync's // "notify" method is used to send data to a specific client. [conference addOnLinkOfferAnswerWithValueBlock:^(EyeballMessengerLinkOfferAnswerArgs *e) { [client notifyWithNotifyArgs:[EyeballWebSyncNotifyArgs notifyArgsWithClientId:[EyeballGuid guidWithG:e.peerId] dataJson:[e.offerAnswer toJson] tag:@"offeranswer"]]; }]; [conference addOnLinkCandidateWithValueBlock:^(EyeballMessengerLinkCandidateArgs *e) { [client notifyWithNotifyArgs:[EyeballWebSyncNotifyArgs notifyArgsWithClientId:[EyeballGuid guidWithG:e.peerId] dataJson:[e.candidate toJson] tag:@"candidate"]]; }]; // Add an event handler to the WebSync client to receive // incoming offers/answers and candidates from a peer.
  15. 15. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. // Call the "receiveOfferAnswer" or "receiveCandidate" // method to pass the information to the conference. [client addOnNotifyWithValueBlock:^(EyeballWebSyncNotifyReceiveArgs *e) { NSString *peerId = [e.notifyingClient.clientId toString]; NSObject *peerState = e.notifyingClient.boundRecords; if ([e.tag isEqualToString:@"offeranswer"]) { [conference receiveOfferAnswerWithOfferAnswer:[EyeballMessengerOfferAnswer fromJsonWithOfferAnswerJson:e.dataJson] peerId:peerId peerState:peerState]; } else if ([e.tag isEqualToString:@"candidate"]) { [conference receiveCandidateWithCandidate:[EyeballMessengerCandidate fromJsonWithCandidateJson:e.dataJson] peerId:peerId]; } }]; // TO BE CONTINUED... Part 4: Kicking It Off Everything is set now. We just need to start the process by having one of the peers call Conference.Link. That peer becomes the "controlling agent". The other peer becomes the "controlled agent" and creates a P2P link automatically when it receives the controlling agent's offer. Here's a detailed breakdown of what happens:
  16. 16. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved.  Peer A: The conference will raise the OnLinkInit event.  Peer A: Messenger will generate an offer and raise the OnOfferAnswer event.  Peer A: Our event handler will use WebSync's Notify method to send that offer to the peer.  Peer B: Our event handler will receive that offer and call Conference.ReceiveOfferAnswer.  Peer B: The conference will raise the OnLinkInit event.  Peer B: Messenger will generate an answer and raise the OnOfferAnswer event.  Peer B: Our event handler will use WebSync's Notify method to send that answer to the peer.  Peer A: Our event handler will receive that answer and call Conference.ReceiveOfferAnswer.  Peer A/B: Messenger will generate candidates and raise the OnCandidate event.  Peer A/B: Our event handler will use WebSync's Notify method to send those candidates to the peer.  Peer A/B: Our event handler will receive those candidates and call Conference.ReceiveCandidate.  Peer A/B: Messenger will successfully establish a P2P link.  Peer A/B: The conference will raise the OnLinkUp event. If the P2P link could not be established, the conference will raise the OnLinkDown event. The event arguments will include an Exception property with details. The most common reason for a link failure is a misconfiguration on the server. (Server address or port is wrong, server firewall is blocking UDP traffic, server is on the same network as one of the clients, server is not publicly accessible, etc.) If you are having trouble setting up your Messenger server, try using our public server, located at messenger.eyeball.com : 3478. A common scenario when using a pub-sub framework like WebSync is to create a P2P link whenever two peers join the same channel (and destroy that link when one peer leaves). // CONTINUED FROM ABOVE... // Subscribe to a WebSync channel. When another client joins the same // channel, create a P2P link. When a client leaves, destroy it. EyeballWebSyncSubscribeArgs *subscribeArgs = [EyeballWebSyncSubscribeArgs subscribeArgsWithChannel:@"/mychannel"]; [subscribeArgs setOnFailureBlock:^(EyeballWebSyncSubscribeFailureArgs *e) { NSLog(@"Could not subscribe to %@. %@", e.channel, e.exception.message); }]; [subscribeArgs setOnReceiveBlock:^(EyeballWebSyncSubscribeReceiveArgs *e) { }];
  17. 17. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. [subscribeArgs setOnClientSubscribeWithOnClientSubscribeBlock:^(EyeballWebSyncSubscribersClientSubscr ibeArgs *e) { NSString *peerId = [e.subscribedClient.clientId toString]; NSObject *peerState = e.subscribedClient.boundRecords; [conference linkWithPeerId:peerId peerState:peerState]; }]; [subscribeArgs setOnClientUnsubscribeWithOnClientUnsubscribeBlock:^(EyeballWebSyncSubscribersClientUn subscribeArgs *e) { NSString *peerId = [e.unsubscribedClient.clientId toString]; [conference unlinkWithPeerId:peerId]; }]; [client subscribeWithSubscribeArgs:subscribeArgs]; }]; [EyeballMessengerWebRTCUserMedia getMediaWithGetMediaArgs:getMediaArgs]; Note that when calling Conference.Link, you must supply a unique peer ID. This string can be anything that uniquely identifies the remote peer. This should be the remote WebSync client ID when using WebSync, the remote SIP client ID when using SIP, the remote XMPP client ID when using XMPP... You get the idea. That's it! Load 'er up and watch the magic!
  18. 18. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. Creating a Conference: Java The Java SDK provides a complete peer-to-peer (P2P) streaming solution, including a complete WebRTC stack for rapidly creating audio, video, and data-channel streams that can be consumed natively in modern web browsers. To get started, you'll need to add some JARs from the SDK to your project classpath (IntelliJ IDEA, Eclipse, command-line):  eyeball.jar (supporting code)  eyeball.messenger.jar (Messenger core)  eyeball.messenger.webrtc.jar (WebRTC stack, optional) Now let's write some code! (See Java.Client, Java.Client.WebRTC, and Java.Client.DataChannels in the SDK Examples folder.) Part 1: Stream Descriptions The first step is to describe the streams we want to use. Stream descriptions are used to define the types of media we are going to be sending back and forth between peers in a conference. Messenger allows us to createcustom streams with any number of possible data formats. The WebRTC extension allows us to create audio, video, and data-channel streams. You can mix and match custom and WebRTC streams, but only WebRTC streams can be consumed natively by web browsers. Part 1a: Custom Stream Descriptions Custom stream descriptions require a "stream type" and one or more "stream formats". The stream type is used in the SDP offer/answer to help differentiate between multiple streams (audio, video, application/binary, etc.). The stream format is used in the SDP offer/answer to help differentiate between multiple data formats supported for a given stream (i.e. G.711, G.722, Opus, etc. for audio). // Create a custom stream description that indicates text // data will be sent using a format labelled "utf8". Stream customStream = new eyeball.messenger.Stream(StreamType.Application, new StreamFormat("utf8")); customStream.addOnLinkReceiveRTP(new SingleAction<StreamLinkReceiveRTPArgs>()
  19. 19. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. { public void invoke(StreamLinkReceiveRTPArgs e) { alert(Encoding.getUTF8().getString(e.getPacket().getPayload())); } }); // To send custom stream data, use Conference.sendRTP // once you have created a conference (see Part 2): // <code> // conference.sendRTP(customStream.getFormat(), new RTPPacket(Encoding.getUTF8().getBytes("Hello, world!"))); // </code> // TO BE CONTINUED... Part 1b: WebRTC Data-Channel Stream Description WebRTC data-channel streams are easy. Describe one or more data channels (name + message handler) and pass them into the stream description. // Create a WebRTC data channel description, including a // handler for processing received messages. DataChannelInfo dataChannelInfo = new eyeball.messenger.webrtc.DataChannelInfo("mydatachannel") {{ setOnReceive(new SingleAction<DataChannelReceiveArgs>() { public void invoke(DataChannelReceiveArgs e) {
  20. 20. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. alert(e.getData()); } }); }}; // Create a WebRTC data channel stream description using // our data channel. DataChannelStream dataChannelStream = new eyeball.messenger.webrtc.DataChannelStream(dataChannelInfo); // To send data-channel stream data, use ConferenceExtensions.sendData // once you have created a conference (see Part 2): // <code> // ConferenceExtensions.sendData(conference, dataChannelInfo, "Hello, world!"); // </code> // TO BE CONTINUED... Part 1c: WebRTC Audio/Video Stream Descriptions WebRTC audio and video stream descriptions require a reference to the local media (microphone, camera, or both), obtained by calling UserMedia.getMedia. // WebRTC has chosen VP8 as its mandatory video codec. // Since video encoding is best done using native code, // reference the video codec at the application-level. // This is required when using a WebRTC video stream. eyeball.messenger.webrtc.VideoStream.registerCodec("VP8", new EmptyFunction<VideoCodec>() {
  21. 21. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. public VideoCodec invoke() { return new Vp8Codec(); } }, true); // Android requires an Activity context to create Views. // Since the default WebRTC video providers must create // Views, supply a reference to the current activity. // This is required when using a WebRTC video stream on // the Android platform. eyeball.messenger.webrtc.DefaultProviders.setAndroidActivity(this); // WebRTC audio and video streams require us to first get access to // the local media (microphone, camera, or both). eyeball.messenger.webrtc.UserMedia.getMedia(new GetMediaArgs(true, true) {{ setVideoWidth(320); // optional setVideoHeight(240); // optional setVideoFrameRate(30); // optional setOnFailure(new SingleAction<GetMediaFailureArgs>() { public void invoke(GetMediaFailureArgs e) { alert("Could not get media. %s", e.getException().getMessage()); } });
  22. 22. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. setOnSuccess(new SingleAction<GetMediaSuccessArgs>() { public void invoke(GetMediaSuccessArgs e) { // We have successfully acquired access to the local // audio/video device! Grab a reference to the media. // Internally, it maintains access to the local audio // and video feeds coming from the device hardware. LocalMediaStream localMedia = e.getLocalStream(); // This is our local video control, a Java Component // or Android View. It is constantly updated with our // live video feed since we requested video above. Add // it directly to the UI or use the Messenger layout manager, // which we do below. Object localVideoControl = e.getLocalVideoControl(); // Create an Messenger layout manager, which makes the task // of arranging video controls easy. Give it a reference // to a Java Container that can be filled with video feeds. // For Android users, the WebRTC extension includes // AndroidLayoutManager, which accepts an Android ViewGroup. LayoutManager layoutManager = new eyeball.messenger.webrtc.LayoutManager(container); // Position and display the local video control on-screen // by passing it to the layout manager created above.
  23. 23. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. layoutManager.setLocalVideoControl(localVideoControl); // Create a WebRTC audio stream description (requires a // reference to the local audio feed). AudioStream audioStream = new eyeball.messenger.webrtc.AudioStream(localMedia); // Create a WebRTC video stream description (requires a // reference to the local video feed). Whenever a P2P link // initializes using this description, position and display // the remote video control on-screen by passing it to the // layout manager created above. Whenever a P2P link goes // down, remove it. VideoStream videoStream = new eyeball.messenger.webrtc.VideoStream(localMedia); videoStream.addOnLinkInit(new SingleAction<StreamLinkInitArgs>() { public void invoke(final StreamLinkInitArgs e) { Object remoteVideoControl = LinkExtensions.getRemoteVideoControl(e.getLink()); layoutManager.addRemoteVideoControl(e.getPeerId(), remoteVideoControl); } }); videoStream.addOnLinkDown(new SingleAction<StreamLinkDownArgs>() { public void invoke(final StreamLinkDownArgs e) {
  24. 24. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. layoutManager.removeRemoteVideoControl(e.getPeerId()); } }); // TO BE CONTINUED... Part 2: The Conference Now that we have some stream descriptions, we are ready to create a conference - the hub that manages your P2P links. Conferences require the address/port of a STUN/TURN server. STUN provides a way for peers to discover their public IP addresses, and TURN provides a way to relay data through highly restrictive firewalls. Messenger comes bundled with a STUN/TURN server implementation, so you can run your own if you have access to a computer that is directly exposed to the public Internet. Alternatively, there are several public STUN servers that are free-to-use. Free TURN servers are more difficult to find, since relaying with TURN can consume significant server bandwidth. // CONTINUED FROM ABOVE... // Create a conference using our stream descriptions. Replace // 127.0.0.1 : 3478 with your STUN/TURN (Messenger) server address. Conference conference = new eyeball.messenger.Conference("127.0.0.1", 3478, new Stream[] { audioStream, videoStream, dataChannelStream }); // Supply TURN relay credentials in case we are behind a // highly restrictive firewall. These credentials will be // verified by the TURN server. conference.setRelayUsername("test"); conference.setRelayPassword("pa55w0rd!");
  25. 25. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. // Add a few event handlers to the conference so we can see // when a new P2P link is created or changes state. conference.addOnLinkInit(new SingleAction<LinkInitArgs>() { public void invoke(LinkInitArgs e) { writeLine("Link to peer initializing..."); } }); conference.addOnLinkUp(new SingleAction<LinkUpArgs>() { public void invoke(LinkUpArgs e) { writeLine("Link to peer is UP."); } }); conference.addOnLinkDown(new SingleAction<LinkDownArgs>() { public void invoke(LinkDownArgs e) { writeLine("Link to peer is DOWN. " + e.getException().getMessage()); } }); // TO BE CONTINUED... Part 3: Signalling
  26. 26. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. With our conference in hand, we are almost done! Just one more thing to set up. Before we can create a P2P link, the peers have to exchange some information - specifically descriptions of the streams (called the offer/answer) and some local network addresses (called candidates). Messengergenerates this information automatically, but you are responsible for distributing it to the peer as quickly as possible. This is called "signalling". We're going to use WebSync here, but other popular options include SIP and XMPP - any real-time messaging system will do. We use WebSync since it uses HTTP (WebSockets/long-polling) and therefore has no issues with firewalls or connecting from JavaScript-based web applications. // CONTINUED FROM ABOVE... // Create a WebSync client and establish a persistent // connection to the server. Replace localhost with your // WebSync server address. Client client = new eyeball.websync.Client("http://localhost/websync.ashx"); client.connect(new ConnectArgs() {{ setOnFailure(new SingleAction() { public void invoke(ConnectFailureArgs e) { alert("Could not connect to WebSync. " + e.getException().getMessage()); } }); }});
  27. 27. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. // Add a couple event handlers to the conference to send // generated offers/answers and candidates to a peer. // The peer ID is something we define later. In this case, // it represents the remote WebSync client ID. WebSync's // "notify" method is used to send data to a specific client. conference.addOnLinkOfferAnswer(new SingleAction<LinkOfferAnswerArgs>() { public void invoke(LinkOfferAnswerArgs e) { try { client.notify(new NotifyArgs(new Guid(e.getPeerId()), e.getOfferAnswer().toJson(), "offeranswer")); } catch (Exception ex) { ex.printStackTrace(); } } }); conference.addOnLinkCandidate(new SingleAction<LinkCandidateArgs>() { public void invoke(LinkCandidateArgs e) { try { client.notify(new NotifyArgs(new Guid(e.getPeerId()), e.getCandidate().toJson(), "candidate"));
  28. 28. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. } catch (Exception ex) { ex.printStackTrace(); } } }); // Add an event handler to the WebSync client to receive // incoming offers/answers and candidates from a peer. // Call the "receiveOfferAnswer" or "receiveCandidate" // method to pass the information to the conference. client.addOnNotify(new SingleAction<NotifyReceiveArgs>() { public void invoke(NotifyReceiveArgs e) { try { String peerId = e.getNotifyingClient().getClientId().toString(); Object peerState = e.getNotifyingClient().getBoundRecords(); if (e.getTag().equals("offeranswer")) { conference.receiveOfferAnswer(OfferAnswer.fromJson(e.getDataJson()), peerId, peerState); } else if (e.getTag().equals("candidate"))
  29. 29. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. { conference.receiveCandidate(Candidate.fromJson(e.getDataJson()), peerId); } } catch (Exception ex) { ex.printStackTrace(); } } }); // TO BE CONTINUED... Part 4: Kicking It Off Everything is set now. We just need to start the process by having one of the peers call Conference.Link. That peer becomes the "controlling agent". The other peer becomes the "controlled agent" and creates a P2P link automatically when it receives the controlling agent's offer. Here's a detailed breakdown of what happens:  Peer A: The conference will raise the OnLinkInit event.  Peer A: Messenger will generate an offer and raise the OnOfferAnswer event.  Peer A: Our event handler will use WebSync's Notify method to send that offer to the peer.  Peer B: Our event handler will receive that offer and call Conference.ReceiveOfferAnswer.  Peer B: The conference will raise the OnLinkInit event.  Peer B: Messenger will generate an answer and raise the OnOfferAnswer event.  Peer B: Our event handler will use WebSync's Notify method to send that answer to the peer.  Peer A: Our event handler will receive that answer and call Conference.ReceiveOfferAnswer.  Peer A/B: Messenger will generate candidates and raise the OnCandidate event.  Peer A/B: Our event handler will use WebSync's Notify method to send those candidates to the peer.  Peer A/B: Our event handler will receive those candidates and call Conference.ReceiveCandidate.  Peer A/B: Messenger will successfully establish a P2P link.  Peer A/B: The conference will raise the OnLinkUp event.
  30. 30. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. If the P2P link could not be established, the conference will raise the OnLinkDown event. The event arguments will include an Exception property with details. The most common reason for a link failure is a misconfiguration on the server. (Server address or port is wrong, server firewall is blocking UDP traffic, server is on the same network as one of the clients, server is not publicly accessible, etc.) If you are having trouble setting up your Messenger server, try using our public server, located at messenger.eyeball.com : 3478. A common scenario when using a pub-sub framework like WebSync is to create a P2P link whenever two peers join the same channel (and destroy that link when one peer leaves). // CONTINUED FROM ABOVE... // Subscribe to a WebSync channel. When another client joins the same // channel, create a P2P link. When a client leaves, destroy it. SubscribeArgs subscribeArgs = new SubscribeArgs("/mychannel") {{ setOnFailure(new SingleAction<SubscribeFailureArgs>() { public void invoke(SubscribeFailureArgs e) { alert("Could not subscribe to " + e.getChannel() + ". " + e.getException().getMessage()); } }); setOnReceive(new SingleAction<SubscribeReceiveArgs>() { public void invoke(SubscribeReceiveArgs e) { } }); }};
  31. 31. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. SubscribeArgsExtensions.setOnClientSubscribe(subscribeArgs, new SingleAction<ClientSubscribeArgs>() { public void invoke(ClientSubscribeArgs e) { try { String peerId = e.getSubscribedClient().getClientId().toString(); Object peerState = e.getSubscribedClient().getBoundRecords(); conference.link(peerId, peerState); } catch (Exception ex) { ex.printStackTrace(); } } }); SubscribeArgsExtensions.setOnClientUnsubscribe(subscribeArgs, new SingleAction<ClientUnsubscribeArgs>() { public void invoke(ClientUnsubscribeArgs e) { try { String peerId = e.getUnsubscribedClient().getClientId().toString(); conference.unlink(peerId); }
  32. 32. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. catch (Exception ex) { ex.printStackTrace(); } } }); client.subscribe(subscribeArgs); } }); }}); Note that when calling Conference.Link, you must supply a unique peer ID. This string can be anything that uniquely identifies the remote peer. This should be the remote WebSync client ID when using WebSync, the remote SIP client ID when using SIP, the remote XMPP client ID when using XMPP... You get the idea. That's it! Load 'er up and watch the magic! Creating a Conference: JavaScript JavaScript
  33. 33. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. The JavaScript SDK provides a complete WebRTC-based peer-to-peer (P2P) streaming solution, including support for audio, video, and data channels. Native WebRTC implementations, such as those found in Chrome and Firefox, are used if present. For all other cases, the SDK automatically and seamlessly falls back to an embedded Java applet that provides a complete WebRTC stack. This graceful degradation is completely transparent to you as the developer. There are two ways to use the JavaScript client. The first is completely object-oriented, like Java or .NET. var getMediaArgs = new eyeball.messenger.webrtc.getMediaArgs(); getMediaArgs.setAudio(true); getMediaArgs.setVideo(true); getMediaArgs.setOnSuccess(function(e) { ... }); eyeball.messenger.webrtc.userMedia.getMedia(getMediaArgs); The second uses a more JavaScript-style syntax to accomplish the same thing. eyeball.messenger.webrtc.userMedia.getMedia({ audio: true, video: true, onSuccess: function(e) { ... } }); Properties on the getMedia config object (such as "audio", "video", etc) are mapped automatically to the appropriate "setter" methods (such as "setAudio", "setVideo", etc). Just make sure the property names match up to a setter method, and the rest is automatic! To get started, you'll need to add some scripts from the SDK to your page:
  34. 34. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. <!-- supporting code --> <script type="text/javascript" src="eyeball.js"></script> <!-- Messenger core --> <script type="text/javascript" src="eyeball.messenger.js"></script> <!-- WebRTC stack --> <script type="text/javascript" src="eyeball.messenger.webrtc.js"></script> Now let's write some code! (See JavaScript.Client.WebRTC and JavaScript.Client.DataChannels in the SDK Examples folder.) Part 1: Stream Descriptions The first step is to describe the streams we want to use. Stream descriptions are used to define the types of media we are going to be sending back and forth between peers in a conference. The WebRTC extension allows us to create audio, video, and data-channel streams. For this example, we'll create all three. // For backwards-compability with browsers that do not yet support // WebRTC, provide a reference to eyeball.messenger.webrtc.applet.jar, a // Java applet that provides a full WebRTC stack through the exact // same JavaScript API you use for modern browsers. You can set this // for all browsers - only the ones that need it will use it. eyeball.messenger.webrtc.setApplet({ path: './eyeball.messenger.webrtc.applet.jar', name: 'Messenger WebRTC for JavaScript' });
  35. 35. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. // TO BE CONTINUED... Part 1a: WebRTC Data-Channel Stream Description WebRTC data-channel stream descriptions are easy. Describe one or more data channels (name + message handler) and pass them into the stream description. // Create a data channel description, including a handler for // processing received messages. var dataChannelInfo = new eyeball.messenger.webrtc.dataChannelInfo({ label: 'mydatachannel', onReceive: function(e) { alert(e.getData()); } }); // Create a data channel stream description using our data channel. var dataChannelStream = new eyeball.messenger.webrtc.dataChannelStream([ dataChannelInfo ]); // To send data-channel stream data, use conference.sendData // once you have created a conference (see Part 2): // <code> // conference.sendData({ channelInfo: dataChannelInfo, data: 'Hello, world!' }); // </code>
  36. 36. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. // TO BE CONTINUED... Part 1b: WebRTC Audio/Video Stream Descriptions WebRTC audio and video stream descriptions require a reference to the local media (microphone, camera, or both), obtained by calling userMedia.getMedia. // WebRTC audio and video streams require us to first get access to // the local media (microphone, camera, or both). eyeball.messenger.webrtc.userMedia.getMedia({ audio: true, // required if you want to send audio video: true, // required if you want to send video videoWidth: 320, // optional videoHeight: 240, // optional videoFrameRate: 30, // optional onFailure: function(e) { alert('Could not get media. ' + e.getException().message); }, onSuccess: function(e) { // We have successfully acquired access to the local // audio/video device! Grab a reference to the media. // Internally, it maintains access to the local audio // and video feeds coming from the device hardware. var localMedia = e.getLocalStream(); // This is our local video control, a simple DOM element.
  37. 37. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. // It is constantly updated with our live video feed // since we requested video above. To add it to the DOM, // either call XXX.appendChild(localVideoControl) or use // the Messenger layout manager, which we do below. var localVideoControl = e.getLocalVideoControl(); // Create an Messenger layout manager, which makes the task // of arranging video controls easy. Give it a reference // to a DOM element that can be filled with video feeds. var layoutManager = new eyeball.messenger.webrtc.layoutManager(document.getElementById('container')); // Position and display the local video control on-screen // by passing it to the layout manager created above. layoutManager.setLocalVideoControl(localVideoControl); // Create an audio stream description (requires a reference // to the local audio feed). var audioStream = new eyeball.messenger.webrtc.audioStream(localMedia); // Create a video stream description (requires a reference // to the local video feed). Whenever a P2P link initializes // using this description, position and display the remote // video control on-screen by passing it to the layout manager // created above. Whenever a P2P link goes down, remove it. var videoStream = new eyeball.messenger.webrtc.videoStream(localMedia); videoStream.addOnLinkInit(function(e)
  38. 38. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. { var remoteVideoControl = e.getLink().getRemoteVideoControl(); layoutManager.addRemoteVideoControl(e.getPeerId(), remoteVideoControl); }); videoStream.addOnLinkDown(function(e) { layoutManager.removeRemoteVideoControl(e.getPeerId()); }); // TO BE CONTINUED... Part 2: The Conference Now that we have some stream descriptions, we are ready to create a conference - the hub that manages your P2P links. Conferences require the address/port of a STUN/TURN server. STUN provides a way for peers to discover their public IP addresses, and TURN provides a way to relay data through highly restrictive firewalls. Messenger comes bundled with a STUN/TURN server implementation, so you can run your own if you have access to a computer that is directly exposed to the public Internet. Alternatively, there are several public STUN servers that are free-to-use. Free TURN servers are more difficult to find, since relaying with TURN can consume significant server bandwidth. // CONTINUED FROM ABOVE... // Create a conference using our stream descriptions. Replace // 127.0.0.1 : 3478 with your STUN/TURN (Messenger) server address. var conference = new eyeball.messenger.conference('127.0.0.1', 3478, [ audioStream, videoStream, dataChannelStream ]); // Supply TURN relay credentials in case we are behind a
  39. 39. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. // highly restrictive firewall. These credentials will be // verified by the TURN server. conference.setRelayUsername('test'); conference.setRelayPassword('pa55w0rd!'); // Add a few event handlers to the conference so we can see // when a new P2P link is created or changes state. conference.addOnLinkInit(function(e) { if (console && console.log) console.log('Link to peer initializing...'); }); conference.addOnLinkUp(function(e) { if (console && console.log) console.log('Link to peer is UP.'); }); conference.addOnLinkDown(function(e) { if (console && console.log) console.log('Link to peer is DOWN. ' + e.getException().message); }); // TO BE CONTINUED... Part 3: Signalling With our conference in hand, we are almost done! Just one more thing to set up. Before we can create a P2P link, the peers have to exchange some information - specifically descriptions of the streams (called the offer/answer) and some local network addresses (called candidates).
  40. 40. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. Messengergenerates this information automatically, but you are responsible for distributing it to the peer as quickly as possible. This is called "signalling". We're going to use WebSync here, but other popular options include SIP and XMPP - any real-time messaging system will do. We use WebSync since it uses HTTP (WebSockets/long-polling) and therefore has no issues with firewalls or connecting from JavaScript-based web applications. // CONTINUED FROM ABOVE... // Create a WebSync client and establish a persistent // connection to the server. Replace localhost with your // WebSync server address. var client = new eyeball.websync.client('http://localhost/websync.ashx'); client.connect({ onFailure: function(e) { alert('Could not connect to WebSync. ' + e.getException().message); } }); // Add a couple event handlers to the conference to send // generated offers/answers and candidates to a peer. // The peer ID is something we define later. In this case, // it represents the remote WebSync client ID. WebSync's // "notify" method is used to send data to a specific client. conference.addOnLinkOfferAnswer(function(e) { client.notify({
  41. 41. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. clientId: e.getPeerId(), dataJson: e.getOfferAnswer().toJson(), tag: 'offeranswer' }); }); conference.addOnLinkCandidate(function(e) { client.notify({ clientId: e.getPeerId(), dataJson: e.getCandidate().toJson(), tag: 'candidate' }); }); // Add an event handler to the WebSync client to receive // incoming offers/answers and candidates from a peer. // Call the "receiveOfferAnswer" or "receiveCandidate" // method to pass the information to the conference. client.addOnNotify(function(e) { var peerId = e.getNotifyingClient().getClientId().toString(); var peerState = e.getNotifyingClient().getBoundRecords(); // optional if (e.getTag() == 'offeranswer') { conference.receiveOfferAnswer(eyeball.messenger.offerAnswer.fromJson(e.getDataJson()), peerId, peerState); }
  42. 42. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. else if (e.getTag() == 'candidate') { conference.receiveCandidate(eyeball.messenger.candidate.fromJson(e.getDataJson()), peerId); } }); // TO BE CONTINUED... Part 4: Kicking It Off Everything is set now. We just need to start the process by having one of the peers call Conference.Link. That peer becomes the "controlling agent". The other peer becomes the "controlled agent" and creates a P2P link automatically when it receives the controlling agent's offer. Here's a detailed breakdown of what happens:  Peer A: The conference will raise the OnLinkInit event.  Peer A: Messenger will generate an offer and raise the OnOfferAnswer event.  Peer A: Our event handler will use WebSync's Notify method to send that offer to the peer.  Peer B: Our event handler will receive that offer and call Conference.ReceiveOfferAnswer.  Peer B: The conference will raise the OnLinkInit event.  Peer B: Messenger will generate an answer and raise the OnOfferAnswer event.  Peer B: Our event handler will use WebSync's Notify method to send that answer to the peer.  Peer A: Our event handler will receive that answer and call Conference.ReceiveOfferAnswer.  Peer A/B: Messenger will generate candidates and raise the OnCandidate event.  Peer A/B: Our event handler will use WebSync's Notify method to send those candidates to the peer.  Peer A/B: Our event handler will receive those candidates and call Conference.ReceiveCandidate.  Peer A/B: Messenger will successfully establish a P2P link.  Peer A/B: The conference will raise the OnLinkUp event. If the P2P link could not be established, the conference will raise the OnLinkDown event. The event arguments will include an Exception property with details. The most common reason for a link failure is a misconfiguration on the server. (Server address or port is wrong, server firewall is blocking UDP traffic, server is on the same network as one of the clients, server is not publicly accessible, etc.) If you are having trouble setting up your Messenger server, try using our public server, located at messenger.eyeball.com : 3478.
  43. 43. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. A common scenario when using a pub-sub framework like WebSync is to create a P2P link whenever two peers join the same channel (and destroy that link when one peer leaves). // CONTINUED FROM ABOVE... // Subscribe to a WebSync channel. When another client joins the same // channel, create a P2P link. When a client leaves, destroy it. client.subscribe({ channel: '/mychannel', onFailure: function(e) { alert('Could not subscribe to ' + e.getChannel() + '. ' + e.getException().message); }, onReceive: function(e) { }, onClientSubscribe: function(e) { var peerId = e.getSubscribedClient().getClientId().toString(); var peerState = e.getSubscribedClient().getBoundRecords(); conference.link(peerId, peerState); }, onClientUnsubscribe: function(e) { var peerId = e.getUnsubscribedClient().getClientId().toString(); conference.unlink(peerId); } }); }
  44. 44. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. }); Note that when calling Conference.Link, you must supply a unique peer ID. This string can be anything that uniquely identifies the remote peer. This should be the remote WebSync client ID when using WebSync, the remote SIP client ID when using SIP, the remote XMPP client ID when using XMPP... You get the idea. That's it! Load 'er up and watch the magic! Creating a Conference: Xamarin.Android Please read peer-to-peer (P2P) streaming solution, including a partial WebRTC stack for rapidly creating data-channel streams that can be consumed natively in modern web browsers. To get started, you'll need to add some assemblies from the SDK to your project references (Xamarin Studio, Visual Studio, MonoDevelop):  Eyeball.dll (supporting code)  Eyeball.Messenger.dll (Messenger core)  Eyeball.Messenger.WebRTC.dll (WebRTC stack, optional) Now let's write some code! (See Xamarin.Android.Client and Xamarin.Android.Client.DataChannels in the SDK Examples folder.) Part 1: Stream Descriptions
  45. 45. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. The first step is to describe the streams we want to use. Stream descriptions are used to define the types of media we are going to be sending back and forth between peers in a conference. Messenger allows us to createcustom streams with any number of possible data formats. The WebRTC extension allows us to create audio, video, and data-channel streams. You can mix and match custom and WebRTC streams, but only WebRTC streams can be consumed natively by web browsers. Part 1a: Custom Stream Descriptions Custom stream descriptions require a "stream type" and one or more "stream formats". The stream type is used in the SDP offer/answer to help differentiate between multiple streams (audio, video, application/binary, etc.). The stream format is used in the SDP offer/answer to help differentiate between multiple data formats supported for a given stream (i.e. G.711, G.722, Opus, etc. for audio). // Create a custom stream description that indicates text // data will be sent using a format labelled "utf8". var customStream = new Eyeball.Messenger.Stream(StreamType.Application, new StreamFormat("utf8")); customStream.OnLinkReceiveRTP += (e) { Alert(Encoding.UTF8.GetString(e.Packet.Payload)); }; // To send custom stream data, use Conference.SendRTP // once you have created a conference (see Part 2): // <code> // conference.SendRTP(customStream.Format, new RTPPacket(Encoding.UTF8.GetBytes("Hello, world!"))); // </code> // TO BE CONTINUED... Part 1b: WebRTC Data-Channel Stream Description
  46. 46. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. WebRTC data-channel streams are easy. Describe one or more data channels (name + message handler) and pass them into the stream description. // Create a WebRTC data channel description, including a // handler for processing received messages. var dataChannelInfo = new Eyeball.Messenger.WebRTC.DataChannelInfo("mydatachannel") { OnReceive = (e) => { Alert(e.Data); } }; // Create a WebRTC data channel stream description using // our data channel. var dataChannelStream = new Eyeball.Messenger.WebRTC.DataChannelStream(dataChannelInfo); // To send data-channel stream data, use Conference.SendData // once you have created a conference (see Part 2): // <code> // conference.SendData(dataChannelInfo, "Hello, world!"); // </code> // TO BE CONTINUED... Part 2: The Conference
  47. 47. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. Now that we have some stream descriptions, we are ready to create a conference - the hub that manages your P2P links. Conferences require the address/port of a STUN/TURN server. STUN provides a way for peers to discover their public IP addresses, and TURN provides a way to relay data through highly restrictive firewalls. Messenger comes bundled with a STUN/TURN server implementation, so you can run your own if you have access to a computer that is directly exposed to the public Internet. Alternatively, there are several public STUN servers that are free-to-use. Free TURN servers are more difficult to find, since relaying with TURN can consume significant server bandwidth. // CONTINUED FROM ABOVE... // Create a conference using our stream descriptions. Replace // 127.0.0.1 : 3478 with your STUN/TURN (Messenger) server address. var conference = new Eyeball.Messenger.Conference("127.0.0.1", 3478, dataChannelStream); // Supply TURN relay credentials in case we are behind a // highly restrictive firewall. These credentials will be // verified by the TURN server. conference.RelayUsername = "test"; conference.RelayPassword = "pa55w0rd!"; // Add a few event handlers to the conference so we can see // when a new P2P link is created or changes state. conference.OnLinkInit += (e) => { Console.WriteLine("Link to peer initializing..."); };
  48. 48. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. conference.OnLinkUp += (e) => { Console.WriteLine("Link to peer is UP."); }; conference.OnLinkDown += (e) => { Console.WriteLine("Link to peer is DOWN. " + e.Exception.Message); }; // TO BE CONTINUED... Part 3: Signalling With our conference in hand, we are almost done! Just one more thing to set up. Before we can create a P2P link, the peers have to exchange some information - specifically descriptions of the streams (called the offer/answer) and some local network addresses (called candidates). Messengergenerates this information automatically, but you are responsible for distributing it to the peer as quickly as possible. This is called "signalling". We're going to use WebSync here, but other popular options include SIP and XMPP - any real-time messaging system will do. We use WebSync since it uses HTTP (WebSockets/long-polling) and therefore has no issues with firewalls or connecting from JavaScript-based web applications. // CONTINUED FROM ABOVE... // Create a WebSync client and establish a persistent // connection to the server. Replace localhost with your // WebSync server address. var client = new Eyeball.WebSync.Client("http://localhost/websync.ashx"); client.Connect(new ConnectArgs()
  49. 49. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. { OnFailure = (e) => { Alert("Could not connect to WebSync. " + e.Exception.Message); }); }); // Add a couple event handlers to the conference to send // generated offers/answers and candidates to a peer. // The peer ID is something we define later. In this case, // it represents the remote WebSync client ID. WebSync's // "notify" method is used to send data to a specific client. conference.OnLinkOfferAnswer += (e) => { client.Notify(new NotifyArgs(new Guid(e.PeerId), e.OfferAnswer.ToJson(), "offeranswer")); }; conference.OnLinkCandidate += (e) => { client.Notify(new NotifyArgs(new Guid(e.PeerId), e.Candidate.ToJson(), "candidate")); }; // Add an event handler to the WebSync client to receive // incoming offers/answers and candidates from a peer. // Call the "ReceiveOfferAnswer" or "ReceiveCandidate" // method to pass the information to the conference. client.OnNotify += (e) =>
  50. 50. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. { var peerId = e.NotifyingClient.ClientId.ToString(); var peerState = e.NotifyingClient.BoundRecords; if (e.Tag == "offeranswer") { conference.ReceiveOfferAnswer(OfferAnswer.FromJson(e.DataJson), peerId, peerState); } else if (e.Tag == "candidate") { conference.ReceiveCandidate(Candidate.FromJson(e.DataJson), peerId); } }; // TO BE CONTINUED... Part 4: Kicking It Off Everything is set now. We just need to start the process by having one of the peers call Conference.Link. That peer becomes the "controlling agent". The other peer becomes the "controlled agent" and creates a P2P link automatically when it receives the controlling agent's offer. Here's a detailed breakdown of what happens:  Peer A: The conference will raise the OnLinkInit event.  Peer A: Messenger will generate an offer and raise the OnOfferAnswer event.  Peer A: Our event handler will use WebSync's Notify method to send that offer to the peer.  Peer B: Our event handler will receive that offer and call Conference.ReceiveOfferAnswer.  Peer B: The conference will raise the OnLinkInit event.  Peer B: Messenger will generate an answer and raise the OnOfferAnswer event.  Peer B: Our event handler will use WebSync's Notify method to send that answer to the peer.  Peer A: Our event handler will receive that answer and call Conference.ReceiveOfferAnswer.  Peer A/B: Messenger will generate candidates and raise the OnCandidate event.  Peer A/B: Our event handler will use WebSync's Notify method to send those candidates to the peer.  Peer A/B: Our event handler will receive those candidates and call Conference.ReceiveCandidate.  Peer A/B: Messenger will successfully establish a P2P link.
  51. 51. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved.  Peer A/B: The conference will raise the OnLinkUp event. If the P2P link could not be established, the conference will raise the OnLinkDown event. The event arguments will include an Exception property with details. The most common reason for a link failure is a misconfiguration on the server. (Server address or port is wrong, server firewall is blocking UDP traffic, server is on the same network as one of the clients, server is not publicly accessible, etc.) If you are having trouble setting up your Messenger server, try using our public server, located at messenger.eyeball.com : 3478. A common scenario when using a pub-sub framework like WebSync is to create a P2P link whenever two peers join the same channel (and destroy that link when one peer leaves). // CONTINUED FROM ABOVE... // Subscribe to a WebSync channel. When another client joins the same // channel, create a P2P link. When a client leaves, destroy it. client.Subscribe(new SubscribeArgs("/mychannel") { OnFailure = (e) => { Alert("Could not subscribe to " + e.Channel + ". " + e.Exception.Message); }, OnReceive = (e) => { } } .SetOnClientSubscribe((e) => { var peerId = e.SubscribedClient.ClientId.ToString(); var peerState = e.SubscribedClient.BoundRecords; conference.Link(peerId, peerState);
  52. 52. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. }) .SetOnClientUnsubscribe((e) => { var peerId = e.UnsubscribedClient.ClientId.ToString(); conference.Unlink(peerId); })); Note that when calling Conference.Link, you must supply a unique peer ID. This string can be anything that uniquely identifies the remote peer. This should be the remote WebSync client ID when using WebSync, the remote SIP client ID when using SIP, the remote XMPP client ID when using XMPP... You get the idea. That's it! Load 'er up and watch the magic! Creating a Conference: Xamarin.iOS The Xamarin.iOS SDK provides a complete peer-to-peer (P2P) streaming solution, including a partial WebRTC stack for rapidly creating data-channel streams that can be consumed natively in modern web browsers. To get started, you'll need to add some assemblies from the SDK to your project references (Xamarin Studio, Visual Studio, MonoDevelop):  Eyeball.dll (supporting code)  Eyeball.Messenger.dll (Messenger core)  Eyeball.Messenger.WebRTC.dll (WebRTC stack, optional) Now let's write some code! (See Xamarin.iOS.Client and Xamarin.iOS.Client.DataChannels in the SDK Examples folder.) Part 1: Stream Descriptions The first step is to describe the streams we want to use. Stream descriptions are used to define the types of media we are going to be sending back and forth between peers in a conference. Messenger allows us to createcustom streams with any number of possible data formats. The WebRTC extension allows us to
  53. 53. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. create audio, video, and data-channel streams. You can mix and match custom and WebRTC streams, but only WebRTC streams can be consumed natively by web browsers. Part 1a: Custom Stream Descriptions Custom stream descriptions require a "stream type" and one or more "stream formats". The stream type is used in the SDP offer/answer to help differentiate between multiple streams (audio, video, application/binary, etc.). The stream format is used in the SDP offer/answer to help differentiate between multiple data formats supported for a given stream (i.e. G.711, G.722, Opus, etc. for audio). // Create a custom stream description that indicates text // data will be sent using a format labelled "utf8". var customStream = new Eyeball.Messenger.Stream(StreamType.Application, new StreamFormat("utf8")); customStream.OnLinkReceiveRTP += (e) { Alert(Encoding.UTF8.GetString(e.Packet.Payload)); }; // To send custom stream data, use Conference.SendRTP // once you have created a conference (see Part 2): // <code> // conference.SendRTP(customStream.Format, new RTPPacket(Encoding.UTF8.GetBytes("Hello, world!"))); // </code> // TO BE CONTINUED... Part 1b: WebRTC Data-Channel Stream Description WebRTC data-channel streams are easy. Describe one or more data channels (name + message handler) and pass them into the stream description.
  54. 54. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. // Create a WebRTC data channel description, including a // handler for processing received messages. var dataChannelInfo = new Eyeball.Messenger.WebRTC.DataChannelInfo("mydatachannel") { OnReceive = (e) => { Alert(e.Data); } }; // Create a WebRTC data channel stream description using // our data channel. var dataChannelStream = new Eyeball.Messenger.WebRTC.DataChannelStream(dataChannelInfo); // To send data-channel stream data, use Conference.SendData // once you have created a conference (see Part 2): // <code> // conference.SendData(dataChannelInfo, "Hello, world!"); // </code> // TO BE CONTINUED... Part 2: The Conference Now that we have some stream descriptions, we are ready to create a conference - the hub that manages your P2P links.
  55. 55. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. Conferences require the address/port of a STUN/TURN server. STUN provides a way for peers to discover their public IP addresses, and TURN provides a way to relay data through highly restrictive firewalls. Messenger comes bundled with a STUN/TURN server implementation, so you can run your own if you have access to a computer that is directly exposed to the public Internet. Alternatively, there are several public STUN servers that are free-to-use. Free TURN servers are more difficult to find, since relaying with TURN can consume significant server bandwidth. // CONTINUED FROM ABOVE... // Create a conference using our stream descriptions. Replace // 127.0.0.1 : 3478 with your STUN/TURN (Messenger) server address. var conference = new Eyeball.Messenger.Conference("127.0.0.1", 3478, dataChannelStream); // Supply TURN relay credentials in case we are behind a // highly restrictive firewall. These credentials will be // verified by the TURN server. conference.RelayUsername = "test"; conference.RelayPassword = "pa55w0rd!"; // Add a few event handlers to the conference so we can see // when a new P2P link is created or changes state. conference.OnLinkInit += (e) => { Console.WriteLine("Link to peer initializing..."); }; conference.OnLinkUp += (e) => { Console.WriteLine("Link to peer is UP.");
  56. 56. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. }; conference.OnLinkDown += (e) => { Console.WriteLine("Link to peer is DOWN. " + e.Exception.Message); }; // TO BE CONTINUED... Part 3: Signalling With our conference in hand, we are almost done! Just one more thing to set up. Before we can create a P2P link, the peers have to exchange some information - specifically descriptions of the streams (called the offer/answer) and some local network addresses (called candidates). Messengergenerates this information automatically, but you are responsible for distributing it to the peer as quickly as possible. This is called "signalling". We're going to use WebSync here, but other popular options include SIP and XMPP - any real-time messaging system will do. We use WebSync since it uses HTTP (WebSockets/long-polling) and therefore has no issues with firewalls or connecting from JavaScript-based web applications. // CONTINUED FROM ABOVE... // Create a WebSync client and establish a persistent // connection to the server. Replace localhost with your // WebSync server address. var client = new Eyeball.WebSync.Client("http://localhost/websync.ashx"); client.Connect(new ConnectArgs() { OnFailure = (e) => {
  57. 57. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. Alert("Could not connect to WebSync. " + e.Exception.Message); }); }); // Add a couple event handlers to the conference to send // generated offers/answers and candidates to a peer. // The peer ID is something we define later. In this case, // it represents the remote WebSync client ID. WebSync's // "notify" method is used to send data to a specific client. conference.OnLinkOfferAnswer += (e) => { client.Notify(new NotifyArgs(new Guid(e.PeerId), e.OfferAnswer.ToJson(), "offeranswer")); }; conference.OnLinkCandidate += (e) => { client.Notify(new NotifyArgs(new Guid(e.PeerId), e.Candidate.ToJson(), "candidate")); }; // Add an event handler to the WebSync client to receive // incoming offers/answers and candidates from a peer. // Call the "ReceiveOfferAnswer" or "ReceiveCandidate" // method to pass the information to the conference. client.OnNotify += (e) => { var peerId = e.NotifyingClient.ClientId.ToString(); var peerState = e.NotifyingClient.BoundRecords;
  58. 58. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. if (e.Tag == "offeranswer") { conference.ReceiveOfferAnswer(OfferAnswer.FromJson(e.DataJson), peerId, peerState); } else if (e.Tag == "candidate") { conference.ReceiveCandidate(Candidate.FromJson(e.DataJson), peerId); } }; // TO BE CONTINUED... Part 4: Kicking It Off Everything is set now. We just need to start the process by having one of the peers call Conference.Link. That peer becomes the "controlling agent". The other peer becomes the "controlled agent" and creates a P2P link automatically when it receives the controlling agent's offer. Here's a detailed breakdown of what happens:  Peer A: The conference will raise the OnLinkInit event.  Peer A: Messenger will generate an offer and raise the OnOfferAnswer event.  Peer A: Our event handler will use WebSync's Notify method to send that offer to the peer.  Peer B: Our event handler will receive that offer and call Conference.ReceiveOfferAnswer.  Peer B: The conference will raise the OnLinkInit event.  Peer B: Messenger will generate an answer and raise the OnOfferAnswer event.  Peer B: Our event handler will use WebSync's Notify method to send that answer to the peer.  Peer A: Our event handler will receive that answer and call Conference.ReceiveOfferAnswer.  Peer A/B: Messenger will generate candidates and raise the OnCandidate event.  Peer A/B: Our event handler will use WebSync's Notify method to send those candidates to the peer.  Peer A/B: Our event handler will receive those candidates and call Conference.ReceiveCandidate.  Peer A/B: Messenger will successfully establish a P2P link.  Peer A/B: The conference will raise the OnLinkUp event. If the P2P link could not be established, the conference will raise the OnLinkDown event. The event arguments will include an Exception property with details. The most common reason for a link failure is
  59. 59. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. a misconfiguration on the server. (Server address or port is wrong, server firewall is blocking UDP traffic, server is on the same network as one of the clients, server is not publicly accessible, etc.) If you are having trouble setting up your Messenger server, try using our public server, located at messenger.eyeball.com : 3478. A common scenario when using a pub-sub framework like WebSync is to create a P2P link whenever two peers join the same channel (and destroy that link when one peer leaves). // CONTINUED FROM ABOVE... // Subscribe to a WebSync channel. When another client joins the same // channel, create a P2P link. When a client leaves, destroy it. client.Subscribe(new SubscribeArgs("/mychannel") { OnFailure = (e) => { Alert("Could not subscribe to " + e.Channel + ". " + e.Exception.Message); }, OnReceive = (e) => { } } .SetOnClientSubscribe((e) => { var peerId = e.SubscribedClient.ClientId.ToString(); var peerState = e.SubscribedClient.BoundRecords; conference.Link(peerId, peerState); }) .SetOnClientUnsubscribe((e) => {
  60. 60. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. var peerId = e.UnsubscribedClient.ClientId.ToString(); conference.Unlink(peerId); })); Note that when calling Conference.Link, you must supply a unique peer ID. This string can be anything that uniquely identifies the remote peer. This should be the remote WebSync client ID when using WebSync, the remote SIP client ID when using SIP, the remote XMPP client ID when using XMPP... You get the idea. That's it! Load 'er up and watch the magic!
  61. 61. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. Creating a Conference: .NET, .NET Compact, and Mono The .NET SDK provides a complete peer-to-peer (P2P) streaming solution, including a complete WebRTC stack for rapidly creating audio, video, and data-channel streams that can be consumed natively in modern web browsers. To get started, you'll need to add some assemblies from the SDK to your project references (Visual Studio, MonoDevelop):  Eyeball.dll (supporting code)  Eyeball.Messenger.dll (Messenger core)  Eyeball.Messenger.WebRTC.dll (WebRTC stack, optional) Now let's write some code! (See NET.Client, NET.Client.WebRTC, and NET.Client.DataChannels in the SDK Examples folder.) Part 1: Stream Descriptions The first step is to describe the streams we want to use. Stream descriptions are used to define the types of media we are going to be sending back and forth between peers in a conference. Messenger allows us to createcustom streams with any number of possible data formats. The WebRTC extension allows us to create audio, video, and data-channel streams. You can mix and match custom and WebRTC streams, but only WebRTC streams can be consumed natively by web browsers. Part 1a: Custom Stream Descriptions
  62. 62. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. Custom stream descriptions require a "stream type" and one or more "stream formats". The stream type is used in the SDP offer/answer to help differentiate between multiple streams (audio, video, application/binary, etc.). The stream format is used in the SDP offer/answer to help differentiate between multiple data formats supported for a given stream (i.e. G.711, G.722, Opus, etc. for audio). // Create a custom stream description that indicates text // data will be sent using a format labelled "utf8". var customStream = new Eyeball.Messenger.Stream(StreamType.Application, new StreamFormat("utf8")); customStream.OnLinkReceiveRTP += (e) { Alert(Encoding.UTF8.GetString(e.Packet.Payload)); }; // To send custom stream data, use Conference.SendRTP // once you have created a conference (see Part 2): // <code> // conference.SendRTP(customStream.Format, new RTPPacket(Encoding.UTF8.GetBytes("Hello, world!"))); // </code> // TO BE CONTINUED... Part 1b: WebRTC Data-Channel Stream Description WebRTC data-channel streams are easy. Describe one or more data channels (name + message handler) and pass them into the stream description. // Create a WebRTC data channel description, including a // handler for processing received messages. var dataChannelInfo = new Eyeball.Messenger.WebRTC.DataChannelInfo("mydatachannel")
  63. 63. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. { OnReceive = (e) => { Alert(e.Data); } }; // Create a WebRTC data channel stream description using // our data channel. var dataChannelStream = new Eyeball.Messenger.WebRTC.DataChannelStream(dataChannelInfo); // To send data-channel stream data, use Conference.SendData // once you have created a conference (see Part 2): // <code> // conference.SendData(dataChannelInfo, "Hello, world!"); // </code> // TO BE CONTINUED... Part 1c: WebRTC Audio/Video Stream Descriptions WebRTC audio and video stream descriptions require a reference to the local media (microphone, camera, or both), obtained by calling UserMedia.GetMedia. // WebRTC has chosen VP8 as its mandatory video codec. // Since video encoding is best done using native code, // reference the video codec at the application-level. // This is required when using a WebRTC video stream.
  64. 64. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. Eyeball.Messenger.WebRTC.VideoStream.RegisterCodec("VP8", () => { return new Win.VP8.Codec(); }, true); // WebRTC audio and video streams require us to first get access to // the local media (microphone, camera, or both). Eyeball.Messenger.WebRTC.UserMedia.GetMedia(new GetMediaArgs(true, true) { // Specify an audio capture provider, a video // capture provider, an audio render provider, // and a video render provider. The Messenger // SDK comes bundled with video support for // WinForms and WPF, and uses NAudio for audio // support. For lower latency audio, use ASIO. AudioCaptureProvider = new NAudioCaptureProvider(), VideoCaptureProvider = new AForgeVideoCaptureProvider(), CreateAudioRenderProvider = () => { return new NAudioRenderProvider(); }, CreateVideoRenderProvider = () => { return new PictureBoxVideoRenderProvider(); }, VideoWidth = 320, // optional VideoHeight = 240, // optional
  65. 65. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. VideoFrameRate = 30, // optional OnFailure = (e) => { Alert("Could not get media. {0}", e.Exception.Message); }, OnSuccess = (ea) => { // We have successfully acquired access to the local // audio/video device! Grab a reference to the media. // Internally, it maintains access to the local audio // and video feeds coming from the device hardware. var localMedia = ea.LocalStream; // This is our local video control, a WinForms Control or // WPF FrameworkElement. It is constantly updated with // our live video feed since we requested video above. // Add it directly to the UI or use the Messenger layout // manager, which we do below. var localVideoControl = ea.LocalVideoControl; // Create an Messenger layout manager, which makes the task // of arranging video controls easy. Give it a reference // to a WinForms control that can be filled with video feeds. // For WPF users, the WebRTC extension includes // WpfLayoutManager, which accepts a Canvas. var layoutManager = new Eyeball.Messenger.WebRTC.LayoutManager(container);
  66. 66. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. // Position and display the local video control on-screen // by passing it to the layout manager created above. layoutManager.SetLocalVideoControl(localVideoControl); // Create a WebRTC audio stream description (requires a // reference to the local audio feed). var audioStream = new Eyeball.Messenger.WebRTC.AudioStream(localMedia); // Create a WebRTC video stream description (requires a // reference to the local video feed). Whenever a P2P link // initializes using this description, position and display // the remote video control on-screen by passing it to the // layout manager created above. Whenever a P2P link goes // down, remove it. var videoStream = new Eyeball.Messenger.WebRTC.VideoStream(localMedia); videoStream.OnLinkInit += (e) => { var remoteVideoControl = e.Link.GetRemoteVideoControl(); layoutManager.AddRemoteVideoControl(e.PeerId, remoteVideoControl); }; videoStream.OnLinkDown += (e) => { layoutManager.RemoveRemoteVideoControl(e.PeerId); }; // TO BE CONTINUED...
  67. 67. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. Part 2: The Conference Now that we have some stream descriptions, we are ready to create a conference - the hub that manages your P2P links. Conferences require the address/port of a STUN/TURN server. STUN provides a way for peers to discover their public IP addresses, and TURN provides a way to relay data through highly restrictive firewalls. Messenger comes bundled with a STUN/TURN server implementation, so you can run your own if you have access to a computer that is directly exposed to the public Internet. Alternatively, there are several public STUN servers that are free-to-use. Free TURN servers are more difficult to find, since relaying with TURN can consume significant server bandwidth. // CONTINUED FROM ABOVE... // Create a conference using our stream descriptions. Replace // 127.0.0.1 : 3478 with your STUN/TURN (Messenger) server address. var conference = new Eyeball.Messenger.Conference("127.0.0.1", 3478, new[] { audioStream, videoStream, dataChannelStream }); // Supply TURN relay credentials in case we are behind a // highly restrictive firewall. These credentials will be // verified by the TURN server. conference.RelayUsername = "test"; conference.RelayPassword = "pa55w0rd!"; // Add a few event handlers to the conference so we can see // when a new P2P link is created or changes state. conference.OnLinkInit += (e) => { Console.WriteLine("Link to peer initializing...");
  68. 68. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. }; conference.OnLinkUp += (e) => { Console.WriteLine("Link to peer is UP."); }; conference.OnLinkDown += (e) => { Console.WriteLine("Link to peer is DOWN. " + e.Exception.Message); }; // TO BE CONTINUED... Part 3: Signalling With our conference in hand, we are almost done! Just one more thing to set up. Before we can create a P2P link, the peers have to exchange some information - specifically descriptions of the streams (called the offer/answer) and some local network addresses (called candidates). Messengergenerates this information automatically, but you are responsible for distributing it to the peer as quickly as possible. This is called "signalling". We're going to use WebSync here, but other popular options include SIP and XMPP - any real-time messaging system will do. We use WebSync since it uses HTTP (WebSockets/long-polling) and therefore has no issues with firewalls or connecting from JavaScript-based web applications. // CONTINUED FROM ABOVE... // Create a WebSync client and establish a persistent // connection to the server. Replace localhost with your // WebSync server address. var client = new Eyeball.WebSync.Client("http://localhost/websync.ashx");
  69. 69. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. client.Connect(new ConnectArgs() { OnFailure = (e) => { Alert("Could not connect to WebSync. " + e.Exception.Message); }); }); // Add a couple event handlers to the conference to send // generated offers/answers and candidates to a peer. // The peer ID is something we define later. In this case, // it represents the remote WebSync client ID. WebSync's // "notify" method is used to send data to a specific client. conference.OnLinkOfferAnswer += (e) => { client.Notify(new NotifyArgs(new Guid(e.PeerId), e.OfferAnswer.ToJson(), "offeranswer")); }; conference.OnLinkCandidate += (e) => { client.Notify(new NotifyArgs(new Guid(e.PeerId), e.Candidate.ToJson(), "candidate")); }; // Add an event handler to the WebSync client to receive // incoming offers/answers and candidates from a peer. // Call the "ReceiveOfferAnswer" or "ReceiveCandidate" // method to pass the information to the conference.
  70. 70. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. client.OnNotify += (e) => { var peerId = e.NotifyingClient.ClientId.ToString(); var peerState = e.NotifyingClient.BoundRecords; if (e.Tag == "offeranswer") { conference.ReceiveOfferAnswer(OfferAnswer.FromJson(e.DataJson), peerId, peerState); } else if (e.Tag == "candidate") { conference.ReceiveCandidate(Candidate.FromJson(e.DataJson), peerId); } }; // TO BE CONTINUED... Part 4: Kicking It Off Everything is set now. We just need to start the process by having one of the peers call Conference.Link. That peer becomes the "controlling agent". The other peer becomes the "controlled agent" and creates a P2P link automatically when it receives the controlling agent's offer. Here's a detailed breakdown of what happens:  Peer A: The conference will raise the OnLinkInit event.  Peer A: Messenger will generate an offer and raise the OnOfferAnswer event.  Peer A: Our event handler will use WebSync's Notify method to send that offer to the peer.  Peer B: Our event handler will receive that offer and call Conference.ReceiveOfferAnswer.  Peer B: The conference will raise the OnLinkInit event.  Peer B: Messenger will generate an answer and raise the OnOfferAnswer event.  Peer B: Our event handler will use WebSync's Notify method to send that answer to the peer.  Peer A: Our event handler will receive that answer and call Conference.ReceiveOfferAnswer.  Peer A/B: Messenger will generate candidates and raise the OnCandidate event.  Peer A/B: Our event handler will use WebSync's Notify method to send those candidates to the peer.
  71. 71. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved.  Peer A/B: Our event handler will receive those candidates and call Conference.ReceiveCandidate.  Peer A/B: Messenger will successfully establish a P2P link.  Peer A/B: The conference will raise the OnLinkUp event. If the P2P link could not be established, the conference will raise the OnLinkDown event. The event arguments will include an Exception property with details. The most common reason for a link failure is a misconfiguration on the server. (Server address or port is wrong, server firewall is blocking UDP traffic, server is on the same network as one of the clients, server is not publicly accessible, etc.) If you are having trouble setting up your Messenger server, try using our public server, located at messenger.eyeball.com : 3478. A common scenario when using a pub-sub framework like WebSync is to create a P2P link whenever two peers join the same channel (and destroy that link when one peer leaves). // CONTINUED FROM ABOVE... // Subscribe to a WebSync channel. When another client joins the same // channel, create a P2P link. When a client leaves, destroy it. client.Subscribe(new SubscribeArgs("/mychannel") { OnFailure = (e) => { Alert("Could not subscribe to " + e.Channel + ". " + e.Exception.Message); }, OnReceive = (e) => { } } .SetOnClientSubscribe((e) => { var peerId = e.SubscribedClient.ClientId.ToString(); var peerState = e.SubscribedClient.BoundRecords;
  72. 72. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. conference.Link(peerId, peerState); }) .SetOnClientUnsubscribe((e) => { var peerId = e.UnsubscribedClient.ClientId.ToString(); conference.Unlink(peerId); })); } }); Note that when calling Conference.Link, you must supply a unique peer ID. This string can be anything that uniquely identifies the remote peer. This should be the remote WebSync client ID when using WebSync, the remote SIP client ID when using SIP, the remote XMPP client ID when using XMPP... You get the idea. That's it! Load 'er up and watch the magic!
  73. 73. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. Creating a Conference: Windows Phone The Windows Phone SDK provides a complete peer-to-peer (P2P) streaming solution, including a partial WebRTC stack for rapidly creating data-channel streams that can be consumed natively in modern web browsers. To get started, you'll need to add some assemblies from the SDK to your project references (Visual Studio):  Eyeball.dll (supporting code)  Eyeball.Messenger.dll (Messenger core)  Eyeball.Messenger.WebRTC.dll (WebRTC stack, optional) Now let's write some code! (See WindowsPhone.Client and WindowsPhone.Client.DataChannels in the SDK Examples folder.) Part 1: Stream Descriptions The first step is to describe the streams we want to use. Stream descriptions are used to define the types of media we are going to be sending back and forth between peers in a conference. Messenger allows us to createcustom streams with any number of possible data formats. The WebRTC extension allows us to create audio, video, and data-channel streams. You can mix and match custom and WebRTC streams, but only WebRTC streams can be consumed natively by web browsers. Part 1a: Custom Stream Descriptions Custom stream descriptions require a "stream type" and one or more "stream formats". The stream type is used in the SDP offer/answer to help differentiate between multiple streams (audio, video,
  74. 74. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. application/binary, etc.). The stream format is used in the SDP offer/answer to help differentiate between multiple data formats supported for a given stream (i.e. G.711, G.722, Opus, etc. for audio). // Create a custom stream description that indicates text // data will be sent using a format labelled "utf8". var customStream = new Eyeball.Messenger.Stream(StreamType.Application, new StreamFormat("utf8")); customStream.OnLinkReceiveRTP += (e) { Alert(Encoding.UTF8.GetString(e.Packet.Payload)); }; // To send custom stream data, use Conference.SendRTP // once you have created a conference (see Part 2): // <code> // conference.SendRTP(customStream.Format, new RTPPacket(Encoding.UTF8.GetBytes("Hello, world!"))); // </code> // TO BE CONTINUED... Part 1b: WebRTC Data-Channel Stream Description WebRTC data-channel streams are easy. Describe one or more data channels (name + message handler) and pass them into the stream description. // Create a WebRTC data channel description, including a // handler for processing received messages. var dataChannelInfo = new Eyeball.Messenger.WebRTC.DataChannelInfo("mydatachannel") { OnReceive = (e) =>
  75. 75. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. { Alert(e.Data); } }; // Create a WebRTC data channel stream description using // our data channel. var dataChannelStream = new Eyeball.Messenger.WebRTC.DataChannelStream(dataChannelInfo); // To send data-channel stream data, use Conference.SendData // once you have created a conference (see Part 2): // <code> // conference.SendData(dataChannelInfo, "Hello, world!"); // </code> // TO BE CONTINUED... Part 2: The Conference Now that we have some stream descriptions, we are ready to create a conference - the hub that manages your P2P links. Conferences require the address/port of a STUN/TURN server. STUN provides a way for peers to discover their public IP addresses, and TURN provides a way to relay data through highly restrictive firewalls. Messenger comes bundled with a STUN/TURN server implementation, so you can run your own if you have access to a computer that is directly exposed to the public Internet. Alternatively, there are several public STUN servers that are free-to-use. Free TURN servers are more difficult to find, since relaying with TURN can consume significant server bandwidth. // CONTINUED FROM ABOVE...
  76. 76. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. // Create a conference using our stream descriptions. Replace // 127.0.0.1 : 3478 with your STUN/TURN (Messenger) server address. var conference = new Eyeball.Messenger.Conference("127.0.0.1", 3478, dataChannelStream); // Supply TURN relay credentials in case we are behind a // highly restrictive firewall. These credentials will be // verified by the TURN server. conference.RelayUsername = "test"; conference.RelayPassword = "pa55w0rd!"; // Add a few event handlers to the conference so we can see // when a new P2P link is created or changes state. conference.OnLinkInit += (e) => { Console.WriteLine("Link to peer initializing..."); }; conference.OnLinkUp += (e) => { Console.WriteLine("Link to peer is UP."); }; conference.OnLinkDown += (e) => { Console.WriteLine("Link to peer is DOWN. " + e.Exception.Message); };
  77. 77. Copyright © 2002-2015 Eyeball Networks Inc. Patented and patents pending. All rights reserved. // TO BE CONTINUED... Part 3: Signalling With our conference in hand, we are almost done! Just one more thing to set up. Before we can create a P2P link, the peers have to exchange some information - specifically descriptions of the streams (called the offer/answer) and some local network addresses (called candidates). Messengergenerates this information automatically, but you are responsible for distributing it to the peer as quickly as possible. This is called "signalling". We're going to use WebSync here, but other popular options include SIP and XMPP - any real-time messaging system will do. We use WebSync since it uses HTTP (WebSockets/long-polling) and therefore has no issues with firewalls or connecting from JavaScript-based web applications. // CONTINUED FROM ABOVE... // Create a WebSync client and establish a persistent // connection to the server. Replace localhost with your // WebSync server address. var client = new Eyeball.WebSync.Client("http://localhost/websync.ashx"); client.Connect(new ConnectArgs() { OnFailure = (e) => { Alert("Could not connect to WebSync. " + e.Exception.Message); }); }); // Add a couple event handlers to the conference to send // generated offers/answers and candidates to a peer.

×