Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Getting a live_transcript_of_your_call_using_the_ari

852 views

Published on

Stream audio out of Asterisk using wazo res_ari_stream of Asterisk ARI

Published in: Technology
  • Login to see the comments

  • Be the first to like this

Getting a live_transcript_of_your_call_using_the_ari

  1. 1. GETTING A LIVE TRANSCRIPT OF YOUR CALL PASCAL CADOTTE MICHAUD | CORE DEVELOPER ASTRICON, ATLANTA, OCT. 2019
  2. 2. WHO AM I? OPEN SOURCE ENTHUSIAST PASCAL CADOTTE MICHAUD, CORE DEVELOPER @ WAZO 10+ YEARS IN DEVELOPMENT PROGRAMMING SOFTWARES USING ASTERISK FOR ALMOST 10 YEARS USING ● AMI ● DIALPLAN ● AGI ● ARI AND STASIS APPLICATIONS @PCADOTTE @PC-M
  3. 3. WAZO PLATFORM PROJECT MISSION LEVERAGE THE BEST OPEN SOURCE COMPONENTS TO TRANSFORM THE WAY WE BUILD IP TELECOM INFRASTRUCTURE AND CREATE COMMUNICATION SERVICES
  4. 4. WHY WOULD YOU WANT TO STREAM THE AUDIO OUT OF ASTERISK
  5. 5. INTERESTING USES CASES WHY WOULD I STREAM SOUNDS OUT OF ASTERISK? ● ADDING SUBTITLES TO A CALL ● TRANSLATIONS ● DETECTING CUSTOMER FEELING ● CALL PRIORITIZATION ● FRAUD DETECTION ● INDEXATION ● CALL ROUTING IVR http://www.wazo-platform.org/blog/solving-the-emergency-call-prioritization-issue-with-programmable-telecom
  6. 6. TIMELINE AUGUST 2018 AUDIO SOCKET OCTOBER 2018 ASTRIDEVCON JANUARY 2019 WAZO PARIS HACKATHON RES_ARI_STREAM JULY 2019 ASTRICON CFP OCTOBER 2019 ASTERISK 16.6.0 WITH EXTERNALMEDIA OUR JOURNEY
  7. 7. AUDIO SOCKET BY SEÁN C. MCCORD “AUDIOSOCKET IS A SIMPLE TCP-BASED PROTOCOL FOR SENDING AND RECEIVING REALTIME AUDIO STREAMS. THERE EXISTS A PROTOCOL DEFINITION, A GO LIBRARY, AND AN ASTERISK APPLICATION.” VERY SIMPLE PROTOCOL USING AN ASTERISK DIALPLAN APPLICATION. SEE SEÁN’S PRESENTATION TOMORROW AT 10:30 FOR MORE INTERESTING DETAILS. https://wiki.asterisk.org/wiki/display/AST/AudioSocket
  8. 8. A FIRST WORKING IMPLEMENTATION RES_ARI_STREAM RES_ARI_STREAM IS A MODULE DEVELOPED AS A HACKATHON PROJECT TO GET A GOOD FEEL OF THE PROBLEM. IT USES THE ARI WEBSOCKET WITH THE SUB-PROTOCOL “STREAM-CHANNEL”.
  9. 9. HOW DOES IT WORK RES_ARI_STREAM 1. ON LOAD ADDS A SUB-PROTOCOL TO THE WEBSOCKET 2. ON SUBSCRIPTION A HOOK ON RECEIVED FRAMES IS ADDED 3. ON VOICE FRAME THE VOICE IS TRANSCODED TO SLIN16 4. THEN THE FRAMES ARE WRITTEN TO THE SUBSCRIBED WEBSOCKET https://github.com/sboily/wazo-hackathon-asterisk-stream-module
  10. 10. A VERY BASIC EXAMPLE WRITING THE AUDIO FRAMES TO A FILE import websocket import sys with open("out.wav", "wb") as f: ws = websocket.WebSocketApp( "ws://localhost:5039/ws", on_message=lambda ws, msg: out.write(msg), on_error=lambda ws, error: print(error), on_close=lambda ws: print(“### closed ###”), subprotocols=["stream-channel"], header=[“Channel-ID: “ + sys.argv[1]], ) ws.run_forever()
  11. 11. A VERY SIMPLE EXAMPLE USAGE RECORDING THE AUDIO TO A FILE LISTENING TO YOUR RECORDING play -t raw -b 16 --endian little -c 1 -e signed-integer -r 16000 out.wav python3 record-channel-id.py “<my-channel>”
  12. 12. ASTERISK IMPLEMENTATION IN ASTERISK 16.6.0 POST /channels/externalmedia 1. CREATE AN EXTERNAL MEDIA CHANNEL WITH AN HOST AND PORT 2. ASTERISK WILL SEND RTP TO THE HOST AND PORT 3. ADD THE EXTERNAL MEDIA CHANNEL TO THE BRIDGE YOU WISH TO LISTEN 4. READ THE RTP FROM THE SOCKET THE UNICASTRTP_LOCAL_ADDRESS AND UNICASTRTP_LOCAL_PORT CHANNEL VARIABLES CONTAIN WHAT YOU NEED TO SEND MEDIA BACK TO ASTERISK. https://wiki.asterisk.org/wiki/display/AST/External+Media+and+ARI https://github.com/asterisk/asterisk-external-media
  13. 13. DIFFERENCES BETWEEN THE 3 OPTIONS RES_ARI_STREAM ARI EXTERNALMEDIA AUDIOSOCKET MEDIA DIRECTION UNI-DIRECTIONAL BI-DIRECTIONAL BI-DIRECTIONAL USAGE HTTP ENDPOINT WITH A SPECIFIC SUB-PROTOCOL FOR THE WEBSOCKET LIMITED TO CHANNELS IN STASIS DIALPLAN APPLICATION AND CHANNEL DRIVER ARI RESOURCE IN A GERRIT PULL REQUEST FORMAT SLIN16 IS RECEIVED RTP WITH SPECIFIED CODEC IS RECEIVED VERY SIMPLE PROTOCOL WITH SLIN16 PAYLOAD CONNECTIVITY CONNECTS TO A CHANNEL CONNECTS TO A BRIDGE CONNECTS TO A CHANNEL OR A BRIDGE TRANSPORT WEBSOCKET UDP TCP
  14. 14. RES_ARI_STREAM DEMO
  15. 15. DEMO OF THE RES_ARI_STREAM MODULE https://github.com/pc-m/transcript-demo/blob/master/transcript_demo/wazo.py https://github.com/pc-m/transcript-demo/blob/master/transcript_demo/transcription.py INCOMING CALL AN INCOMING CALL IS ANSWERED BY ALICE THIS IS A NORMAL CALL, NO STASIS APPLICATION IS REQUIRED HERE TRANSCRIPT SCRIPT TRANSCRIPTION SCRIPT IS LAUNCHED ON THE INCOMING CALL THE SCRIPT CONNECTS TO THE ASTERISK WEBSOCKET AND REGISTERS ITSELF TO THE INCOMING CALL CHANNEL ID GOOGLE SPEECH API TRANSCRIPTION GOOGLE SPEECH API IS USED FOR THE TRANSCRIPTION USER HTTP SERVER THE TRANSLATED TEXT IS SERVED TO THE USER
  16. 16. ARI EXTERNAL MEDIA DEMO
  17. 17. DEMO OF THE ASTERISK ARI VERSION https://github.com/pc-m/transcript-demo/blob/master/transcript_demo/ari_stasis.py https://github.com/pc-m/transcript-demo/blob/master/transcript_demo/ari_server.py https://github.com/pc-m/transcript-demo/blob/master/transcript_demo/transcription.py STASIS THE CALL ENTERS STASIS THE STASIS APPLICATION ADDS THE CALL TO ITS BRIDGE UDP SERVER THE SERVER RECEIVES THE RTP FROM ASTERISK THE SERVER CREATES THE EXTERNAL MEDIA CHANNELS WHICH GET ADDED TO THE STASIS BRIDGE AND RECEIVES THE RTP GOOGLE SPEECH API TRANSCRIPTION GOOGLE SPEECH API IS USED FOR THE TRANSCRIPTION USER HTTP SERVER THE TRANSLATED TEXT IS SERVED TO THE USER
  18. 18. WAZO PLATFORM: COLLABORATION OPEN COLLABORATION & OPEN DEVELOPMENT OPEN SOURCE PROJECT https://www.wazo-platform.org/ PR WORKFLOW @ GITHUB https://github.com/wazo-platform/ MATTERMOST CHAN https://mm.wazo.community/wazo-platform/ SERVICE CODE UNDER GLPV3+ LICENCE LIBRARY CODE UNDER MIT LICENCE
  19. 19. LET’S CONNECT BUILD GREAT SOLUTIONS WITH WAZO PLATFORM BOOTH WAZO W/ THE TEAM : FRED, SYLVAIN, PASCAL & JÉROME TO KNOW MORE : - JEROME’S TALK : WEDNESDAY - 1:30PM MORE INFO : - BLOG POST - OFFICIAL DEMO @GITHUB
  20. 20. QUESTIONS

×