Integrating SMS Messaging with your Rails Application Blythe Dunham [email_address] spongecell .com / snowgiraffe .com

Blythe Dunham

[email_address]

spongecell .com / snowgiraffe .com

Why SMS? Oldest most reliable means to communicate with mobile users Cell phone agnostic Everybody is doing it Great way to receive emergency messages at the bar SERVER IS DOWN SPARKS ARE UP

Oldest most reliable means to communicate with mobile users

Cell phone agnostic

Everybody is doing it

Great way to receive emergency messages at the bar

Popular Applications to Send SMS Standard Email yournumber@gatewayaddr Example:2065551234@txt.att.net Find your provider gateway Instant Message AOL (US only) Skype (small fee) Other Teleflip RIP remindblast .com - Send SMS reminders to be awesome or do laundry Facebook SMS app Cell providers web page (ex. http://www.t-mobile.com )

Standard Email

yournumber@gatewayaddr Example:2065551234@txt.att.net

Find your provider gateway

Instant Message

AOL (US only)

Skype (small fee)

Other

Teleflip RIP

remindblast .com - Send SMS reminders to be awesome or do laundry

Facebook SMS app

Cell providers web page (ex. http://www.t-mobile.com )

Overview: One Way Messaging Mobile Terminated (MT) or Outbound Black Box of Your application & gateway & other Party at A1A Beach front Ave Mobile Terminated (MT)

Mobile Terminated (MT) or Outbound

Overview: Two Way Messaging Black Box of Your application & gateway & other Mobile Originated (MO) or Inbound ROCK MO Respond with "ROCK" for Free party tix MT

Simplest approach: SMS as email

Pros and Cons of sending SMS through email The Good Free to application (end-user with phone might pay) Easy to integrate. Only email setup required. Excellent for use with known numbers Emergency Messages with plugins like Exception Notification

The Good

Free to application (end-user with phone might pay)

Easy to integrate. Only email setup required.

Excellent for use with known numbers

Emergency Messages with plugins like Exception Notification

Pros and Cons of sending SMS through email The Tricky User carrier must be known additional user input storage considerations No delivery confirmation Supported by most US but not all cell phone carriers No two way messaging

The Tricky

User carrier must be known

additional user input

storage considerations

No delivery confirmation

Supported by most US but not all cell phone carriers

No two way messaging

Giant Black Box Explained: SMSCs, SMS Gateways, Service Providers Before exploring the other available SMS options, lets look at how SMS works

Before exploring the other available SMS options, lets look at how SMS works

SMS Centers (SMSC) SMSC SMSC SMSC

GSM Modem Super App GSM Modem (Cell Phone) SMSC AT Commands

GSM Modem APIs and Applications Kannel in C (open source) supports an API RubyGSM - Written by Unicef for RapidSMS to inexpensively send text messages

Kannel in C (open source) supports an API

RubyGSM - Written by Unicef for RapidSMS to inexpensively send text messages

GSM Modem LoDown Advantages Cheap Can use almost anywhere Two way messaging support Disadvantages Difficult to setup Slow Doesn’t scale (only 6 to 10 messages per second) Must connect modem to a server

Advantages

Cheap

Can use almost anywhere

Two way messaging support

Disadvantages

Difficult to setup

Slow

Doesn’t scale (only 6 to 10 messages per second)

Must connect modem to a server

Talk directly to all SMS Centers SMSC SMSC SMSC Awesome App

Mission Impossible: Coding for each SMSC Large number of SMS Centers Each use different and proprietary communication protocols Don’t do this unless you are me!

Large number of SMS Centers

Each use different and proprietary communication protocols

SMS Gateway translates SMSC protocols SMSC SMSC SMSC Rockstar App SMS Gateway

SMS Gateways Free Open Source Gateways Kannel - written in C supports many SMSCs and also acts as a WAP gateway Must setup/configure on a server yourself

Free Open Source Gateways

Kannel - written in C supports many SMSCs and also acts as a WAP gateway

Must setup/configure on a server yourself

Pay the middleman! Charge for sending messages ~ 5 cents per message Bulk discounts Provide convenient APIs Most provide two way messaging Can help obtain short code

Charge for sending messages

~ 5 cents per message

Bulk discounts

Provide convenient APIs

Most provide two way messaging

Can help obtain short code

SMS Gateway: Sending Outbound (MT) Typical approach for Outbound messages is simple: Create an account Buy credits (or messages) Start sending messages through the API

Typical approach for Outbound messages is simple:

Create an account

Buy credits (or messages)

Start sending messages through the API

Ruby Outbound (MT) message with HTTPS http = Net::HTTP.new(CLICKATELL_BASE_URL, 443 ) http.use_ssl = true http.enable_post_connection_check = true http.start {response = http.request_get(url_string)}

http = Net::HTTP.new(CLICKATELL_BASE_URL, 443 )

http.use_ssl = true

http.enable_post_connection_check = true

http.start {response = http.request_get(url_string)}

Choosing an API (if you have a choice) HTTP/S (GET & POST) By far the easiest to implement Great for sending < 10k messages a month SMPP Best for bulk messages (over 10k per month) Leaves Connection open STMP - send through mail Other options (not so hot) FTP - Bulk option for legacy systems XML (use HTTP if available) SOAP (use HTTP if available) Com Object (Seriously? What wah?)

HTTP/S (GET & POST)

By far the easiest to implement

Great for sending < 10k messages a month

SMPP

Best for bulk messages (over 10k per month)

Leaves Connection open

STMP - send through mail

Other options (not so hot)

FTP - Bulk option for legacy systems

XML (use HTTP if available)

SOAP (use HTTP if available)

Com Object (Seriously? What wah?)

SMS Gateway: Two way messaging and MOs Typically enable Outbound messaging Obtain or rent a short code Respond to Inbound MOs Use a provider campaign management tool ($$$) Parse yourself and write some regexp! /^OPT(s*[-_]?s*)OUT|STOP|END|QUIT| UNSUBSCRIBE|EXIT|HALT| FUCK(s*[-_]?s*)OFF|REMOVE|CANCEL)/i

Typically enable Outbound messaging

Obtain or rent a short code

Respond to Inbound MOs

Use a provider campaign management tool ($$$)

Parse yourself and write some regexp!

Two way messaging: Pull Setup a cron task to pull new messages every few seconds.

Setup a cron task to pull new messages every few seconds.

Two way messaging: Push Provide a callback URL that service provider invokes for each new message

Provide a callback URL that service provider invokes for each new message

Do I need a short code? If your awesome app is: A giant spam machine Implementing two way messaging (Recommended) Sending > 5000 messages per month

If your awesome app is:

A giant spam machine

Implementing two way messaging

(Recommended) Sending > 5000 messages per month

Shared (Rented) Versus Dedicated Short Codes Shared short code shared with others requires keyword like &quot;RUBY&quot; Dedicated short code all to yourself Takes around 2 months to obtain $1000 for a selected code or $500 for a random code per month

Shared short code

shared with others

requires keyword like &quot;RUBY&quot;

Dedicated short code

all to yourself

Takes around 2 months to obtain

$1000 for a selected code or $500 for a random code per month

Chewing on a Cactus Choose a short code (selected or random) Apply by sending registration data for 3, 6 or 12 months Receive approval email from CSCA Pay all up front $500 per month for Random, $1000/mo for Selected Apply for approval from each wireless service provider Sign a contract with each provider Implement and Test each connection with each provider Sms gateway service providers(connection aggregators) will help with this process or you can go directly through the CSCA in the US

Choose a short code (selected or random)

Apply by sending registration data for 3, 6 or 12 months

Receive approval email from CSCA

Pay all up front $500 per month for Random, $1000/mo for Selected

Apply for approval from each wireless service provider

Sign a contract with each provider

Implement and Test each connection with each provider

Opt out concerns Handled by most providers Implement yourself Tracking Opting out different senders

Handled by most providers

Implement yourself

Tracking

Opting out different senders

Choosing a Service Provider Cost: how many sms messages will you send per month? Credit vs. sms Hidden costs Monthly fee Coverage: does the provider cover all needed areas? Implementation Time: is the API you want to use easy to integrate? Opt-out: is it handled? Two way messaging: is it supported? Do you need it? Help with short code cactus eating experience?

Cost: how many sms messages will you send per month?

Credit vs. sms

Hidden costs

Monthly fee

Coverage: does the provider cover all needed areas?

Implementation Time: is the API you want to use easy to integrate?

Opt-out: is it handled?

Two way messaging: is it supported? Do you need it?

Help with short code cactus eating experience?

Developer Integration Concerns

Save message information Do you need to save a history or audit trail for reporting? Lots of messages to send? Can't send in real time (before controller action times out?)

Do you need to save a history or audit trail for reporting?

Lots of messages to send?

Can't send in real time (before controller action times out?)

Inside the SMS ORM: Check out SMS ON RAILS Draft Message Delivery By Date PhoneNumber Number WhiteList? DoNotSend? Outbound Status Time Sent Service Provider UID PhoneCarrier Name Email gateway Sender/User Username Password Favorite beer

Outbound Status: Produce and Consume Web action creates an Outbound record with status NOT_PROCESSED Cron task processes messages with status NOT_PROCESSED and marks them SUCCESS or FAIL Controller Action Producer Cron/Background Task Consumer NOT_PROCESSED SUCCESS

Web action creates an Outbound record with status NOT_PROCESSED

Cron task processes messages with status NOT_PROCESSED and marks them SUCCESS or FAIL

Locking and Delivering: Outbound Status No transactions; no take backs When delivering a message, try locking Outbound ORM object NOT_PROCESSED PROCESSING SUCCESS FAILED

No transactions; no take backs

When delivering a message, try locking Outbound ORM object

Bulk Message Delivery Use batch API call to send multiple phone numbers the same msg For huge deliveries, use SMPP to maintain an open connection Set delivery date and send to provider in advance Lock multiple records with your database (MySQL FOR UPDATE) Use Multiple servers

Use batch API call to send multiple phone numbers the same msg

For huge deliveries, use SMPP to maintain an open connection

Set delivery date and send to provider in advance

Lock multiple records with your database (MySQL FOR UPDATE)

Use Multiple servers

Bulk Message Delivery Tips Reduce contention for competing servers storing data Assign a group of records (ID mod X srvrs) to each server (non dedicated servers) put tasks on staggered timers (ex. every 2 min) that execute for a shorter duration(ex. 45 seconds) Identify fatal errors and stop the task Fatal, stop task: Ex. No response from service provider Single message error: Mark failed and continue processing Cron to Recover messages in 'PROCESSING' state with updated_at field > duration of consumer task (ex. 10 min) Delivery status background task to request delivery information Set priority on messages

Reduce contention for competing servers storing data

Assign a group of records (ID mod X srvrs) to each server

(non dedicated servers) put tasks on staggered timers (ex. every 2 min) that execute for a shorter duration(ex. 45 seconds)

Identify fatal errors and stop the task

Fatal, stop task: Ex. No response from service provider

Single message error: Mark failed and continue processing

Cron to Recover messages in 'PROCESSING' state with updated_at field > duration of consumer task (ex. 10 min)

Delivery status background task to request delivery information

Set priority on messages

Sanitize Phone Numbers PhoneNumber .create!(:number => '(206)-555-1212 ' ) PhoneNumber .find_by_number( '12065551212' ) == nil Your database without data integrity

PhoneNumber .create!(:number => '(206)-555-1212 ' )

PhoneNumber .find_by_number( '12065551212' ) == nil

Sanitize Phone Numbers Sanitize all phone numbers before search or insert or update Store with country code (for USA auto add a 1 for 10 digit numbers not inputted with a + or 0) My database runs on flowers

Sanitize all phone numbers before search or insert or update

Store with country code (for USA auto add a 1 for 10 digit numbers not inputted with a + or 0)

Outbound Attributes: White List and Opt Out Save safe numbers on a white list and enable checking to prevent accidents (test mode) restrict access Store Opt-Out Information Prevent paying for a message to a recipient provider opted out Can make a distinction between opt out, opt in, and bounce, abuse smsi

Save safe numbers on a white list and enable checking to

prevent accidents (test mode)

restrict access

Store Opt-Out Information

Prevent paying for a message to a recipient provider opted out

Can make a distinction between opt out, opt in, and bounce, abuse

SMS Gateway Service Providers Short List of Providers http://www. redoxygen .com http://www. redrabbitsms .com/ http:// clickatell .com http:// quios .com http://messagepub.com/ http://www.openmarMeket.com/products-and-services/message-and-content-delivery.html http://www.mobilemarketing.net http://www.textmarks.com/ http://www.mozes.com/ Long list and comparison table of providers: http://www. developershome .com/ sms / smsGatewayProvComp .asp

Short List of Providers

http://www. redoxygen .com

http://www. redrabbitsms .com/

http:// clickatell .com

http:// quios .com

http://messagepub.com/

http://www.openmarMeket.com/products-and-services/message-and-content-delivery.html

http://www.mobilemarketing.net

http://www.textmarks.com/

http://www.mozes.com/

Long list and comparison table of providers:

http://www. developershome .com/ sms / smsGatewayProvComp .asp

Useful References General SMS: http://www. developershome .com/ sms / Provider Gateway addresses: http://en. wikipedia .org/ wiki /SMS_gateway Sms On Rails Engine: http:// github .com/ blythedunham / smsonrails Clickatell Gem: http://clickatell.rubyforge.org/ Exception Notification Plugin: http:// github .com/rails/exception_notification/tree/master Kannel: http://www. kannel .org/ RapidSms http://mobileactive.org/tagging/rapid-sms RubyGSM http://mobilehacking.org/index.php/ RubyGSM More Mobile Tools: http://mobilehacking.org/index.php/Main_Page Common Short Code Admin(CSCA) http://www. usshortcodes .com/

General SMS: http://www. developershome .com/ sms /

Provider Gateway addresses: http://en. wikipedia .org/ wiki /SMS_gateway

Sms On Rails Engine: http:// github .com/ blythedunham / smsonrails

Clickatell Gem: http://clickatell.rubyforge.org/

Exception Notification Plugin:

http:// github .com/rails/exception_notification/tree/master

Kannel: http://www. kannel .org/

RapidSms http://mobileactive.org/tagging/rapid-sms

RubyGSM http://mobilehacking.org/index.php/ RubyGSM

More Mobile Tools: http://mobilehacking.org/index.php/Main_Page

Common Short Code Admin(CSCA) http://www. usshortcodes .com/

SMS On Rails Engine: SMS in minutes script/plugin install git :// github .com/ blythedunham / smsonrails Demo Support for: Email Gateway Integration with phone carrier email domains Clickatell HTTPS Integration ORM/DB tables and migrations to store messages RESTful admin views to show message history Safe Locking and delivery Safely store phone numbers Dynamic parameter substitution Not included: (yet) Crons tasks or bulk message delivery Stale message recovery Pretty UI with rounded corners customized to your application Inbound Message support Please log tickets on github!

script/plugin install git :// github .com/ blythedunham / smsonrails

Support for:

Email Gateway Integration with phone carrier email domains

Clickatell HTTPS Integration

ORM/DB tables and migrations to store messages

RESTful admin views to show message history

Safe Locking and delivery

Safely store phone numbers

Dynamic parameter substitution

Not included: (yet)

Crons tasks or bulk message delivery

Stale message recovery

Pretty UI with rounded corners customized to your application

Inbound Message support

Please log tickets on github!

Questions? Yo Dawg, I heard you like Rails Apps so I put an SMS Rails App in your Rails App so you can SMS yourself to be awesome

Photos Trash can http://www. flickr .com/photos/ knittingskwerlgurl /396165625/ Flowers http://www. flickr .com/photos/ blondepowers /2722511769/ Timepiece: http://www. flickr .com/photos/ lebron /223223725/ Bear http://www. flickr .com/photos/ guenterleitenbauer /697065143/ Email http://www. flickr .com/photos/ charuca /2448685976/ Golden gate: http://www.flickr.com/photos/h2orange/63606734/ Beetles: http://www.flickr.com/photos/lhtang2006/3490285407/ Computer Cat: http://www.flickr.com/photos/sammrowe/3491788169/ Giant: http://www.flickr.com/photos/andreatallone/2841979961/

Trash can http://www. flickr .com/photos/ knittingskwerlgurl /396165625/

Flowers http://www. flickr .com/photos/ blondepowers /2722511769/

Timepiece: http://www. flickr .com/photos/ lebron /223223725/

Bear http://www. flickr .com/photos/ guenterleitenbauer /697065143/

Email http://www. flickr .com/photos/ charuca /2448685976/

Golden gate: http://www.flickr.com/photos/h2orange/63606734/

Beetles: http://www.flickr.com/photos/lhtang2006/3490285407/

Computer Cat: http://www.flickr.com/photos/sammrowe/3491788169/

Giant: http://www.flickr.com/photos/andreatallone/2841979961/