SlideShare a Scribd company logo
1 of 32
Download to read offline
 


                                                                                                                                                                                          11	
  
             	
  
                                                                                                                                                                                 Q3	
  
             	
                                                     	
  




Nemertes	
  Research	
  PilotHouse	
  Awards	
  
Unified	
  Communications	
  
The	
  Nemertes	
  Research	
  annual	
  PilotHouse	
  Awards	
  provide	
  insight	
  on	
  the	
  performance	
  of	
  technology	
  
vendors,	
  according	
  to	
  feedback	
  from	
  IT	
  decision-­‐makers	
  who	
  use	
  their	
  products	
  or	
  services.	
  
	
  
	
  
	
  
	
  




                                                                                                                                                 	
  




          N e m e r t e s 	
   R e s e a r c h 	
   	
   	
   	
   	
   	
   	
   w w w . n e m e r t e s . c o m 	
   	
   	
   	
   	
   	
   + 1 	
   8 8 8 . 2 4 1 . 2 6 8 5 	
  
 
	
  
	
  
T ABLE	
  OF	
   C ONTENTS 	
  
Unified	
  Communications	
  .......................................................................................................	
  3	
  
  Award	
  Definition	
  ...............................................................................................................................	
  3	
  
  Overview	
  ..............................................................................................................................................	
  3	
  
  Market	
  Classification	
  .......................................................................................................................	
  3	
  
  Ratings	
  ..................................................................................................................................................	
  4	
  
  Ratings	
  Categories	
  ............................................................................................................................	
  5	
  
    Technology	
  .........................................................................................................................................................	
  5	
  
    Customer	
  Service	
  .............................................................................................................................................	
  5	
  
            ......................................................................................................................................................................	
  5	
  
    Value	
  
  Results	
  Summary	
  ...............................................................................................................................	
  6	
  
Analysis	
  .......................................................................................................................................	
  7	
  
PilotHouse	
  Market	
  Challenger	
  Winner	
  ............................................................................	
  9	
  
  Siemens	
  .................................................................................................................................................	
  9	
  
     Technology	
  ......................................................................................................................................................	
  10	
  
     Customer	
  Service	
  ..........................................................................................................................................	
  10	
  
          ...................................................................................................................................................................	
  10	
  
     Value	
  
PilotHouse	
  Market	
  Leader	
  Winner	
  .................................................................................	
  11	
  
  Cisco	
  ....................................................................................................................................................	
  11	
  
     Technology	
  ......................................................................................................................................................	
  12	
  
     Customer	
  Service	
  ..........................................................................................................................................	
  12	
  
     Value	
   ...................................................................................................................................................................	
  12	
  
Pilothouse	
  Finalists:	
  Market	
  Leaders	
  ............................................................................	
  13	
  
  Avaya	
  ..................................................................................................................................................	
  14	
  
  Microsoft	
  ...........................................................................................................................................	
  16	
  
  IBM	
  Lotus	
  ..........................................................................................................................................	
  18	
  
PilotHouse	
  Finalists:	
  Market	
  Challengers	
  ....................................................................	
  20	
  
  ShoreTel	
  ............................................................................................................................................	
  21	
  
  Mitel	
  ....................................................................................................................................................	
  23	
  
Other	
  Market	
  Challengers	
  .................................................................................................	
  26	
  
Conclusion	
  ...............................................................................................................................	
  27	
  
Methodology	
  ...........................................................................................................................	
  28	
  
 Sample	
  Frame	
  .................................................................................	
  Error!	
  Bookmark	
  not	
  defined.	
  
   Planned	
  Sample	
  Size	
  ......................................................................	
  Error!	
  Bookmark	
  not	
  defined.	
  
   Survey	
  Sub-­‐Groups/Stratification	
  ............................................	
  Error!	
  Bookmark	
  not	
  defined.	
  
 Awards	
  ..............................................................................................	
  Error!	
  Bookmark	
  not	
  defined.	
  
 Timing	
  ...............................................................................................	
  Error!	
  Bookmark	
  not	
  defined.	
  
   Incentives	
  to	
  Participate	
  &	
  Time	
  Commitment	
  ..................	
  Error!	
  Bookmark	
  not	
  defined.	
  
   Future	
  Plans	
  .......................................................................................	
  Error!	
  Bookmark	
  not	
  defined.	
  



©Nemertes	
  Research	
  2011	
  ±	
  www.nemertes.com	
  ±	
  888-­‐241-­‐2685	
  ±	
  DN	
  1530	
                                                                                 2	
  
 
	
  
	
  

U NIFIED	
   C OMMUNICATIONS 	
  
	
  
By	
  Irwin	
  Lazar	
  
VP	
  and	
  Service	
  Director,	
  Nemertes	
  Research	
  
	
  

Award	
  Definition	
  
The	
  Nemertes	
  PilotHouse	
  award	
  for	
  Unified	
  Communications	
  (UC)	
  recognizes	
  vendors	
  
identified	
   and	
   rated	
   by	
   IT	
   professionals	
   as	
   their	
   strategic	
   partner	
   for	
   delivering	
   UC	
  
products,	
   which	
   integrate	
   voice,	
   video,	
   conferencing,	
   messaging,	
   and	
   presence	
   with	
  
office	
   and	
   business-­‐process	
   applications	
   to	
   improve	
   collaboration.	
   IT	
   professionals	
  
who	
   use	
   these	
   services	
   rated	
   their	
   providers	
   on	
   technology,	
   value,	
   and	
   customer	
  
service.	
  

Overview	
  
± The	
   goal	
   of	
   PilotHouse	
   awards	
   is	
   to	
   provide	
   analysis	
   of	
   vendor	
   and	
   service-­‐
     provider	
  performance	
  from	
  the	
  perspective	
  of	
  their	
  business	
  users.	
  
± Many	
   research	
   firms	
   offer	
   market	
   ranking;	
   Nemertes’	
   research	
   and	
   analysis	
   is	
  
     unique,	
   based	
   100%	
   on	
   the	
   views	
   and	
   experience	
   of	
   actual	
   Unified	
  
     Communications	
  users.	
  
± Research	
  is	
  wholly	
  independent	
  and	
  not	
  sponsored;	
  Nemertes	
  has	
  no	
  influence	
  
     over	
  vendor	
  or	
  service	
  provider	
  performance.	
  
± Opinions	
   are	
   those	
   of	
   the	
   IT	
   professionals	
   who	
   have	
   selected,	
   designed	
   and	
  
     deployed	
  the	
  technology	
  or	
  service.	
  	
  
± By	
   combining	
   benchmarking	
   (direct	
   user	
   interviews)	
   and	
   surveys,	
   Nemertes	
   is	
  
     able	
  to	
  provide	
  unique	
  insight	
  into	
  why	
  IT	
  professionals	
  rated	
  vendors	
  the	
  way	
  
     they	
  did.	
  
	
  
For	
  this	
  award,	
  Nemertes	
  gathered	
  ratings	
  on	
  UC	
  system	
  and	
  application	
  providers	
  
with	
   a	
   range	
   of	
   offerings.	
   (More	
   detail	
   on	
   the	
   program,	
   and	
   demographics	
   of	
  
participating	
   IT	
   professionals	
   is	
   available	
   in	
   the	
   methodology	
   at	
   the	
   end	
   of	
   this	
  
report.)	
  	
  	
  

Market	
  Classification	
  
We	
   segmented	
   UC	
   providers	
   into	
   two	
   categories:	
   Market	
   Leaders	
   and	
   Market	
  
Challengers,	
   and	
   offered	
   awards	
   within	
   each	
   category.	
   To	
   determine	
   the	
  
categorizations,	
   Nemertes’	
   analysts	
   evaluated	
   UC	
   market	
   presence	
   (looking	
   at	
  
revenue,	
   device	
   shipments,	
   and	
   number	
   of	
   customers)	
   based	
   on	
   our	
   own	
   research	
  



©Nemertes	
  Research	
  2011	
  ±	
  www.nemertes.com	
  ±	
  888-­‐241-­‐2685	
  ±	
  DN	
  1530	
                               3	
  
 
	
  
and	
  publicly	
  available	
  data.	
  Analysts	
  also	
  examined	
  natural	
  breakpoints	
  in	
  the	
  data,	
  
and	
   segmented	
   the	
   Market	
   Leaders	
   as	
   those	
   who	
   collectively	
   accounted	
   for	
   the	
   vast	
  
majority	
   of	
   each	
   market,	
   and	
   Market	
   Challengers	
   who	
   accounted	
   for	
   a	
   smaller	
  
percentage	
  of	
  the	
  overall	
  market.	
  
	
  
                                                UNIFIED COMMUNICATIONS
                      Market Leaders                                                          Market Challengers
                                                                              Alcatel-Lucent, Mitel, NEC, ShoreTel,
     Avaya, Cisco, IBM Lotus, Microsoft                                                     Siemens

Table	
  1:	
  Vendor	
  Classification,	
  Unified	
  Communications,	
  2011	
  

	
  
Nemertes	
  defines	
  the	
  UC	
  market	
  as	
  “segmented,”	
  meaning	
  that	
  no	
  vendor	
  controls	
  
more	
  than	
  30%	
  of	
  market	
  share,	
  and	
  no	
  two	
  vendors	
  control	
  more	
  than	
  50%.	
  	
  As	
  
published	
  in	
  various	
  public	
  sources,	
  Cisco,	
  Avaya,	
  IBM	
  Lotus,	
  and	
  Microsoft	
  account	
  
for	
  more	
  than	
  50%	
  of	
  the	
  UC	
  market.	
  	
  However	
  the	
  UC	
  market	
  is	
  often	
  difficult	
  to	
  
classify,	
  thanks	
  to	
  varying	
  definitions	
  of	
  UC.	
  	
  Nemertes	
  defines	
  UC	
  as	
  the	
  integration	
  
of	
  various	
  forms	
  of	
  real-­‐time	
  and	
  non-­‐real	
  time	
  collaboration	
  (e.g.	
  voice,	
  video,	
  
messaging,	
  conferencing)	
  into	
  a	
  set	
  of	
  applications	
  sharing	
  presence,	
  and	
  enabling	
  
establishment	
  of	
  any	
  mode	
  of	
  communications	
  (e.g.	
  escalating	
  an	
  IM	
  into	
  a	
  phone	
  
call,	
  video	
  chat,	
  or	
  Web	
  conference,	
  all	
  through	
  the	
  same	
  user	
  interface).	
  	
  Vendors	
  
typically	
  classify	
  any	
  of	
  their	
  standalone	
  offerings	
  in	
  the	
  UC	
  space	
  (e.g.	
  voice,	
  unified	
  
messaging,	
  Web	
  conferencing,	
  etc.)	
  as	
  “UC,”	
  making	
  it	
  difficult	
  to	
  determine	
  specific	
  
components	
  of	
  a	
  UC	
  implementation.	
  
	
  
The	
  Market	
  Leaders	
  reflect	
  the	
  dualities	
  of	
  the	
  UC	
  market:	
  Those	
  with	
  large	
  market	
  
share	
  in	
  voice	
  (Avaya,	
  Cisco)	
  that	
  are	
  broadening	
  into	
  the	
  desktop;	
  and	
  those	
  with	
  
large	
  market	
  share	
  in	
  desktop	
  collaboration	
  (IBM	
  Lotus,	
  Microsoft)	
  that	
  are	
  
broadening	
  into	
  voice.	
  	
  Market	
  Challengers	
  have	
  smaller	
  market	
  shares,	
  or	
  have	
  
traditionally	
  focused	
  on	
  the	
  small/midsize	
  business	
  market	
  or	
  a	
  limited	
  set	
  of	
  
verticals.	
  	
  It’s	
  worth	
  noting	
  that	
  our	
  data	
  set	
  predominantly	
  reflects	
  U.S.-­‐centric	
  
enterprises,	
  thus	
  we	
  classify	
  vendors	
  such	
  as	
  Alcatel-­‐Lucent	
  and	
  Siemens,	
  with	
  large	
  
market	
  share	
  outside	
  the	
  U.S.,	
  as	
  Challengers.	
  

Ratings	
  
We	
  asked	
  IT	
  professionals	
  to	
  rate	
  UC	
  providers	
  using	
  a	
  5-­‐point	
  scale,	
  where	
  5	
  is	
  
excellent,	
  4	
  is	
  good,	
  3	
  is	
  fair,	
  2	
  is	
  poor,	
  and	
  1	
  is	
  unacceptable.	
  Nemertes	
  then	
  used	
  
these	
  raw	
  scores	
  to	
  compute	
  average	
  scores	
  for	
  each	
  category.	
  	
  	
  
	
  




©Nemertes	
  Research	
  2011	
  ±	
  www.nemertes.com	
  ±	
  888-­‐241-­‐2685	
  ±	
  DN	
  1530	
                                     4	
  
 
	
  
The	
  maximum	
  possible	
  score	
  is	
  a	
  5.0	
  (and	
  although	
  some	
  vendors	
  did	
  receive	
  
perfect	
  scores	
  from	
  individual	
  IT	
  participants,	
  none	
  received	
  a	
  perfect	
  score	
  when	
  
all	
  ratings	
  were	
  averaged).	
  

Ratings	
  Categories	
  
The	
  participants	
  rated	
  their	
  technology	
  providers	
  in	
  three	
  areas:	
  
	
  
⇒ Technology	
  
⇒ Customer	
  Service	
  
⇒ Value	
  
⇒ Overall	
  Rating	
  (average	
  of	
  Technology,	
  Customer	
  Service,	
  and	
  Value)	
  

Technology	
  
Technology	
   ratings	
   gauge	
   how	
   customers	
   view	
   the	
   sophistication,	
   features,	
   and	
  
implementation	
   of	
   the	
   UC	
   product	
   they’re	
   rating.	
   Additionally,	
   this	
   score	
   reflects	
  
how	
   much	
   of	
   a	
   leader	
   a	
   vendor	
   is	
   in	
   the	
   UC	
   industry,	
   from	
   the	
   perspective	
   of	
   the	
  
customer.	
  

Customer	
  Service	
  
Customer-­‐service	
   ratings	
   cover	
   how	
   providers	
   perform	
   in	
   areas	
   such	
   as	
   technical	
  
support,	
  responsiveness	
  to	
  deployment	
  problems	
  and	
  concerns,	
  sales	
  support,	
  and	
  
general	
   customer	
   care.	
   Additionally,	
   technology	
   users	
   considered	
   the	
   willingness	
  
and	
  ability	
  of	
  the	
  vendors	
  to	
  answer	
  questions	
  effectively	
  and	
  promptly.	
  

Value	
  
Value	
   ratings	
   are	
   essentially	
   the	
   way	
   customers	
   perceive	
   what	
   they	
   get	
   for	
   what	
  
they	
  pay	
  for.	
  In	
  other	
  words,	
  are	
  they	
  getting	
  their	
  “bang	
  for	
  the	
  buck?”	
  
	
  




©Nemertes	
  Research	
  2011	
  ±	
  www.nemertes.com	
  ±	
  888-­‐241-­‐2685	
  ±	
  DN	
  1530	
                                         5	
  
 
	
  

Results	
  Summary	
  

                                   Unified Communications: All Vendors, Overall Scores
       4.30

                     4.27
       4.20

                                4.17
       4.10
                                            4.11

       4.00                                                 4.05


       3.90                                                                3.95           3.93

       3.80
                                                                                                            3.79          3.78
       3.70
                                                                                                                                          3.70

       3.60


       3.50
                   s




                            co




                                          ya




                                                                            l




                                                                                           s




                                                                                                            l




                                                                                                                       EC




                                                                                                                                         nt
                                                             t




                                                                         Te




                                                                                                        ite
                 en




                                                                                         tu
                                                          of




                                                                                                                                          e
                                           a
                            is




                                                                        e




                                                                                                                      N
                                                                                                      M
                                                                                       Lo




                                                                                                                                       uc
                                                       os
                                        Av
                em




                                                                     or
                            C




                                                                                                                                    l-L
                                                        r




                                                                   Sh




                                                                                    M
                                                     ic
              Si




                                                                                  IB




                                                                                                                                   te
                                                   M




                                                                                                                                    a
                                                                                                                                 lc
                                                                                                                            A
                                       Market Leaders                                                           Market Challengers



Chart	
  1:	
  Overall	
  Scores,	
  Unified	
  Communications,	
  2011	
  

	
  
⇒ Among	
  Market	
  Challengers,	
  Siemens	
  wins	
  the	
  PilotHouse	
  Award.	
  
     • Siemens’	
  overall	
  score	
  is	
  4.27.	
  
     	
  
⇒ Among	
  Market	
  Leaders,	
  Cisco	
  wins	
  the	
  PilotHouse	
  Award.	
  
     • Cisco’s	
  overall	
  score	
  is	
  4.17.	
  
          	
  
⇒ A	
  total	
  of	
  nine	
  providers	
  received	
  enough	
  responses	
  for	
  us	
  to	
  include	
  them	
  in	
  
     this	
  year’s	
  analysis.	
  
     • Four	
  providers	
  are	
  Market	
  Leaders;	
  five	
  are	
  Market	
  Challengers.	
  
     • Market	
  Leaders’	
  overall	
  average	
  score	
  is	
  4.06.	
  
     • Market	
  Challengers’	
  overall	
  average	
  score	
  is	
  3.90.	
  




©Nemertes	
  Research	
  2011	
  ±	
  www.nemertes.com	
  ±	
  888-­‐241-­‐2685	
  ±	
  DN	
  1530	
                                          6	
  
 
	
  

A NALYSIS 	
  
                            Nemertes 2011 PilotHouse Awards
                                  Unified Communications
                   Market Leaders                  Market Challengers
             Avaya, Cisco, IBM Lotus, Microsoft                             Alcatel-Lucent, Mitel, NEC, ShoreTel, Siemens

                                                                                                     Customer
       Winners                                   Overall                 Technology                   Service       Value
       Siemens                                     4.27                       4.27                          4.18    4.36
       Cisco                                       4.17                       4.23                          4.16    4.12
       Other Finalists
       Avaya                                       4.11                       4.19                          4.10    4.03
       Microsoft                                   4.05                       4.13                          3.96    4.06
       ShoreTel                                    3.95                       3.92                          3.85    4.08
       IBM Lotus                                   3.93                       3.88                          3.90    4.03
       Mitel                                       3.79                       3.64                          4.05    3.68
       NEC                                         3.78                       3.67                          3.67    4.00
       Alcatel-Lucent                              3.70                       3.72                          3.67    3.72
                            Rating Scale: 5=Excellent; 4=Good; 3=Fair; 2=Poor; 1=Unacceptable

Table	
  2:	
  Vendor	
  Scores,	
  Unified	
  Communications,	
  2011	
  

Overall,	
  scores	
  for	
  UC	
  lagged	
  slightly	
  behind	
  those	
  for	
  other	
  technology	
  areas,	
  with	
  
only	
  four	
  vendors	
  scoring	
  higher	
  than	
  a	
  4.0.	
  	
  Winners	
  significantly	
  outperformed	
  
non-­‐winners,	
  especially	
  in	
  the	
  Market	
  Challenger	
  category	
  where	
  Siemens	
  was	
  the	
  
only	
  vendor	
  to	
  score	
  higher	
  than	
  a	
  4.0	
  in	
  all	
  categories.	
  
	
  
Notable	
  is	
  the	
  lack	
  of	
  any	
  consistency	
  of	
  scores	
  among	
  ratings	
  categories.	
  	
  Cisco,	
  for	
  
example	
  saw	
  its	
  highest	
  score	
  in	
  technology,	
  while	
  Siemens	
  scored	
  best	
  in	
  value.	
  	
  
Absent	
  are	
  any	
  consistently	
  poor	
  scores	
  in	
  a	
  single	
  category	
  (e.g.	
  “customer	
  service	
  
consistently	
  lagging	
  behind	
  value”).	
  	
  	
  
	
  
What	
  does	
  this	
  mean?	
  	
  Each	
  vendor	
  has	
  strengths	
  and	
  weaknesses	
  that	
  it	
  must	
  focus	
  
on	
  improving,	
  rather	
  than	
  a	
  weak	
  area	
  (or	
  areas)	
  existing	
  for	
  the	
  entire	
  UC	
  
marketplace.	
  	
  	
  Cisco’s	
  biggest	
  weakness	
  (customer	
  service)	
  isn’t	
  the	
  same	
  as	
  Avaya’s	
  
(value)	
  or	
  IBM	
  Lotus’	
  (technology).	
  It	
  also	
  reflects	
  the	
  relative	
  “newness”	
  of	
  the	
  UC	
  
market	
  and	
  the	
  fact	
  that	
  it’s	
  rapidly	
  changing	
  in	
  terms	
  of	
  product	
  capabilities.	
  
	
  
Like	
  last	
  year,	
  those	
  who	
  approach	
  the	
  UC	
  market	
  from	
  a	
  telephony	
  perspective	
  
(Cisco,	
  Avaya)	
  outscored	
  those	
  who	
  have	
  moved	
  from	
  IM/messaging	
  into	
  telephony	
  


©Nemertes	
  Research	
  2011	
  ±	
  www.nemertes.com	
  ±	
  888-­‐241-­‐2685	
  ±	
  DN	
  1530	
                          7	
  
 
	
  
(Microsoft,	
  IBM	
  Lotus).	
  	
  This	
  reflects	
  the	
  continued	
  struggle	
  for	
  IM/messaging	
  
vendors	
  to	
  play	
  in	
  a	
  space	
  where	
  customers	
  focus	
  far	
  more	
  attention	
  on	
  reliability,	
  
resiliency,	
  performance,	
  and	
  support	
  than	
  in	
  the	
  desktop	
  applications	
  space.	
  	
  This	
  
year,	
  like	
  last,	
  both	
  UC	
  winners	
  come	
  from	
  telephony	
  backgrounds.	
  	
  IM/messaging	
  
vendors	
  must	
  continue	
  their	
  efforts	
  to	
  demonstrate	
  to	
  IT	
  leaders	
  that	
  they	
  
understand	
  the	
  real-­‐time	
  application	
  requirements,	
  of	
  telephony	
  and	
  increasingly	
  
video.	
  
	
  
⇒ Overall,	
  Market	
  Leaders	
  score	
  a	
  4.06;	
  Challengers	
  earn	
  a	
  3.90.	
  	
  Siemens	
  is	
  the	
  
     exception	
  to	
  the	
  rule,	
  outperforming	
  all	
  vendors	
  regardless	
  of	
  size.	
  
     • Reason:	
  The	
  primary	
  gaps	
  between	
  Market	
  Leaders	
  and	
  Market	
  Challengers	
  
            exists	
  in	
  technology	
  and	
  customer	
  service.	
  	
  Market	
  Leaders	
  have	
  deeper	
  
            pockets,	
  larger	
  support	
  networks,	
  and	
  a	
  large	
  installed	
  base	
  from	
  which	
  they	
  
            can	
  cross-­‐sell;	
  using	
  bundled	
  licensing	
  programs	
  to	
  deliver	
  UC	
  as	
  an	
  add-­‐on	
  
            to	
  existing	
  telephony,	
  video,	
  or	
  IM/messaging	
  or	
  other	
  application	
  licenses.	
  
            	
  
⇒ In	
  technology,	
  Market	
  Leaders	
  compile	
  a	
  score	
  of	
  4.11	
  and	
  Challengers	
  get	
  a	
  
     3.84.	
  
     • Reason:	
  Market	
  Leaders	
  typically	
  have	
  the	
  broadest	
  product	
  offering,	
  the	
  
            largest	
  R&D	
  budgets	
  and	
  often	
  the	
  greatest	
  support	
  for	
  mobility,	
  a	
  hot	
  area	
  
            among	
  IT	
  leaders	
  these	
  days.	
  
            	
  
⇒ In	
  customer	
  service,	
  Market	
  Leaders	
  receive	
  a	
  score	
  of	
  4.03	
  and	
  Challengers	
  earn	
  
     a	
  3.88.	
  
     • Reason:	
  Market	
  Leaders	
  typically	
  have	
  the	
  fiscal	
  wherewithal	
  to	
  devote	
  
            substantial	
  resources	
  to	
  not	
  just	
  first-­‐tier	
  customer	
  service,	
  but	
  higher-­‐level	
  
            technical	
  support.	
  	
  Market	
  Challengers	
  tend	
  to	
  more	
  often	
  sell	
  and	
  support	
  
            customers	
  through	
  channels,	
  where	
  support	
  may	
  vary.	
  	
  Many	
  IT	
  leaders	
  say	
  
            they	
  see	
  differences	
  in	
  the	
  support	
  they	
  receive	
  from	
  multiple	
  channels	
  for	
  
            the	
  same	
  vendor.	
  
            	
  
⇒ In	
  value,	
  Market	
  Leaders	
  garner	
  a	
  4.06	
  and	
  Challengers	
  receive	
  a	
  3.97.	
  
     • Reason:	
  Here	
  is	
  where	
  Market	
  Challengers	
  are	
  placing	
  their	
  competitive	
  
            emphasis,	
  using	
  innovative	
  approaches	
  such	
  as	
  cloud,	
  or	
  support	
  for	
  
            virtualization	
  to	
  offer	
  compelling	
  services	
  as	
  a	
  reduced	
  cost.	
  	
  Although	
  they	
  
            aren’t	
  yet	
  demonstrating	
  greater	
  value	
  than	
  Market	
  Leaders,	
  they	
  are	
  close.	
  
            	
  
⇒ No	
  single	
  score	
  factored	
  into	
  winner’s	
  advantages,	
  rather	
  all	
  winners	
  outscored	
  
     their	
  competitors	
  across	
  the	
  board	
  in	
  all	
  scores.	
  
     • Siemens	
  has	
  the	
  top	
  overall	
  scores	
  among	
  all	
  vendors,	
  in	
  all	
  scoring	
  
            categories.	
  
     • No	
  Market	
  Leader	
  outscored	
  Cisco	
  in	
  any	
  of	
  the	
  three	
  ratings	
  categories.	
  


©Nemertes	
  Research	
  2011	
  ±	
  www.nemertes.com	
  ±	
  888-­‐241-­‐2685	
  ±	
  DN	
  1530	
                           8	
  
 
	
  

P ILOT H OUSE	
   M ARKET	
   C HALLENGER	
   W INNER 	
  
                                                         Siemens Ratings
       4.40



                                                                                                                      4.36


       4.30


                      4.27                               4.27



       4.20


                                                                                            4.18




       4.10




       4.00
                    Overall                         Technology                     Customer Service                  Value

Chart	
  2:	
  PilotHouse	
  Market	
  Challenger	
  Winner:	
  Siemens,	
  Unified	
  Communications,	
  2011	
  

Siemens	
  
Summary:	
  	
  Siemens’	
  4.27	
  is	
  the	
  top	
  overall	
  score	
  among	
  Market	
  Challengers,	
  and	
  it’s	
  
also	
  the	
  highest	
  overall	
  score	
  among	
  all	
  UC	
  vendors.	
  	
  In	
  fact,	
  Siemens	
  receives	
  the	
  
highest	
  score	
  in	
  each	
  of	
  the	
  three	
  categories	
  measured.	
  	
  Siemens	
  is	
  somewhat	
  of	
  a	
  
pioneer	
  in	
  the	
  UC	
  space,	
  having	
  introduced	
  arguably	
  the	
  first	
  UC	
  product,	
  
OpenScape,	
  back	
  in	
  the	
  early	
  2000s	
  (now	
  knows	
  as	
  OpenScape	
  UC	
  Server).	
  	
  Siemens	
  
continues	
  to	
  innovate,	
  offering	
  both	
  on-­‐premise	
  and	
  hosted	
  solutions,	
  as	
  well	
  as	
  a	
  
cloud-­‐based	
  service.	
  	
  Though	
  Siemens	
  lacks	
  the	
  U.S.	
  market	
  share	
  of	
  Cisco	
  and	
  
Avaya,	
  its	
  offering	
  is	
  just	
  as	
  broad,	
  featuring	
  a	
  full	
  suite	
  of	
  UC	
  and	
  telephony	
  services	
  
covering	
  small	
  and	
  large	
  offices;	
  contact	
  centers,	
  and	
  specific	
  vertical	
  solutions	
  (e.g.	
  
trading	
  floors).	
  “OpenScape	
  is	
  a	
  great	
  tool/service	
  for	
  us,”	
  says	
  the	
  IT	
  manager	
  of	
  a	
  
state	
  government	
  agency.	
  	
  
	
  
⇒ Then	
  and	
  Now:	
  	
  Siemens	
  improved	
  its	
  scores	
  across	
  the	
  board	
  from	
  last	
  year,	
  
     from	
  an	
  overall	
  3.75	
  to	
  this	
  year’s	
  4.27	
  with	
  the	
  largest	
  jumps	
  coming	
  in	
  
     technology	
  (3.71	
  to	
  4.27)	
  and	
  value	
  (3.71	
  to	
  4.36).	
  	
  These	
  scores	
  reflect	
  Siemens’	
  



©Nemertes	
  Research	
  2011	
  ±	
  www.nemertes.com	
  ±	
  888-­‐241-­‐2685	
  ±	
  DN	
  1530	
                                9	
  
 
	
  
  efforts	
  to	
  not	
  only	
  broaden	
  its	
  solution	
  set,	
  but	
  to	
  differentiate	
  itself	
  from	
  
  competitors	
  by	
  offering	
  delivery	
  models	
  such	
  as	
  cloud-­‐based	
  services.	
  
  	
  
⇒ Future	
  Direction:	
  The	
  big	
  challenge	
  for	
  Siemens	
  is	
  to	
  expand	
  its	
  reach	
  in	
  the	
  U.S.	
  
  market	
  beyond	
  the	
  SMB	
  and	
  specific	
  verticals	
  such	
  as	
  healthcare,	
  education,	
  and	
  
  government,	
  where	
  it	
  has	
  had	
  historical	
  strength.	
  	
  This	
  year,	
  about	
  80%	
  of	
  the	
  
  firms	
  rating	
  Siemens	
  were	
  smaller	
  than	
  $50	
  million	
  in	
  revenue.	
  	
  The	
  good	
  thing	
  
  for	
  Siemens	
  is	
  that	
  customers	
  evenly	
  praise	
  its	
  performance	
  across	
  all	
  ratings	
  
  areas.	
  	
  IT	
  leaders	
  in	
  hospitality	
  and	
  manufacturing	
  both	
  praise	
  its	
  quality,	
  while	
  
  the	
  IT	
  director	
  of	
  a	
  small	
  education	
  institution	
  says,	
  “It’s	
  all	
  about	
  service.”	
  	
  	
  

Technology	
  
⇒ Siemens’	
  4.27,	
  like	
  all	
  its	
  scores,	
  is	
  the	
  highest	
  among	
  all	
  UC	
  vendors,	
  Market	
  
  Leader	
  and	
  Market	
  Challenger	
  alike.	
  	
  With	
  its	
  broad	
  product	
  set,	
  and	
  ability	
  to	
  
  support	
  multiple	
  delivery	
  models,	
  Siemens’	
  customers	
  consider	
  it	
  to	
  be	
  a	
  
  technology	
  leader.	
  

Customer	
  Service	
  
⇒ Siemens’	
  customer-­‐service	
  score	
  of	
  4.18	
  is	
  its	
  lowest	
  rating,	
  considerably	
  lower	
  
  than	
  its	
  other	
  scores	
  but	
  overall	
  still	
  the	
  highest	
  among	
  all	
  vendors.	
  	
  Siemens’	
  
  score	
  is	
  particularly	
  impressive	
  given	
  the	
  change	
  the	
  company	
  has	
  gone	
  through	
  
  over	
  the	
  last	
  few	
  years	
  as	
  Siemens	
  AG	
  sold	
  a	
  majority	
  stake	
  in	
  the	
  firm	
  to	
  
  investment	
  firm	
  Gores	
  Group.	
  	
  
  • One	
  strength	
  for	
  Siemens	
  is	
  its	
  international	
  footprint.	
  	
  “We	
  choose	
  Siemens	
  
     because	
  it	
  is	
  international,	
  able	
  to	
  support	
  us	
  in	
  multiple	
  countries,”	
  says	
  the	
  
     CIO	
  for	
  a	
  midsize	
  professional-­‐services	
  firm.	
  

Value	
  
⇒ Value	
  is	
  Siemens	
  top-­‐scoring	
  area,	
  with	
  a	
  4.36,	
  and	
  again,	
  the	
  highest	
  score	
  
         among	
  all	
  rated	
  firms.	
  	
  Customers	
  perceive	
  Siemens	
  as	
  providing	
  tremendous	
  
         bang	
  for	
  the	
  buck.	
  
         • “For	
  us,	
  it’s	
  price	
  that	
  makes	
  us	
  choose	
  Siemens,”	
  says	
  the	
  IT	
  buyer	
  for	
  a	
  
            small	
  construction	
  company.	
  	
  	
  
	
  	
  
	
                                               	
  




©Nemertes	
  Research	
  2011	
  ±	
  www.nemertes.com	
  ±	
  888-­‐241-­‐2685	
  ±	
  DN	
  1530	
                               10	
  
 
	
  
P ILOT H OUSE	
   M ARKET	
   L EADER	
   W INNER 	
  
                                                            Cisco Ratings
       4.30




                                                         4.23

       4.20




                      4.17
                                                                                            4.16




       4.10
                                                                                                                     4.12




       4.00
                    Overall                         Technology                     Customer Service                 Value

Chart	
  3:	
  PilotHouse	
  Market	
  Leader	
  Winner:	
  Cisco,	
  Unified	
  Communications,	
  2011	
  

Cisco	
  
⇒ Summary:	
  	
  Cisco’s	
  4.17	
  is	
  the	
  second-­‐highest	
  overall	
  score,	
  and	
  the	
  top	
  overall	
  
     score	
  among	
  Market	
  Leaders.	
  	
  Cisco	
  wins	
  its	
  second	
  consecutive	
  Market	
  Leader	
  
     PilotHouse	
  award.	
  	
  Customers	
  routinely	
  cite	
  reliability,	
  service	
  and	
  support,	
  and	
  
     Cisco’s	
  breadth	
  of	
  products	
  as	
  key	
  buying	
  criteria.	
  	
  
     • Says	
  the	
  IT	
  director	
  for	
  a	
  global	
  publishing	
  and	
  media	
  company,	
  “Cisco	
  
             provides	
  us	
  good	
  range	
  of	
  options	
  for	
  voice,	
  video,	
  conferencing,	
  and	
  
             messaging.”	
  	
  Adds	
  the	
  IT	
  manager	
  for	
  a	
  regional	
  healthcare	
  firm,	
  “Cisco	
  
             means	
  reliability.”	
  
	
  
⇒ Then	
  and	
  Now:	
  Cisco	
  continues	
  to	
  demonstrate	
  continued	
  improvement;	
  raising	
  
     its	
  score	
  from	
  a	
  4.08	
  in	
  2010	
  (after	
  increasing	
  from	
  a	
  3.88	
  in	
  2009).	
  	
  	
  Cisco’s	
  
     increasing	
  scores	
  demonstrate	
  its	
  payoff	
  from	
  recent	
  acquisitions	
  to	
  broaden	
  its	
  
     UC	
  portfolio	
  including	
  Jabber	
  and	
  Tandberg.	
  	
  Overall	
  Cisco	
  is	
  a	
  reliable	
  vendor	
  
     with	
  good	
  support	
  services,	
  says	
  the	
  IT	
  director	
  for	
  a	
  regional	
  manufacturing	
  
     firm.	
  
                	
  


©Nemertes	
  Research	
  2011	
  ±	
  www.nemertes.com	
  ±	
  888-­‐241-­‐2685	
  ±	
  DN	
  1530	
                             11	
  
 
	
  
⇒ Future	
  Direction:	
  For	
  Cisco,	
  the	
  key	
  challenge	
  is	
  branching	
  out	
  beyond	
  voice	
  and	
  
     selling	
  its	
  customers	
  on	
  its	
  vision	
  of	
  video	
  emerging	
  as	
  the	
  key	
  component	
  of	
  a	
  
     collaboration	
  strategy.	
  	
  “Cisco	
  envisions	
  presence	
  and	
  video	
  and	
  all	
  this	
  usability.	
  
     We	
  aren't	
  really	
  seeing	
  the	
  value	
  out	
  of	
  it	
  yet,”	
  says	
  the	
  telecom	
  manager	
  for	
  a	
  
     global	
  manufacturing	
  organization.	
  	
  However	
  Cisco’s	
  push	
  to	
  drive	
  technology	
  
     change	
  is	
  resonating	
  with	
  its	
  customers,	
  its	
  4.23	
  technology	
  score	
  is	
  its	
  highest,	
  
     and	
  the	
  highest	
  of	
  all	
  Market	
  Leaders.	
  

Technology	
  
⇒ As	
  noted,	
  Cisco	
  scores	
  a	
  4.23	
  on	
  technology,	
  highest	
  of	
  all	
  Market	
  Leaders,	
  and	
  a	
  
  solid	
  improvement	
  over	
  last	
  year’s	
  4.08.	
  	
  Cisco	
  continues	
  to	
  differentiate	
  its	
  UC	
  
  offerings	
  through	
  support	
  for	
  ubiquitous	
  video	
  and	
  extensibility	
  beyond	
  the	
  
  enterprise	
  firewall,	
  an	
  effort	
  that	
  is	
  resonating	
  with	
  its	
  customers.	
  
  • “Cisco	
  makes	
  it	
  easier	
  for	
  us	
  to	
  stay	
  connected	
  to	
  the	
  outside	
  world,”	
  says	
  the	
  
        CTO	
  of	
  a	
  midsize	
  manufacturing	
  organization.	
  

Customer	
  Service	
  
⇒ Cisco	
  scores	
  a	
  4.16	
  on	
  customer	
  service,	
  just	
  .02	
  below	
  overall	
  winner	
  Siemens,	
  
  but	
  ahead	
  of	
  its	
  fellow	
  Market	
  Leaders.	
  	
  Here	
  again,	
  Cisco	
  improved	
  on	
  its	
  2009	
  
  score	
  of	
  3.99.	
  	
  Customers	
  continue	
  to	
  praise	
  Cisco’s	
  service	
  and	
  support,	
  which	
  is	
  
  a	
  departure	
  from	
  recent	
  years	
  and	
  a	
  clear,	
  concerted	
  effort	
  on	
  Cisco’s	
  part.	
  
  • “With	
  Cisco	
  we	
  always	
  get	
  fast	
  response	
  times	
  whenever	
  we	
  need	
  support,”	
  
        says	
  the	
  CIO	
  of	
  a	
  small	
  manufacturing	
  company.	
  
  • “Cisco	
  provides	
  great	
  service	
  across	
  the	
  board,”	
  says	
  the	
  head	
  of	
  IT	
  for	
  a	
  
        midsize	
  education	
  organization	
  notes.	
  

Value	
  
⇒ Cisco’s	
  value	
  score	
  of	
  4.12	
  was	
  its	
  lowest	
  overall	
  score	
  in	
  any	
  rating	
  area,	
  but	
  
  even	
  here	
  Cisco	
  beats	
  all	
  of	
  its	
  Market	
  Leader	
  competition	
  (and	
  improves	
  from	
  a	
  
  3.86	
  in	
  2009,	
  its	
  largest	
  improvement	
  in	
  any	
  area).	
  	
  We’ve	
  often	
  heard	
  the	
  
  perception	
  that	
  Cisco	
  is	
  a	
  premium	
  brand,	
  more	
  expensive,	
  than	
  other	
  products,	
  
  but	
  that	
  “nobody	
  gets	
  fired	
  for	
  buying	
  Cisco.”	
  Cisco’s	
  improved	
  score	
  this	
  year	
  is	
  
  evidence	
  that	
  it	
  is	
  increasingly	
  able	
  to	
  demonstrate	
  the	
  value	
  of	
  its	
  products,	
  
  even	
  in	
  cases	
  where	
  the	
  actual	
  price	
  is	
  more	
  expensive	
  than	
  competitors.	
  	
  
  Perhaps	
  more	
  importantly,	
  we	
  hear	
  praise	
  for	
  Cisco’s	
  cost	
  competitiveness,	
  
  something	
  we	
  rarely	
  heard	
  before.	
  
  • “We	
  looked	
  at	
  ShoreTel,	
  and	
  ShoreTel	
  was	
  more	
  expensive.	
  We	
  looked	
  at	
  
       video	
  this	
  year	
  with	
  Lifesize;	
  Cisco	
  was	
  cheaper	
  there,	
  too,”	
  says	
  the	
  senior	
  
       architect	
  of	
  a	
  global	
  manufacturing	
  company.	
  	
  
  	
                                              	
  



©Nemertes	
  Research	
  2011	
  ±	
  www.nemertes.com	
  ±	
  888-­‐241-­‐2685	
  ±	
  DN	
  1530	
                              12	
  
 
	
  
P ILOTHOUSE	
   F INALISTS : 	
   M ARKET	
   L EADERS	
   	
  
                                Unified Communications: Market Leaders
                                 Cisco                     Avaya                 Microsoft                  IBM Lotus
       4.30



                                                              4.23
       4.20
                             4.17                             4.19
                                                                                               4.16

                             4.11                             4.13
       4.10
                                                                                                                        4.12
                                                                                               4.10
                                                                                                                        4.06
                             4.05
                                                                                                                        4.03
       4.00                                                                                                             4.03

                                                                                               3.96
                             3.93
       3.90
                                                                                              3.90
                                                              3.88



       3.80
                      Overall                           Technology               Customer Service                   Value


Chart	
  4:	
  PilotHouse	
  Market	
  Leaders:	
  Unified	
  Communications,	
  2011	
  

	
                                               	
  




©Nemertes	
  Research	
  2011	
  ±	
  www.nemertes.com	
  ±	
  888-­‐241-­‐2685	
  ±	
  DN	
  1530	
                        13	
  
 
	
  

                                                       Avaya Ratings
       4.20

                                                    4.19




       4.10         4.11
                                                                                    4.10




                                                                                                            4.03


       4.00
                  Overall                       Technology                  Customer Service                Value
                                                                                                                         	
  
Chart	
  5:	
  Market	
  Leader:	
  Avaya,	
  Unified	
  Communications,	
  2011	
  

Avaya	
  
⇒ Avaya	
  and	
  Overall	
  Scores	
  
Avaya	
  closes	
  the	
  gap	
  between	
  it	
  and	
  Cisco	
  this	
  year,	
  narrowing	
  its	
  overall	
  
disadvantage	
  to	
  just	
  .06,	
  versus	
  .13	
  in	
  2010.	
  	
  It’s	
  impressive	
  that	
  it	
  improved	
  its	
  
competitive	
  positioning	
  despite	
  the	
  upheaval	
  of	
  executing	
  on	
  its	
  integration	
  of	
  
Nortel,	
  shifting	
  to	
  a	
  channel-­‐based	
  model	
  for	
  the	
  SME,	
  and	
  delivering	
  a	
  new	
  line	
  of	
  
products	
  focused	
  around	
  its	
  Aura	
  SIP	
  session	
  manager.	
  	
  Avaya’s	
  scores	
  are	
  marked	
  
by	
  significant	
  disparity	
  between	
  its	
  technology	
  (4.19)	
  and	
  value	
  (4.03)	
  ratings,	
  
reflecting	
  on	
  both	
  the	
  success	
  of	
  Avaya	
  in	
  using	
  Aura,	
  it’s	
  expanded	
  mobility	
  
products,	
  new	
  video	
  offerings,	
  and	
  its	
  recently	
  launched	
  Flare	
  user	
  experience	
  to	
  
portray	
  itself	
  as	
  a	
  technical	
  leader;	
  and	
  the	
  failure	
  of	
  Avaya	
  to	
  improve	
  its	
  value	
  
score	
  as	
  much	
  as	
  its	
  other	
  scores	
  (value	
  is	
  the	
  one	
  ratings	
  category	
  where	
  Avaya	
  
trails	
  the	
  average	
  within	
  its	
  market	
  classification).	
  	
  For	
  comparison,	
  the	
  margin	
  
between	
  Avaya’s	
  highest	
  and	
  lowest	
  score	
  in	
  2010	
  was	
  .13	
  where	
  this	
  year	
  it	
  grows	
  
to	
  .16.	
  	
  While	
  Avaya	
  improved	
  its	
  technology	
  score	
  from	
  a	
  3.95	
  to	
  4.19,	
  it	
  only	
  
improved	
  value	
  from	
  3.82	
  to	
  4.03.	
  	
  
⇒ The	
  average	
  overall	
  score	
  of	
  all	
  Market	
  Leaders	
  is	
  4.06;	
  Avaya’s	
  is	
  4.11.	
  
⇒ The	
  average	
  technology	
  score	
  of	
  all	
  Market	
  Leaders	
  is	
  4.11;	
  Avaya’s	
  is	
  4.19.	
  
       • “I	
  like	
  the	
  fact	
  that	
  Avaya	
  is	
  an	
  open	
  system	
  and	
  will	
  integrate	
  with	
  other	
  
                 applications,”	
  says	
  the	
  IT	
  director	
  of	
  a	
  midsize	
  services	
  firm.	
  



©Nemertes	
  Research	
  2011	
  ±	
  www.nemertes.com	
  ±	
  888-­‐241-­‐2685	
  ±	
  DN	
  1530	
                         14	
  
 
	
  
⇒ The	
  average	
  customer	
  service	
  score	
  of	
  all	
  Market	
  Leaders	
  is	
  4.03;	
  Avaya’s	
  is	
  4.10.	
  
     • “We	
  have	
  great	
  account	
  team.	
  They	
  find	
  us	
  answers,	
  and	
  are	
  willing	
  to	
  work	
  
       with	
  us.	
  You	
  can't	
  ask	
  for	
  much	
  more	
  than	
  that,”	
  says	
  the	
  IT	
  manager	
  of	
  a	
  
       large	
  professional-­‐services	
  company	
  
⇒ The	
  average	
  value	
  score	
  of	
  all	
  Market	
  Leaders	
  is	
  4.06;	
  Avaya’s	
  is	
  4.03.	
  
     • “There	
  are	
  better	
  phones	
  out	
  there	
  at	
  a	
  lower	
  price	
  point,	
  but	
  Avaya’s	
  
       products	
  work,”	
  says	
  the	
  manager	
  of	
  IT	
  at	
  a	
  small	
  educational	
  institution.	
  	
  
	
                                       	
  




©Nemertes	
  Research	
  2011	
  ±	
  www.nemertes.com	
  ±	
  888-­‐241-­‐2685	
  ±	
  DN	
  1530	
                          15	
  
 
	
  

                                                        Microsoft Ratings
       4.20




       4.10                                           4.13

                                                                                                               4.06
                     4.05
       4.00


                                                                                       3.96
       3.90




       3.80




       3.70




       3.60




       3.50
                   Overall                        Technology                   Customer Service               Value
                                                                                                                                  	
  
Chart	
  6:	
  Market	
  Leader:	
  Microsoft,	
  Unified	
  Communications,	
  2011	
  

Microsoft	
  
⇒ Microsoft	
  and	
  Overall	
  Scores	
  
Microsoft’s	
  overall	
  score	
  is	
  a	
  4.05,	
  trailing	
  Avaya	
  by	
  only	
  .06,	
  narrowing	
  the	
  gap	
  
between	
  it	
  and	
  Avaya	
  from	
  .12	
  in	
  2010.	
  	
  Microsoft	
  continues	
  to	
  gain	
  tremendous	
  
momentum	
  for	
  its	
  UC	
  offering,	
  with	
  more	
  companies	
  citing	
  Microsoft	
  as	
  their	
  
strategic	
  vendor	
  for	
  UC	
  than	
  any	
  other	
  vendor.	
  	
  With	
  the	
  introduction	
  of	
  Lync	
  in	
  
2011,	
  Microsoft	
  has	
  taken	
  direct	
  aim	
  at	
  not	
  only	
  the	
  desktop	
  messaging	
  and	
  
conferencing	
  portion	
  of	
  the	
  UC	
  market,	
  but	
  at	
  the	
  core	
  voice	
  services	
  largely	
  owned	
  
by	
  Market	
  Leaders	
  Cisco	
  and	
  Avaya.	
  	
  Microsoft	
  still	
  has	
  work	
  to	
  do	
  to	
  convince	
  IT	
  
buyers	
  that	
  it	
  is	
  able	
  to	
  offer	
  reliable,	
  feature	
  rich	
  voice	
  services,	
  though	
  its	
  slightly-­‐
higher-­‐than-­‐average	
  technology	
  score	
  demonstrates	
  that	
  Microsoft	
  is	
  convincing	
  its	
  
customers	
  that	
  it	
  is	
  driving	
  technology	
  change.	
  	
  Microsoft’s	
  biggest	
  challenge	
  is	
  in	
  
customer	
  service,	
  where	
  it	
  trails	
  the	
  mean	
  Market	
  Leader	
  score	
  by	
  .07.	
  	
  As	
  in	
  2010,	
  
customer	
  service	
  remains	
  Microsoft’s	
  Achilles	
  heel,	
  though	
  its	
  score	
  improved	
  from	
  
3.66	
  last	
  year	
  to	
  3.96	
  this	
  year.	
  	
  If	
  Microsoft	
  can	
  continue	
  to	
  improve	
  on	
  customer	
  
service,	
  it	
  can	
  challenge	
  Market	
  Leaders	
  for	
  the	
  2012	
  PilotHouse	
  award.	
  
⇒ The	
  average	
  overall	
  score	
  of	
  all	
  Market	
  Leaders	
  is	
  4.06;	
  Microsoft’s	
  is	
  4.05.	
  
      • “We	
  haven't	
  had	
  many	
  problems;	
  their	
  overall	
  support	
  has	
  been	
  good.	
  	
  They	
  
          are	
  proactive	
  in	
  including	
  us	
  in	
  demos/pilots,	
  have	
  allowed	
  us	
  to	
  collaborate	
  



©Nemertes	
  Research	
  2011	
  ±	
  www.nemertes.com	
  ±	
  888-­‐241-­‐2685	
  ±	
  DN	
  1530	
                                  16	
  
 
	
  
        with	
  other	
  large	
  companies,”	
  says	
  the	
  director	
  of	
  telecom	
  for	
  a	
  global	
  energy	
  
        firm.	
  
⇒ The	
  average	
  technology	
  score	
  of	
  all	
  Market	
  Leaders	
  is	
  4.11;	
  Microsoft’s	
  is	
  4.13.	
  
     • “It	
  works	
  well	
  for	
  what	
  it	
  does,	
  but	
  the	
  voice	
  side	
  is	
  a	
  little	
  shaky,”	
  says	
  the	
  
        senior	
  architect	
  for	
  a	
  financial-­‐services	
  firm.	
  
⇒ The	
  average	
  customer-­‐service	
  score	
  of	
  all	
  Market	
  Leaders	
  is	
  4.03;	
  Microsoft’s	
  is	
  
     3.96.	
  
     • “They	
  are	
  very	
  confusing	
  and	
  difficult	
  to	
  work	
  with.	
  Sometimes	
  they	
  want	
  to	
  
        work	
  directly	
  with	
  you	
  and	
  sometimes	
  they	
  want	
  to	
  send	
  you	
  to	
  a	
  partner.	
  
        Sometimes	
  they	
  send	
  you	
  to	
  an	
  account	
  team	
  that	
  gives	
  you	
  wrong	
  
        information,”	
  says	
  the	
  IT	
  director	
  for	
  a	
  global	
  manufacturing	
  firm.	
  
⇒ The	
  average	
  value	
  score	
  of	
  all	
  Market	
  Leaders	
  is	
  4.06;	
  Microsoft’s	
  is	
  4.06.	
  
     • “The	
  fact	
  that	
  we've	
  been	
  able	
  to	
  take	
  this	
  product	
  and	
  offer	
  so	
  many	
  
        capabilities	
  is	
  huge	
  value;	
  never	
  thought	
  I'd	
  say	
  Microsoft	
  and	
  value	
  in	
  the	
  
        same	
  sentence,”	
  says	
  the	
  IT	
  manager	
  for	
  a	
  global	
  technology	
  company.	
  
	
                                       	
  




©Nemertes	
  Research	
  2011	
  ±	
  www.nemertes.com	
  ±	
  888-­‐241-­‐2685	
  ±	
  DN	
  1530	
                                 17	
  
 
	
  

                                                            IBM Lotus Ratings
       4.10




       4.00                                                                                                          4.03



                      3.93
       3.90
                                                                                            3.90
                                                         3.88

       3.80




       3.70




       3.60




       3.50
                    Overall                         Technology                     Customer Service                  Value
                                                                                                                                                   	
  
Chart	
  7:	
  Market	
  Leader:	
  IBM	
  Lotus,	
  Unified	
  Communications,	
  2011	
  

IBM	
  Lotus	
  
⇒ IBM	
  and	
  Overall	
  Scores	
  
IBM	
  improved	
  its	
  scores	
  across	
  the	
  board	
  from	
  2010,	
  with	
  the	
  greatest	
  gain	
  coming	
  
in	
  value,	
  where	
  it	
  rose	
  from	
  a	
  3.56	
  in	
  2010	
  to	
  a	
  4.03	
  in	
  2011,	
  tying	
  Market	
  Leader	
  
runner-­‐up	
  Avaya	
  though	
  still	
  trailing	
  Microsoft.	
  	
  IBM	
  Lotus,	
  like	
  Microsoft,	
  comes	
  at	
  
the	
  UC	
  market	
  from	
  a	
  history	
  of	
  providing	
  messaging	
  and	
  non-­‐real-­‐time	
  
collaboration	
  applications.	
  	
  Unlike	
  Microsoft,	
  Lotus	
  isn’t	
  competing	
  for	
  the	
  
telephony	
  market,	
  rather	
  its	
  strategy	
  is	
  based	
  on	
  delivering	
  presence,	
  Web	
  
conferencing,	
  instant	
  messaging	
  and	
  softphone/video	
  clients	
  that	
  integrate	
  with	
  
standards-­‐based	
  voice/video	
  backend	
  platforms	
  from	
  others.	
  	
  Given	
  its	
  more	
  
narrow	
  focus,	
  it’s	
  not	
  surprising	
  that	
  technology	
  is	
  still	
  the	
  area	
  where	
  Lotus	
  greatly	
  
trails	
  the	
  competition.	
  	
  Customers	
  like	
  the	
  value	
  in	
  what	
  they	
  get	
  for	
  their	
  money,	
  
but	
  they	
  don’t	
  perceive	
  IBM	
  Lotus	
  as	
  a	
  technical	
  leader.	
  
	
  
⇒ The	
  average	
  overall	
  score	
  of	
  all	
  Market	
  Leaders	
  is	
  4.06;	
  IBM’s	
  is	
  3.93.	
  
⇒ The	
  average	
  technology	
  score	
  of	
  all	
  Market	
  Leaders	
  is	
  4.11;	
  IBM’s	
  is	
  3.88.	
  
       • “Great	
  integration	
  with	
  Lotus	
  Notes,	
  good	
  IM,	
  but	
  lousy	
  integration	
  with	
  
            other	
  products,”	
  says	
  the	
  IT	
  architect	
  of	
  a	
  global	
  manufacturing	
  firm.	
  	
  
⇒ The	
  average	
  customer-­‐service	
  score	
  of	
  all	
  Market	
  Leaders	
  is	
  4.03;	
  IBM’s	
  is	
  3.90.	
  


©Nemertes	
  Research	
  2011	
  ±	
  www.nemertes.com	
  ±	
  888-­‐241-­‐2685	
  ±	
  DN	
  1530	
                              18	
  
 
	
  
       “We	
  have	
  IBM	
  Lotus	
  UC	
  products,	
  and	
  they	
  work	
  well,	
  but	
  getting	
  good	
  
       •
       customer	
  support	
  is	
  challenging,”	
  says	
  the	
  IT	
  director	
  of	
  a	
  midsize	
  
       manufacturing	
  company.	
  
⇒ The	
  average	
  value	
  score	
  of	
  all	
  Market	
  Leaders	
  is	
  4.06;	
  IBM’s	
  is	
  4.03.	
  
     • “IBM	
  has	
  a	
  much	
  better	
  licensing	
  strategy,	
  and	
  is	
  less	
  aggressive	
  in	
  auditing	
  
       our	
  licensing,	
  especially	
  compared	
  to	
  Microsoft,”	
  says	
  the	
  director	
  of	
  telecom	
  
       for	
  a	
  global	
  manufacturing	
  firm.	
  
	
  




©Nemertes	
  Research	
  2011	
  ±	
  www.nemertes.com	
  ±	
  888-­‐241-­‐2685	
  ±	
  DN	
  1530	
                 19	
  
 
	
  

P ILOT H OUSE	
   F INALISTS : 	
   M ARKET	
  
C HALLENGERS 	
  
                                Unified Communications: Market Challengers
                                 Siemens                 ShoreTel           NEC             Mitel           Alcatel-Lucent
       4.40
                                                                                                                             4.36
       4.30
                             4.27                              4.27

       4.20                                                                                    4.18


       4.10                                                                                                                  4.08
                                                                                               4.05
       4.00                                                                                                                  4.00
                             3.95
                                                               3.92
       3.90
                                                                                               3.85
                     3.79
       3.80
                    3.78
                                                               3.72                                                          3.72
       3.70                 3.70                                                               3.67
                                                        3.67
                                                                                               3.67                          3.68
                                                        3.64
       3.60


       3.50
                      Overall                           Technology               Customer Service                      Value


	
  Chart	
  8:	
  PilotHouse	
  Market	
  Challengers:	
  Unified	
  Communications,	
  2011	
  
	
                                               	
  




©Nemertes	
  Research	
  2011	
  ±	
  www.nemertes.com	
  ±	
  888-­‐241-­‐2685	
  ±	
  DN	
  1530	
                             20	
  
 
	
  

                                                            ShoreTel Ratings
       4.10

                                                                                                                  4.08

       4.00



                      3.95
       3.90                                              3.92


                                                                                            3.85
       3.80




       3.70




       3.60




       3.50
                    Overall                         Technology                     Customer Service              Value

	
  Chart	
  9:	
  Market	
  Challenger:	
  ShoreTel,	
  Unified	
  Communications,	
  2011	
  

ShoreTel	
  
⇒ IBM	
  and	
  Overall	
  Scores	
  
ShoreTel	
  comes	
  in	
  as	
  the	
  Market	
  Challenger	
  runner	
  up,	
  trailing	
  Siemens	
  in	
  all	
  
categories,	
  but	
  leading	
  all	
  other	
  Market	
  Challengers	
  in	
  every	
  ratings	
  area	
  other	
  than	
  
customer	
  service,	
  where	
  it	
  trails	
  Mitel.	
  	
  As	
  a	
  relative	
  newcomer	
  in	
  the	
  UC	
  space	
  
compared	
  with	
  Siemens,	
  ShoreTel	
  is	
  still	
  increasing	
  its	
  portfolio,	
  recently	
  improving	
  
its	
  mobility	
  and	
  messaging	
  services.	
  ShoreTel’s	
  customers	
  largely	
  praise	
  the	
  value	
  
and	
  feature	
  sets	
  of	
  what	
  they	
  are	
  buying,	
  but	
  ShoreTel	
  must	
  address	
  customer-­‐	
  
service	
  concerns	
  (historically	
  a	
  strong	
  spot	
  for	
  it	
  in	
  the	
  IP	
  telephony	
  market)	
  to	
  
increase	
  its	
  UC	
  success.	
  
⇒ 	
  The	
  average	
  overall	
  score	
  of	
  all	
  Market	
  Challengers	
  is	
  3.90;	
  ShoreTel’s	
  is	
  3.95.	
  
⇒ The	
  average	
  technology	
  score	
  of	
  all	
  Market	
  Challengers	
  is	
  3.84;	
  ShoreTel’s	
  is	
  
       3.92.	
  
       • “ShoreTel’s	
  solution	
  is	
  reliable,	
  and	
  provides	
  a	
  complete	
  feature	
  set,”	
  says	
  the	
  
          manager	
  of	
  a	
  small	
  software	
  firm.	
  
⇒ The	
  average	
  customer-­‐service	
  score	
  of	
  all	
  Market	
  Challengers	
  is	
  3.88	
  ShoreTel’s	
  
       was	
  3.85.	
  
       • “They	
  need	
  to	
  improve	
  on	
  their	
  software	
  engineering	
  practices.	
  It	
  seems	
  like	
  
          there	
  are	
  patches.	
  Fix	
  a	
  few,	
  break	
  a	
  few.	
  And	
  that's	
  caused	
  a	
  little	
  it	
  of	
  


©Nemertes	
  Research	
  2011	
  ±	
  www.nemertes.com	
  ±	
  888-­‐241-­‐2685	
  ±	
  DN	
  1530	
                          21	
  
 
	
  
       frustration	
  with	
  us.	
  You	
  put	
  in	
  a	
  patch,	
  and	
  a	
  new	
  issue.	
  I	
  just	
  patched	
  last	
  
       night	
  after	
  identifying	
  three	
  issues.	
  One,	
  receptionists	
  couldn't	
  get	
  to	
  certain	
  
       calls,”	
  says	
  the	
  CIO	
  of	
  a	
  midsize	
  professional-­‐services	
  company.	
  
⇒ The	
  average	
  value	
  score	
  of	
  all	
  Market	
  Challengers	
  is	
  3.97;	
  ShoreTel’s	
  is	
  4.08.	
  
     • “For	
  us	
  ShoreTel’s	
  cost	
  was	
  the	
  biggest	
  driver,”	
  says	
  the	
  head	
  of	
  IT	
  for	
  a	
  small	
  
       professional-­‐services	
  firm.	
  
	
                                         	
  




©Nemertes	
  Research	
  2011	
  ±	
  www.nemertes.com	
  ±	
  888-­‐241-­‐2685	
  ±	
  DN	
  1530	
                            22	
  
 
	
  

                                                                Mitel Ratings
       4.10



                                                                                            4.05
       4.00




       3.90




       3.80

                      3.79


       3.70

                                                                                                                   3.68

                                                         3.64
       3.60




       3.50
                    Overall                         Technology                     Customer Service               Value
                                                                                                                                             	
  
Chart	
  10:	
  Market	
  Challenger:	
  Mitel,	
  Unified	
  Communications,	
  2011	
  

Mitel	
  
⇒ Mitel	
  and	
  Overall	
  Scores	
  
Save	
  not	
  for	
  the	
  second-­‐highest	
  score	
  among	
  Market	
  Challengers	
  in	
  customer	
  
service,	
  Mitel	
  would	
  have	
  finished	
  last	
  among	
  all	
  others	
  in	
  its	
  category,	
  trailing	
  other	
  
Challengers	
  in	
  value	
  and	
  technology.	
  Although	
  Mitel	
  improved	
  its	
  customer-­‐service	
  
score	
  from	
  2010	
  (3.99)	
  to	
  this	
  year’s	
  4.05,	
  it’s	
  technology	
  score	
  dropped	
  from	
  a	
  4.05	
  
to	
  3.64,	
  and	
  its	
  value	
  score	
  also	
  dropped;	
  going	
  from	
  3.86	
  in	
  2010	
  to	
  3.68	
  in	
  2011.	
  
Despite	
  recent	
  turmoil	
  that	
  have	
  resulted	
  in	
  management	
  changes	
  at	
  the	
  top,	
  these	
  
scores	
  suggest	
  Mitel	
  and	
  its	
  channel	
  partners	
  are	
  overall	
  improving	
  their	
  customer	
  
service	
  and	
  support.	
  	
  If	
  Mitel	
  addresses	
  technology	
  and	
  value	
  concerns,	
  it	
  will	
  
position	
  itself	
  among	
  the	
  top	
  of	
  the	
  Market	
  Challengers.	
  
⇒ The	
  average	
  overall	
  score	
  of	
  all	
  Market	
  Challengers	
  is	
  3.90;	
  Mitel’s	
  is	
  3.79.	
  
⇒ The	
  average	
  technology	
  score	
  of	
  all	
  Market	
  Challengers	
  is	
  3.84;	
  Mitel’s	
  is	
  3.64.	
  
       • “Mitel’s	
  products	
  come	
  in	
  at	
  a	
  competitive	
  price,	
  but	
  their	
  technology	
  is	
  
          behind	
  the	
  competition,”	
  says	
  the	
  IT	
  director	
  for	
  a	
  small	
  education	
  
          organization	
  
⇒ The	
  average	
  customer-­‐service	
  score	
  of	
  all	
  Market	
  Challengers	
  is	
  3.88;	
  Mitel’s	
  is	
  
       4.05.	
  



©Nemertes	
  Research	
  2011	
  ±	
  www.nemertes.com	
  ±	
  888-­‐241-­‐2685	
  ±	
  DN	
  1530	
                           23	
  
 
	
  
       •      “Our	
  voice	
  provider	
  pushed	
  us	
  to	
  go	
  with	
  Mitel,	
  Mitel	
  provides	
  great	
  Level	
  2	
  
              support,”	
  says	
  the	
  IT	
  manager	
  of	
  a	
  midsize	
  healthcare	
  firm.	
  	
  
⇒ The	
  average	
  value	
  score	
  of	
  all	
  Market	
  Challengers	
  is	
  3.97;	
  Mitel’s	
  is	
  3.68.	
  
     • “Mitel	
  won	
  our	
  RFP,	
  coming	
  in	
  $15k	
  less	
  than	
  anyone	
  else,	
  and	
  with	
  more	
  
              functionality,”	
  says	
  the	
  IT	
  manager	
  of	
  a	
  small	
  healthcare	
  firm.	
  
     	
  	
  
	
                                              	
  




©Nemertes	
  Research	
  2011	
  ±	
  www.nemertes.com	
  ±	
  888-­‐241-­‐2685	
  ±	
  DN	
  1530	
                            24	
  
 
	
  

                                                       NEC Ratings
       4.10




       4.00
                                                                                                            4.00


       3.90




       3.80

                    3.78

       3.70


                                                   3.67                           3.67

       3.60




       3.50
                  Overall                     Technology                  Customer Service                  Value
                                                                                                                    	
  
Chart	
  11:	
  Market	
  Challenger:	
  NEC,	
  Unified	
  Communications,	
  2011	
  


                                                       Alcatel-Lucent Ratings
       3.80




                                                   3.72                                                     3.72
       3.70
                    3.70


                                                                                  3.67




       3.60




       3.50
                  Overall                     Technology                  Customer Service                  Value
                                                                                                                    	
  
Chart	
  12:	
  Market	
  Challenger:	
  Alcatel-­‐Lucent,	
  Unified	
  Communications,	
  2011	
  




©Nemertes	
  Research	
  2011	
  ±	
  www.nemertes.com	
  ±	
  888-­‐241-­‐2685	
  ±	
  DN	
  1530	
                    25	
  
 
	
  
O THER	
   M ARKET	
   C HALLENGERS 	
  
	
  
⇒ NEC	
  and	
  Alcatel-­‐Lucent	
  finish	
  as	
  the	
  bottom	
  two	
  finalists	
  in	
  the	
  Market	
  
     Challenger	
  category;	
  NEC	
  did	
  post	
  a	
  strong	
  value	
  score,	
  good	
  for	
  third	
  overall	
  
     among	
  Market	
  Challengers;	
  Alcatel-­‐Lucent	
  struggles	
  in	
  all	
  categories.	
  	
  Compared	
  
     to	
  2010,	
  NEC	
  sees	
  strong	
  improvements	
  in	
  technology	
  (3.43	
  to	
  3.67)	
  and	
  
     customer	
  service	
  (3.50	
  to	
  3.67),	
  while	
  its	
  value	
  score	
  improves	
  by	
  a	
  whopping	
  
     .64	
  (3.36	
  to	
  4.00).	
  	
  If	
  NEC	
  can	
  continue	
  to	
  improve	
  in	
  both	
  technology	
  and	
  
     customer	
  service,	
  it	
  stands	
  a	
  chance	
  of	
  contending	
  for	
  next	
  year’s	
  Market	
  
     Challenger	
  award.	
  
	
  
     Alcatel-­‐Lucent’s	
  scores	
  fall	
  across	
  the	
  board	
  from	
  2010.	
  	
  Technology	
  drops	
  from	
  
     a	
  3.80	
  to	
  3.72,	
  customer	
  service	
  falls	
  from	
  3.96	
  to	
  3.67,	
  and	
  value	
  declines	
  from	
  a	
  
     3.96	
  to	
  a	
  3.72.	
  The	
  overall	
  score	
  thus	
  declines	
  from	
  3.91	
  to	
  3.70,	
  representing	
  the	
  
     biggest	
  decline	
  of	
  any	
  UC	
  vendor.	
  	
  Alcatel-­‐Lucent	
  has	
  its	
  work	
  cut	
  out	
  for	
  it	
  if	
  it	
  
     wishes	
  to	
  improve	
  its	
  market	
  position.	
  
	
  




©Nemertes	
  Research	
  2011	
  ±	
  www.nemertes.com	
  ±	
  888-­‐241-­‐2685	
  ±	
  DN	
  1530	
                                  26	
  
 
	
  

C ONCLUSION 	
  
⇒ The	
  UC	
  market	
  continues	
  to	
  consist	
  of	
  largely	
  three	
  types	
  of	
  vendors:	
  
     • Enterprise	
  telephony	
  companies	
  (Avaya,	
  Cisco)	
  fighting	
  to	
  put	
  their	
  
         applications	
  onto	
  the	
  desktop.	
  
     • Desktop	
  IM/messaging	
  vendors	
  (IBM	
  Lotus,	
  Microsoft)	
  attempting	
  to	
  either	
  
         supplant,	
  or	
  exist	
  alongside	
  telephony	
  vendors.	
  
     • Market	
  Challengers	
  offering	
  integrated	
  solutions	
  aiming	
  to	
  grow	
  market	
  
         share,	
  serve	
  vertical	
  markets	
  and/or	
  size	
  segments,	
  and	
  challenge	
  their	
  
         larger	
  competitors.	
  
⇒ For	
  another	
  year,	
  the	
  telephony	
  centric	
  Market	
  Leaders	
  post	
  the	
  highest	
  overall	
  
     scores,	
  but	
  the	
  gap	
  is	
  narrowing	
  as	
  Microsoft	
  continues	
  to	
  show	
  improvement,	
  
     while	
  IBM	
  Lotus	
  again	
  trails	
  other	
  market	
  leaders	
  overall.	
  	
  Continue	
  to	
  evaluate	
  a	
  
     wide	
  variety	
  of	
  UC	
  vendors,	
  paying	
  attention	
  to	
  areas	
  including	
  customer	
  service,	
  
     long-­‐term	
  vision,	
  value,	
  and	
  demonstrated	
  implementation	
  success.	
  
⇒ Vendor	
  Selection:	
  Based	
  on	
  the	
  outcome	
  of	
  the	
  PilotHouse	
  program,	
  here	
  is	
  
     Nemertes’	
  guidance	
  (with	
  vendors	
  listed	
  in	
  priority	
  order):	
  
     • Evaluate	
  a	
  minimum	
  of	
  four	
  providers.	
  Cisco,	
  Siemens,	
  Avaya	
  and	
  Microsoft	
  
         are	
  solid	
  options.	
  	
  For	
  smaller	
  and	
  mid-­‐size	
  firms,	
  consider	
  ShoreTel,	
  as	
  well.	
  	
  	
  
     • If	
  technology	
  is	
  your	
  key	
  concern,	
  consider	
  Siemens,	
  Cisco,	
  Avaya,	
  Microsoft,	
  
         ShoreTel,	
  and	
  IBM	
  Lotus.	
  
     • If	
  customer-­‐service	
  is	
  your	
  key	
  decision	
  criteria,	
  consider	
  Siemens,	
  Cisco,	
  
         Avaya,	
  and	
  Microsoft.	
  
     • If	
  value	
  is	
  your	
  key	
  goal,	
  consider	
  Siemens,	
  Cisco,	
  ShoreTel,	
  Microsoft,	
  Avaya,	
  
         and	
  NEC.	
  
⇒ Differentiation	
  of	
  Leaders	
  and	
  Challengers:	
  Overall	
  Market	
  Leaders	
  outscored	
  
     Market	
  Challengers	
  across	
  the	
  board.	
  	
  Challengers	
  may	
  offer	
  a	
  better	
  solution	
  for	
  
     small	
  and	
  midsize	
  businesses	
  concerned	
  with	
  getting	
  lost	
  among	
  many	
  larger	
  
     companies.	
  And,	
  based	
  on	
  other	
  services	
  you	
  may	
  have	
  with	
  the	
  Challengers,	
  
     they	
  may	
  be	
  a	
  good	
  fit	
  based	
  on	
  minimum	
  annual	
  revenue	
  commitments.	
  
	
  
⇒ Improvement	
  Outlook:	
  Watch	
  out	
  for	
  continued	
  development	
  of	
  new	
  services	
  as	
  
     vendors	
  expand	
  their	
  offerings	
  into	
  voice	
  and/or	
  desktop	
  collaboration.	
  	
  Also	
  
     keep	
  your	
  eye	
  on	
  expanding	
  hosted	
  offerings	
  that	
  will	
  increasingly	
  provide	
  an	
  
     alternative	
  (or	
  complement)	
  to	
  the	
  on-­‐premises	
  solutions	
  evaluated	
  for	
  this	
  
     award.	
  
	
  
	
  
	
  
	
  




©Nemertes	
  Research	
  2011	
  ±	
  www.nemertes.com	
  ±	
  888-­‐241-­‐2685	
  ±	
  DN	
  1530	
                             27	
  
 
	
  

M ETHODOLOGY 	
  
The	
  population	
  includes	
  individuals	
  primarily	
  from	
  U.S.	
  companies	
  (based	
  in	
  the	
  
U.S.,	
  but	
  many	
  of	
  which	
  are	
  global	
  multinationals)	
  who	
  are	
  responsible	
  for	
  selecting,	
  
or	
  influencing	
  the	
  selection	
  of,	
  suppliers	
  of	
  data-­‐center	
  and	
  communications	
  
products	
  and	
  services.	
  	
  

Sample	
  Frame	
  
In	
  selecting	
  the	
  sampling	
  frame,	
  Nemertes	
  has	
  asked	
  individuals	
  in	
  the	
  following	
  
populations	
  to	
  rate	
  their	
  providers:	
  
       ± U.S.	
   business	
   subscriber	
   lists,	
   including	
   individuals	
   who	
   have	
   opted	
   to	
  
           participate	
   in	
   surveys	
   and	
   who	
   have	
   been	
   pre-­‐screened	
   to	
   determine	
  
           responsibility	
  for	
  selecting	
  or	
  influencing	
  relevant	
  products	
  and	
  services.	
  
       ± Nemertes	
   Research	
   IT	
   executive	
   database,	
   limited	
   to	
   individuals	
   who	
   meet	
  
           the	
   criteria	
   for	
   the	
   representative	
   population.	
   Individuals	
   from	
   this	
   list	
  
           represent	
   primarily	
   U.S.	
   companies,	
   but	
   also	
   include	
   companies	
   based	
  
           elsewhere	
   that	
   have	
   presence	
   in	
   North	
   America.	
   The	
   database	
   includes	
  
           individuals	
   who	
   have	
   participated	
   in,	
   or	
   who	
   have	
   expressed	
   interest	
   in	
  
           participating	
   in	
   our	
   research,	
   or	
   with	
   whom	
   Nemertes’	
   analysts	
   have	
  
           established	
  a	
  business	
  relationship.
Individuals	
  participated	
  in	
  this	
  project	
  using	
  three	
  methods:	
  
       ± Web-­‐based	
  survey.	
  This	
  is	
  the	
  largest	
  percentage	
  of	
  the	
  respondents.	
  Those	
  
           who	
   meet	
   the	
   sample	
   frame	
   randomly	
   received	
   invitations	
   to	
   participate	
   in	
  
           the	
  survey.	
  
       ± Visitors	
   to	
   Nemertes’	
   Web	
   site,	
   and	
   recipients	
   of	
   Nemertes’	
   blogs	
   and	
  
           columns	
   in	
   third-­‐party	
   media	
   partners’	
   Web	
   sites.	
   They	
   must	
   meet	
   the	
  
           criteria	
  to	
  participate.	
  	
  
       ± Benchmark	
   interviews.	
   This	
   is	
   a	
   smaller	
   percentage	
   of	
   the	
   respondents.	
  
           Nemertes’	
   analysts	
   asked	
   numerous	
   detailed	
   qualitative	
   questions	
   to	
   gauge	
  
           why	
   they	
   rated	
   their	
   service	
   providers	
   the	
   way	
   they	
   did,	
   as	
   well	
   as	
   gathering	
  
           other	
  information	
  about	
  their	
  usage	
  of	
  communications	
  services.	
  	
  	
  
Benchmark	
  participants	
  spent	
  one	
  to	
  three	
  hours	
  on	
  the	
  phone	
  or	
  in	
  person	
  with	
  a	
  
Nemertes	
  analyst	
  discussing	
  issues	
  relating	
  to	
  their	
  use	
  of	
  products	
  and	
  services.	
  
The	
  Web-­‐based	
  survey	
  participants	
  answered	
  a	
  subset	
  of	
  the	
  benchmark	
  questions	
  
that	
  focus	
  on	
  rating	
  the	
  providers,	
  stack-­‐ranking	
  important	
  criteria,	
  providing	
  
financial	
  data,	
  open-­‐ended	
  comments,	
  and	
  demographics.	
  

Planned	
  Sample	
  Size	
  

According	
  to	
  U.S.	
  Census	
  Bureau	
  figures,	
  there	
  are	
  2,306,070	
  companies	
  with	
  five	
  or	
  
more	
  employees.	
  Our	
  goal	
  was	
  to	
  receive	
  responses	
  from	
  a	
  minimum	
  of	
  1,000	
  



©Nemertes	
  Research	
  2011	
  ±	
  www.nemertes.com	
  ±	
  888-­‐241-­‐2685	
  ±	
  DN	
  1530	
                                   28	
  
 
	
  
individuals,	
  which	
  would	
  give	
  us	
  a	
  95%	
  confidence	
  level	
  and	
  3%	
  margin	
  of	
  error—if	
  
every	
  individual	
  rated	
  every	
  vendor	
  in	
  every	
  technology	
  area	
  rated.	
  We	
  received	
  
substantial	
  ratings	
  for	
  each	
  technology	
  category	
  (several	
  hundred	
  per	
  category),	
  but	
  
each	
  vendor	
  in	
  each	
  area	
  did	
  not	
  receive	
  a	
  rating	
  from	
  every	
  research	
  participant.	
  

About	
  4,000	
  individuals	
  accessed	
  the	
  survey	
  or	
  participated	
  in	
  a	
  benchmark	
  
interview.	
  Of	
  those,	
  about	
  2,000	
  meet	
  Nemertes’	
  standards	
  to	
  be	
  considered	
  “valid.”	
  
Our	
  survey	
  tool	
  automatically	
  exited	
  individuals	
  employed	
  by	
  IT	
  vendors	
  and	
  
providers.	
  Analysts	
  reviewed	
  all	
  other	
  ratings	
  (survey	
  and	
  benchmark)	
  line	
  by	
  line,	
  
and	
  categorized	
  as	
  “invalid”	
  those	
  who	
  demonstrated	
  inconsistencies	
  or	
  
inaccuracies	
  in	
  their	
  responses	
  as	
  part	
  of	
  Nemertes’	
  complex	
  qualification	
  
methodology.	
  

We	
  achieved	
  validity	
  across	
  the	
  survey	
  and	
  interviews	
  by	
  ensuring	
  the	
  questions	
  we	
  
asked	
  were	
  the	
  same	
  and	
  that	
  the	
  interview	
  group	
  and	
  survey	
  group	
  represent	
  
discrete	
  samples	
  of	
  the	
  same	
  population.	
  Nemertes	
  achieves	
  survey	
  and	
  interview	
  
consistency	
  through	
  the	
  use	
  of	
  pre-­‐scripted	
  interview	
  forms	
  and	
  peer	
  review	
  of	
  
interview	
  protocols.	
  Analysts	
  also	
  relied	
  upon	
  their	
  own	
  knowledge	
  of	
  the	
  
technology	
  areas,	
  natural	
  breakpoints	
  in	
  the	
  data,	
  and	
  interview	
  notes	
  from	
  the	
  
survey	
  participants	
  to	
  further	
  validate	
  ratings.	
  

Survey	
  Sub-­‐Groups/Stratification	
  

Nemertes’	
  analysts	
  researched	
  which	
  providers	
  offer	
  products	
  and	
  services	
  in	
  each	
  
category	
  and	
  created	
  lists	
  from	
  which	
  participants	
  identified	
  their	
  primary	
  service	
  
providers.	
  Participants	
  also	
  were	
  able	
  to	
  select	
  “other,”	
  and	
  identify	
  a	
  service	
  
provider	
  they	
  use	
  that	
  may	
  not	
  be	
  included	
  on	
  the	
  explicit	
  list	
  provided.	
  

The	
  challenge	
  is	
  that	
  some	
  providers	
  (Market	
  Leaders)	
  have	
  thousands	
  of	
  business	
  
customers	
  and	
  significant	
  market	
  share,	
  while	
  others	
  (Market	
  Challengers)	
  have	
  a	
  
few	
  hundred	
  or	
  few	
  thousand	
  customers	
  and	
  smaller	
  market	
  share.	
  We	
  realized	
  
some	
  providers	
  would	
  garner	
  a	
  relatively	
  large	
  number	
  of	
  ratings,	
  based	
  on	
  the	
  
number	
  of	
  customers	
  they	
  have,	
  while	
  others	
  would	
  have	
  a	
  relatively	
  small	
  number	
  
of	
  ratings.	
  	
  

Therefore,	
  we	
  created	
  the	
  two	
  distinct	
  categories	
  for	
  the	
  awards,	
  Market	
  Leaders	
  
and	
  Market	
  Challengers,	
  and	
  compared	
  providers	
  within	
  each	
  category.	
  Nemertes	
  
placed	
  providers	
  within	
  each	
  category	
  based	
  on	
  its	
  own	
  research	
  and	
  publicly	
  
available	
  data.	
  Analysts	
  also	
  examined	
  natural	
  breakpoints	
  in	
  the	
  data.	
  Market	
  
Leaders	
  typically	
  have	
  >10%	
  of	
  market	
  share,	
  based	
  on	
  these	
  analyses.	
  Market	
  
Challengers	
  typically	
  have	
  smaller	
  market	
  shares.	
  In	
  some	
  categories,	
  there	
  were	
  
not	
  enough	
  ratings	
  to	
  issue	
  an	
  award	
  in	
  the	
  Market	
  Challenger	
  category,	
  or	
  the	
  



©Nemertes	
  Research	
  2011	
  ±	
  www.nemertes.com	
  ±	
  888-­‐241-­‐2685	
  ±	
  DN	
  1530	
                      29	
  
 
	
  
market	
  is	
  so	
  new	
  that	
  all	
  vendors	
  are	
  considered	
  Challengers.	
  In	
  these	
  cases,	
  
Nemertes	
  issues	
  an	
  award	
  only	
  in	
  the	
  appropriate	
  category.	
  	
  

Nemertes	
  reserves	
  the	
  right	
  to	
  address	
  acquisitions	
  occurring	
  during	
  the	
  
benchmark	
  and	
  survey	
  period	
  on	
  a	
  case-­‐by-­‐case	
  basis.	
  	
  Unless	
  otherwise	
  noted,	
  an	
  
acquisition	
  merging	
  two	
  companies	
  in	
  the	
  same	
  award	
  category	
  must	
  be	
  complete	
  
before	
  the	
  start	
  of	
  the	
  survey	
  and	
  benchmark	
  interview	
  period	
  to	
  be	
  counted	
  as	
  one	
  
company	
  in	
  the	
  ratings.	
  




©Nemertes	
  Research	
  2011	
  ±	
  www.nemertes.com	
  ±	
  888-­‐241-­‐2685	
  ±	
  DN	
  1530	
                     30	
  
 
	
  

Awards	
  
Nemertes	
  is	
  issuing	
  awards	
  in	
  the	
  following	
  categories:	
  
                                    Nemertes PilotHouse Awards, 2011
                    Award Category                                Market Leaders                    Market Challengers
         Advanced Communications Services
         MPLS Services                                                         ü                              ü
         Carrier Ethernet Services                                             ü                              ü
         Internet Access Services                                              ü                              ü
         SIP Trunking Services                                                 ü                           No award
         Managed Router Services                                               ü                              ü
         Managed Internet Services                                             ü                              ü
         Wireless and Mobility
         Wireless LANs                                                         ü                              ü
         Wireless Voice & Data Services                                        ü                              ü
         Application Delivery
         Application Delivery Optimization                                     ü                              ü
         Virtual Desktops                                                      ü                              ü
         Voice Communications
         IP Telephony                                                          ü                              ü
         Managed IP Telephony                                                  ü                              ü
         Hosted Voice Over IP                                                  ü                           No award
         Data-Center Technologies
         Servers for Virtualization                                            ü                              ü
         Storage for Virtualization                                            ü                              ü
         Data-Center Colocation                                                ü                              ü
         Unified Communications
         Unified Communications                                                ü                              ü
         IP Contact Centers                                                    ü                              ü
         Security
         Managed Firewall/IDS/IPS                                              ü                              ü
         Data-Center Firewalls                                                 ü                              ü
         Small Branch Firewalls                                                ü                              ü
         Cloud
         Software as a Service: Office                                         ü                           No award


	
  


©Nemertes	
  Research	
  2011	
  ±	
  www.nemertes.com	
  ±	
  888-­‐241-­‐2685	
  ±	
  DN	
  1530	
                  31	
  
 
	
  
Timing	
  
The	
  Web-­‐based	
  survey	
  was	
  conducted	
  between	
  March	
  and	
  May	
  2011.	
  The	
  
benchmark	
  research	
  was	
  conducted	
  between	
  January	
  and	
  April	
  2011.	
  

Incentives	
  to	
  Participate	
  &	
  Time	
  Commitment	
  
Participants	
  of	
  the	
  Web-­‐based	
  survey	
  received	
  a	
  small	
  incentive	
  for	
  participating	
  in	
  
the	
  survey.	
  Participants	
  from	
  Nemertes’	
  database	
  receive	
  the	
  findings	
  and	
  are	
  
invited	
  to	
  participate	
  in	
  a	
  Webcast,	
  in	
  exchange	
  for	
  their	
  time.	
  The	
  Web-­‐based	
  
survey	
  takes	
  about	
  15	
  minutes	
  to	
  complete;	
  the	
  benchmark	
  requires	
  one	
  to	
  three	
  
hours	
  of	
  participants’	
  time.	
  

Future	
  Plans	
  
Nemertes	
  plans	
  to	
  conduct	
  its	
  PilotHouse	
  Awards	
  program	
  annually,	
  though	
  it	
  
retains	
  the	
  right	
  to	
  cancel	
  the	
  project	
  at	
  any	
  time.	
  
	
  
	
  
About	
  Nemertes	
  Research:	
  	
  
Nemertes	
  Research	
  is	
  a	
  research-­‐advisory	
  and	
  strategic-­‐consulting	
  firm	
  that	
  
specializes	
  in	
  analyzing	
  and	
  quantifying	
  the	
  business	
  value	
  of	
  emerging	
  
technologies.	
  You	
  can	
  learn	
  more	
  about	
  Nemertes	
  Research	
  at	
  our	
  Website:	
  
http://www.nemertes.com	
  

	
  
	
  
	
  
	
  




©Nemertes	
  Research	
  2011	
  ±	
  www.nemertes.com	
  ±	
  888-­‐241-­‐2685	
  ±	
  DN	
  1530	
                 32	
  

More Related Content

Similar to Nemertes Pilot House Awards UC 2011 12

Nemertes Research Report
Nemertes Research ReportNemertes Research Report
Nemertes Research ReportLanair
 
Net Neutrality 2.0 - Lubricate The Market
Net Neutrality 2.0 - Lubricate The MarketNet Neutrality 2.0 - Lubricate The Market
Net Neutrality 2.0 - Lubricate The Marketskripnikov
 
OST Energy - Global Services
OST Energy - Global Services OST Energy - Global Services
OST Energy - Global Services Sion Haswell
 
3 openerp hr-book.complete
3 openerp hr-book.complete3 openerp hr-book.complete
3 openerp hr-book.completeopenerpwiki
 
Resource Paper of Enterprise-Wide Deployment of EDM
Resource Paper of Enterprise-Wide Deployment of EDMResource Paper of Enterprise-Wide Deployment of EDM
Resource Paper of Enterprise-Wide Deployment of EDMGlen Alleman
 
Shahed.Anwar 061708556
Shahed.Anwar 061708556Shahed.Anwar 061708556
Shahed.Anwar 061708556mashiur
 
Factors Influencing Brand Switching in Telecommunication Industry of United K...
Factors Influencing Brand Switching in Telecommunication Industry of United K...Factors Influencing Brand Switching in Telecommunication Industry of United K...
Factors Influencing Brand Switching in Telecommunication Industry of United K...ghostwriter ghostwritingmania@yahoo.com
 
SAP Business Objects Planning and Consolidaton, Version for SAP NetWeaver
SAP Business Objects Planning and Consolidaton, Version for SAP NetWeaverSAP Business Objects Planning and Consolidaton, Version for SAP NetWeaver
SAP Business Objects Planning and Consolidaton, Version for SAP NetWeaverIBM India Smarter Computing
 
Nintex Workflow for Sharepoint - Return on Investment Whitepaper by Forrester...
Nintex Workflow for Sharepoint - Return on Investment Whitepaper by Forrester...Nintex Workflow for Sharepoint - Return on Investment Whitepaper by Forrester...
Nintex Workflow for Sharepoint - Return on Investment Whitepaper by Forrester...David J Rosenthal
 
Not all XML Gateways are Created Equal
Not all XML Gateways are Created EqualNot all XML Gateways are Created Equal
Not all XML Gateways are Created EqualCA API Management
 
How to choose the right penetration testing company
How to choose the right penetration testing companyHow to choose the right penetration testing company
How to choose the right penetration testing companyPenetration_Testing
 

Similar to Nemertes Pilot House Awards UC 2011 12 (20)

Nemertes Research Report
Nemertes Research ReportNemertes Research Report
Nemertes Research Report
 
Skelta Awards & Accolades
Skelta Awards & AccoladesSkelta Awards & Accolades
Skelta Awards & Accolades
 
ISO_6
ISO_6ISO_6
ISO_6
 
Net Neutrality 2.0 - Lubricate The Market
Net Neutrality 2.0 - Lubricate The MarketNet Neutrality 2.0 - Lubricate The Market
Net Neutrality 2.0 - Lubricate The Market
 
OST Energy - Global Services
OST Energy - Global Services OST Energy - Global Services
OST Energy - Global Services
 
Password Management Project Roadmap
Password Management Project RoadmapPassword Management Project Roadmap
Password Management Project Roadmap
 
3 openerp hr-book.complete
3 openerp hr-book.complete3 openerp hr-book.complete
3 openerp hr-book.complete
 
Resource Paper of Enterprise-Wide Deployment of EDM
Resource Paper of Enterprise-Wide Deployment of EDMResource Paper of Enterprise-Wide Deployment of EDM
Resource Paper of Enterprise-Wide Deployment of EDM
 
Shahed.Anwar 061708556
Shahed.Anwar 061708556Shahed.Anwar 061708556
Shahed.Anwar 061708556
 
NCS
NCSNCS
NCS
 
Decumentation
DecumentationDecumentation
Decumentation
 
Factors Influencing Brand Switching in Telecommunication Industry of United K...
Factors Influencing Brand Switching in Telecommunication Industry of United K...Factors Influencing Brand Switching in Telecommunication Industry of United K...
Factors Influencing Brand Switching in Telecommunication Industry of United K...
 
LyRex Business Plan
LyRex Business PlanLyRex Business Plan
LyRex Business Plan
 
SAP Business Objects Planning and Consolidaton, Version for SAP NetWeaver
SAP Business Objects Planning and Consolidaton, Version for SAP NetWeaverSAP Business Objects Planning and Consolidaton, Version for SAP NetWeaver
SAP Business Objects Planning and Consolidaton, Version for SAP NetWeaver
 
Nintex Workflow for Sharepoint - Return on Investment Whitepaper by Forrester...
Nintex Workflow for Sharepoint - Return on Investment Whitepaper by Forrester...Nintex Workflow for Sharepoint - Return on Investment Whitepaper by Forrester...
Nintex Workflow for Sharepoint - Return on Investment Whitepaper by Forrester...
 
Not all XML Gateways are Created Equal
Not all XML Gateways are Created EqualNot all XML Gateways are Created Equal
Not all XML Gateways are Created Equal
 
How to choose the right penetration testing company
How to choose the right penetration testing companyHow to choose the right penetration testing company
How to choose the right penetration testing company
 
Electronics industry brief
Electronics industry briefElectronics industry brief
Electronics industry brief
 
Euro Outsourcing Markets
Euro Outsourcing MarketsEuro Outsourcing Markets
Euro Outsourcing Markets
 
Identity Management Project Roadmap
Identity Management Project RoadmapIdentity Management Project Roadmap
Identity Management Project Roadmap
 

Recently uploaded

Testing tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesTesting tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesKari Kakkonen
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.Curtis Poe
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxLoriGlavin3
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfLoriGlavin3
 
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...Wes McKinney
 
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfNeo4j
 
2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch TuesdayIvanti
 
Assure Ecommerce and Retail Operations Uptime with ThousandEyes
Assure Ecommerce and Retail Operations Uptime with ThousandEyesAssure Ecommerce and Retail Operations Uptime with ThousandEyes
Assure Ecommerce and Retail Operations Uptime with ThousandEyesThousandEyes
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Hiroshi SHIBATA
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxLoriGlavin3
 
Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Farhan Tariq
 
Data governance with Unity Catalog Presentation
Data governance with Unity Catalog PresentationData governance with Unity Catalog Presentation
Data governance with Unity Catalog PresentationKnoldus Inc.
 
Decarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityDecarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityIES VE
 
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentEmixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentPim van der Noll
 
Time Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsTime Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsNathaniel Shimoni
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsSergiu Bodiu
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .Alan Dix
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024Lonnie McRorey
 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
Potential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsPotential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsRavi Sanghani
 

Recently uploaded (20)

Testing tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesTesting tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examples
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdf
 
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
 
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdf
 
2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch Tuesday
 
Assure Ecommerce and Retail Operations Uptime with ThousandEyes
Assure Ecommerce and Retail Operations Uptime with ThousandEyesAssure Ecommerce and Retail Operations Uptime with ThousandEyes
Assure Ecommerce and Retail Operations Uptime with ThousandEyes
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
 
Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...
 
Data governance with Unity Catalog Presentation
Data governance with Unity Catalog PresentationData governance with Unity Catalog Presentation
Data governance with Unity Catalog Presentation
 
Decarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityDecarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a reality
 
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentEmixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
 
Time Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsTime Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directions
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024
 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
Potential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsPotential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and Insights
 

Nemertes Pilot House Awards UC 2011 12

  • 1.   11     Q3       Nemertes  Research  PilotHouse  Awards   Unified  Communications   The  Nemertes  Research  annual  PilotHouse  Awards  provide  insight  on  the  performance  of  technology   vendors,  according  to  feedback  from  IT  decision-­‐makers  who  use  their  products  or  services.             N e m e r t e s   R e s e a r c h               w w w . n e m e r t e s . c o m             + 1   8 8 8 . 2 4 1 . 2 6 8 5  
  • 2.       T ABLE  OF   C ONTENTS   Unified  Communications  .......................................................................................................  3   Award  Definition  ...............................................................................................................................  3   Overview  ..............................................................................................................................................  3   Market  Classification  .......................................................................................................................  3   Ratings  ..................................................................................................................................................  4   Ratings  Categories  ............................................................................................................................  5   Technology  .........................................................................................................................................................  5   Customer  Service  .............................................................................................................................................  5   ......................................................................................................................................................................  5   Value   Results  Summary  ...............................................................................................................................  6   Analysis  .......................................................................................................................................  7   PilotHouse  Market  Challenger  Winner  ............................................................................  9   Siemens  .................................................................................................................................................  9   Technology  ......................................................................................................................................................  10   Customer  Service  ..........................................................................................................................................  10   ...................................................................................................................................................................  10   Value   PilotHouse  Market  Leader  Winner  .................................................................................  11   Cisco  ....................................................................................................................................................  11   Technology  ......................................................................................................................................................  12   Customer  Service  ..........................................................................................................................................  12   Value   ...................................................................................................................................................................  12   Pilothouse  Finalists:  Market  Leaders  ............................................................................  13   Avaya  ..................................................................................................................................................  14   Microsoft  ...........................................................................................................................................  16   IBM  Lotus  ..........................................................................................................................................  18   PilotHouse  Finalists:  Market  Challengers  ....................................................................  20   ShoreTel  ............................................................................................................................................  21   Mitel  ....................................................................................................................................................  23   Other  Market  Challengers  .................................................................................................  26   Conclusion  ...............................................................................................................................  27   Methodology  ...........................................................................................................................  28   Sample  Frame  .................................................................................  Error!  Bookmark  not  defined.   Planned  Sample  Size  ......................................................................  Error!  Bookmark  not  defined.   Survey  Sub-­‐Groups/Stratification  ............................................  Error!  Bookmark  not  defined.   Awards  ..............................................................................................  Error!  Bookmark  not  defined.   Timing  ...............................................................................................  Error!  Bookmark  not  defined.   Incentives  to  Participate  &  Time  Commitment  ..................  Error!  Bookmark  not  defined.   Future  Plans  .......................................................................................  Error!  Bookmark  not  defined.   ©Nemertes  Research  2011  ±  www.nemertes.com  ±  888-­‐241-­‐2685  ±  DN  1530   2  
  • 3.       U NIFIED   C OMMUNICATIONS     By  Irwin  Lazar   VP  and  Service  Director,  Nemertes  Research     Award  Definition   The  Nemertes  PilotHouse  award  for  Unified  Communications  (UC)  recognizes  vendors   identified   and   rated   by   IT   professionals   as   their   strategic   partner   for   delivering   UC   products,   which   integrate   voice,   video,   conferencing,   messaging,   and   presence   with   office   and   business-­‐process   applications   to   improve   collaboration.   IT   professionals   who   use   these   services   rated   their   providers   on   technology,   value,   and   customer   service.   Overview   ± The   goal   of   PilotHouse   awards   is   to   provide   analysis   of   vendor   and   service-­‐ provider  performance  from  the  perspective  of  their  business  users.   ± Many   research   firms   offer   market   ranking;   Nemertes’   research   and   analysis   is   unique,   based   100%   on   the   views   and   experience   of   actual   Unified   Communications  users.   ± Research  is  wholly  independent  and  not  sponsored;  Nemertes  has  no  influence   over  vendor  or  service  provider  performance.   ± Opinions   are   those   of   the   IT   professionals   who   have   selected,   designed   and   deployed  the  technology  or  service.     ± By   combining   benchmarking   (direct   user   interviews)   and   surveys,   Nemertes   is   able  to  provide  unique  insight  into  why  IT  professionals  rated  vendors  the  way   they  did.     For  this  award,  Nemertes  gathered  ratings  on  UC  system  and  application  providers   with   a   range   of   offerings.   (More   detail   on   the   program,   and   demographics   of   participating   IT   professionals   is   available   in   the   methodology   at   the   end   of   this   report.)       Market  Classification   We   segmented   UC   providers   into   two   categories:   Market   Leaders   and   Market   Challengers,   and   offered   awards   within   each   category.   To   determine   the   categorizations,   Nemertes’   analysts   evaluated   UC   market   presence   (looking   at   revenue,   device   shipments,   and   number   of   customers)   based   on   our   own   research   ©Nemertes  Research  2011  ±  www.nemertes.com  ±  888-­‐241-­‐2685  ±  DN  1530   3  
  • 4.     and  publicly  available  data.  Analysts  also  examined  natural  breakpoints  in  the  data,   and   segmented   the   Market   Leaders   as   those   who   collectively   accounted   for   the   vast   majority   of   each   market,   and   Market   Challengers   who   accounted   for   a   smaller   percentage  of  the  overall  market.     UNIFIED COMMUNICATIONS Market Leaders Market Challengers Alcatel-Lucent, Mitel, NEC, ShoreTel, Avaya, Cisco, IBM Lotus, Microsoft Siemens Table  1:  Vendor  Classification,  Unified  Communications,  2011     Nemertes  defines  the  UC  market  as  “segmented,”  meaning  that  no  vendor  controls   more  than  30%  of  market  share,  and  no  two  vendors  control  more  than  50%.    As   published  in  various  public  sources,  Cisco,  Avaya,  IBM  Lotus,  and  Microsoft  account   for  more  than  50%  of  the  UC  market.    However  the  UC  market  is  often  difficult  to   classify,  thanks  to  varying  definitions  of  UC.    Nemertes  defines  UC  as  the  integration   of  various  forms  of  real-­‐time  and  non-­‐real  time  collaboration  (e.g.  voice,  video,   messaging,  conferencing)  into  a  set  of  applications  sharing  presence,  and  enabling   establishment  of  any  mode  of  communications  (e.g.  escalating  an  IM  into  a  phone   call,  video  chat,  or  Web  conference,  all  through  the  same  user  interface).    Vendors   typically  classify  any  of  their  standalone  offerings  in  the  UC  space  (e.g.  voice,  unified   messaging,  Web  conferencing,  etc.)  as  “UC,”  making  it  difficult  to  determine  specific   components  of  a  UC  implementation.     The  Market  Leaders  reflect  the  dualities  of  the  UC  market:  Those  with  large  market   share  in  voice  (Avaya,  Cisco)  that  are  broadening  into  the  desktop;  and  those  with   large  market  share  in  desktop  collaboration  (IBM  Lotus,  Microsoft)  that  are   broadening  into  voice.    Market  Challengers  have  smaller  market  shares,  or  have   traditionally  focused  on  the  small/midsize  business  market  or  a  limited  set  of   verticals.    It’s  worth  noting  that  our  data  set  predominantly  reflects  U.S.-­‐centric   enterprises,  thus  we  classify  vendors  such  as  Alcatel-­‐Lucent  and  Siemens,  with  large   market  share  outside  the  U.S.,  as  Challengers.   Ratings   We  asked  IT  professionals  to  rate  UC  providers  using  a  5-­‐point  scale,  where  5  is   excellent,  4  is  good,  3  is  fair,  2  is  poor,  and  1  is  unacceptable.  Nemertes  then  used   these  raw  scores  to  compute  average  scores  for  each  category.         ©Nemertes  Research  2011  ±  www.nemertes.com  ±  888-­‐241-­‐2685  ±  DN  1530   4  
  • 5.     The  maximum  possible  score  is  a  5.0  (and  although  some  vendors  did  receive   perfect  scores  from  individual  IT  participants,  none  received  a  perfect  score  when   all  ratings  were  averaged).   Ratings  Categories   The  participants  rated  their  technology  providers  in  three  areas:     ⇒ Technology   ⇒ Customer  Service   ⇒ Value   ⇒ Overall  Rating  (average  of  Technology,  Customer  Service,  and  Value)   Technology   Technology   ratings   gauge   how   customers   view   the   sophistication,   features,   and   implementation   of   the   UC   product   they’re   rating.   Additionally,   this   score   reflects   how   much   of   a   leader   a   vendor   is   in   the   UC   industry,   from   the   perspective   of   the   customer.   Customer  Service   Customer-­‐service   ratings   cover   how   providers   perform   in   areas   such   as   technical   support,  responsiveness  to  deployment  problems  and  concerns,  sales  support,  and   general   customer   care.   Additionally,   technology   users   considered   the   willingness   and  ability  of  the  vendors  to  answer  questions  effectively  and  promptly.   Value   Value   ratings   are   essentially   the   way   customers   perceive   what   they   get   for   what   they  pay  for.  In  other  words,  are  they  getting  their  “bang  for  the  buck?”     ©Nemertes  Research  2011  ±  www.nemertes.com  ±  888-­‐241-­‐2685  ±  DN  1530   5  
  • 6.     Results  Summary   Unified Communications: All Vendors, Overall Scores 4.30 4.27 4.20 4.17 4.10 4.11 4.00 4.05 3.90 3.95 3.93 3.80 3.79 3.78 3.70 3.70 3.60 3.50 s co ya l s l EC nt t Te ite en tu of e a is e N M Lo uc os Av em or C l-L r Sh M ic Si IB te M a lc A Market Leaders Market Challengers Chart  1:  Overall  Scores,  Unified  Communications,  2011     ⇒ Among  Market  Challengers,  Siemens  wins  the  PilotHouse  Award.   • Siemens’  overall  score  is  4.27.     ⇒ Among  Market  Leaders,  Cisco  wins  the  PilotHouse  Award.   • Cisco’s  overall  score  is  4.17.     ⇒ A  total  of  nine  providers  received  enough  responses  for  us  to  include  them  in   this  year’s  analysis.   • Four  providers  are  Market  Leaders;  five  are  Market  Challengers.   • Market  Leaders’  overall  average  score  is  4.06.   • Market  Challengers’  overall  average  score  is  3.90.   ©Nemertes  Research  2011  ±  www.nemertes.com  ±  888-­‐241-­‐2685  ±  DN  1530   6  
  • 7.     A NALYSIS   Nemertes 2011 PilotHouse Awards Unified Communications Market Leaders Market Challengers Avaya, Cisco, IBM Lotus, Microsoft Alcatel-Lucent, Mitel, NEC, ShoreTel, Siemens Customer Winners Overall Technology Service Value Siemens 4.27 4.27 4.18 4.36 Cisco 4.17 4.23 4.16 4.12 Other Finalists Avaya 4.11 4.19 4.10 4.03 Microsoft 4.05 4.13 3.96 4.06 ShoreTel 3.95 3.92 3.85 4.08 IBM Lotus 3.93 3.88 3.90 4.03 Mitel 3.79 3.64 4.05 3.68 NEC 3.78 3.67 3.67 4.00 Alcatel-Lucent 3.70 3.72 3.67 3.72 Rating Scale: 5=Excellent; 4=Good; 3=Fair; 2=Poor; 1=Unacceptable Table  2:  Vendor  Scores,  Unified  Communications,  2011   Overall,  scores  for  UC  lagged  slightly  behind  those  for  other  technology  areas,  with   only  four  vendors  scoring  higher  than  a  4.0.    Winners  significantly  outperformed   non-­‐winners,  especially  in  the  Market  Challenger  category  where  Siemens  was  the   only  vendor  to  score  higher  than  a  4.0  in  all  categories.     Notable  is  the  lack  of  any  consistency  of  scores  among  ratings  categories.    Cisco,  for   example  saw  its  highest  score  in  technology,  while  Siemens  scored  best  in  value.     Absent  are  any  consistently  poor  scores  in  a  single  category  (e.g.  “customer  service   consistently  lagging  behind  value”).         What  does  this  mean?    Each  vendor  has  strengths  and  weaknesses  that  it  must  focus   on  improving,  rather  than  a  weak  area  (or  areas)  existing  for  the  entire  UC   marketplace.      Cisco’s  biggest  weakness  (customer  service)  isn’t  the  same  as  Avaya’s   (value)  or  IBM  Lotus’  (technology).  It  also  reflects  the  relative  “newness”  of  the  UC   market  and  the  fact  that  it’s  rapidly  changing  in  terms  of  product  capabilities.     Like  last  year,  those  who  approach  the  UC  market  from  a  telephony  perspective   (Cisco,  Avaya)  outscored  those  who  have  moved  from  IM/messaging  into  telephony   ©Nemertes  Research  2011  ±  www.nemertes.com  ±  888-­‐241-­‐2685  ±  DN  1530   7  
  • 8.     (Microsoft,  IBM  Lotus).    This  reflects  the  continued  struggle  for  IM/messaging   vendors  to  play  in  a  space  where  customers  focus  far  more  attention  on  reliability,   resiliency,  performance,  and  support  than  in  the  desktop  applications  space.    This   year,  like  last,  both  UC  winners  come  from  telephony  backgrounds.    IM/messaging   vendors  must  continue  their  efforts  to  demonstrate  to  IT  leaders  that  they   understand  the  real-­‐time  application  requirements,  of  telephony  and  increasingly   video.     ⇒ Overall,  Market  Leaders  score  a  4.06;  Challengers  earn  a  3.90.    Siemens  is  the   exception  to  the  rule,  outperforming  all  vendors  regardless  of  size.   • Reason:  The  primary  gaps  between  Market  Leaders  and  Market  Challengers   exists  in  technology  and  customer  service.    Market  Leaders  have  deeper   pockets,  larger  support  networks,  and  a  large  installed  base  from  which  they   can  cross-­‐sell;  using  bundled  licensing  programs  to  deliver  UC  as  an  add-­‐on   to  existing  telephony,  video,  or  IM/messaging  or  other  application  licenses.     ⇒ In  technology,  Market  Leaders  compile  a  score  of  4.11  and  Challengers  get  a   3.84.   • Reason:  Market  Leaders  typically  have  the  broadest  product  offering,  the   largest  R&D  budgets  and  often  the  greatest  support  for  mobility,  a  hot  area   among  IT  leaders  these  days.     ⇒ In  customer  service,  Market  Leaders  receive  a  score  of  4.03  and  Challengers  earn   a  3.88.   • Reason:  Market  Leaders  typically  have  the  fiscal  wherewithal  to  devote   substantial  resources  to  not  just  first-­‐tier  customer  service,  but  higher-­‐level   technical  support.    Market  Challengers  tend  to  more  often  sell  and  support   customers  through  channels,  where  support  may  vary.    Many  IT  leaders  say   they  see  differences  in  the  support  they  receive  from  multiple  channels  for   the  same  vendor.     ⇒ In  value,  Market  Leaders  garner  a  4.06  and  Challengers  receive  a  3.97.   • Reason:  Here  is  where  Market  Challengers  are  placing  their  competitive   emphasis,  using  innovative  approaches  such  as  cloud,  or  support  for   virtualization  to  offer  compelling  services  as  a  reduced  cost.    Although  they   aren’t  yet  demonstrating  greater  value  than  Market  Leaders,  they  are  close.     ⇒ No  single  score  factored  into  winner’s  advantages,  rather  all  winners  outscored   their  competitors  across  the  board  in  all  scores.   • Siemens  has  the  top  overall  scores  among  all  vendors,  in  all  scoring   categories.   • No  Market  Leader  outscored  Cisco  in  any  of  the  three  ratings  categories.   ©Nemertes  Research  2011  ±  www.nemertes.com  ±  888-­‐241-­‐2685  ±  DN  1530   8  
  • 9.     P ILOT H OUSE   M ARKET   C HALLENGER   W INNER   Siemens Ratings 4.40 4.36 4.30 4.27 4.27 4.20 4.18 4.10 4.00 Overall Technology Customer Service Value Chart  2:  PilotHouse  Market  Challenger  Winner:  Siemens,  Unified  Communications,  2011   Siemens   Summary:    Siemens’  4.27  is  the  top  overall  score  among  Market  Challengers,  and  it’s   also  the  highest  overall  score  among  all  UC  vendors.    In  fact,  Siemens  receives  the   highest  score  in  each  of  the  three  categories  measured.    Siemens  is  somewhat  of  a   pioneer  in  the  UC  space,  having  introduced  arguably  the  first  UC  product,   OpenScape,  back  in  the  early  2000s  (now  knows  as  OpenScape  UC  Server).    Siemens   continues  to  innovate,  offering  both  on-­‐premise  and  hosted  solutions,  as  well  as  a   cloud-­‐based  service.    Though  Siemens  lacks  the  U.S.  market  share  of  Cisco  and   Avaya,  its  offering  is  just  as  broad,  featuring  a  full  suite  of  UC  and  telephony  services   covering  small  and  large  offices;  contact  centers,  and  specific  vertical  solutions  (e.g.   trading  floors).  “OpenScape  is  a  great  tool/service  for  us,”  says  the  IT  manager  of  a   state  government  agency.       ⇒ Then  and  Now:    Siemens  improved  its  scores  across  the  board  from  last  year,   from  an  overall  3.75  to  this  year’s  4.27  with  the  largest  jumps  coming  in   technology  (3.71  to  4.27)  and  value  (3.71  to  4.36).    These  scores  reflect  Siemens’   ©Nemertes  Research  2011  ±  www.nemertes.com  ±  888-­‐241-­‐2685  ±  DN  1530   9  
  • 10.     efforts  to  not  only  broaden  its  solution  set,  but  to  differentiate  itself  from   competitors  by  offering  delivery  models  such  as  cloud-­‐based  services.     ⇒ Future  Direction:  The  big  challenge  for  Siemens  is  to  expand  its  reach  in  the  U.S.   market  beyond  the  SMB  and  specific  verticals  such  as  healthcare,  education,  and   government,  where  it  has  had  historical  strength.    This  year,  about  80%  of  the   firms  rating  Siemens  were  smaller  than  $50  million  in  revenue.    The  good  thing   for  Siemens  is  that  customers  evenly  praise  its  performance  across  all  ratings   areas.    IT  leaders  in  hospitality  and  manufacturing  both  praise  its  quality,  while   the  IT  director  of  a  small  education  institution  says,  “It’s  all  about  service.”       Technology   ⇒ Siemens’  4.27,  like  all  its  scores,  is  the  highest  among  all  UC  vendors,  Market   Leader  and  Market  Challenger  alike.    With  its  broad  product  set,  and  ability  to   support  multiple  delivery  models,  Siemens’  customers  consider  it  to  be  a   technology  leader.   Customer  Service   ⇒ Siemens’  customer-­‐service  score  of  4.18  is  its  lowest  rating,  considerably  lower   than  its  other  scores  but  overall  still  the  highest  among  all  vendors.    Siemens’   score  is  particularly  impressive  given  the  change  the  company  has  gone  through   over  the  last  few  years  as  Siemens  AG  sold  a  majority  stake  in  the  firm  to   investment  firm  Gores  Group.     • One  strength  for  Siemens  is  its  international  footprint.    “We  choose  Siemens   because  it  is  international,  able  to  support  us  in  multiple  countries,”  says  the   CIO  for  a  midsize  professional-­‐services  firm.   Value   ⇒ Value  is  Siemens  top-­‐scoring  area,  with  a  4.36,  and  again,  the  highest  score   among  all  rated  firms.    Customers  perceive  Siemens  as  providing  tremendous   bang  for  the  buck.   • “For  us,  it’s  price  that  makes  us  choose  Siemens,”  says  the  IT  buyer  for  a   small  construction  company.               ©Nemertes  Research  2011  ±  www.nemertes.com  ±  888-­‐241-­‐2685  ±  DN  1530   10  
  • 11.     P ILOT H OUSE   M ARKET   L EADER   W INNER   Cisco Ratings 4.30 4.23 4.20 4.17 4.16 4.10 4.12 4.00 Overall Technology Customer Service Value Chart  3:  PilotHouse  Market  Leader  Winner:  Cisco,  Unified  Communications,  2011   Cisco   ⇒ Summary:    Cisco’s  4.17  is  the  second-­‐highest  overall  score,  and  the  top  overall   score  among  Market  Leaders.    Cisco  wins  its  second  consecutive  Market  Leader   PilotHouse  award.    Customers  routinely  cite  reliability,  service  and  support,  and   Cisco’s  breadth  of  products  as  key  buying  criteria.     • Says  the  IT  director  for  a  global  publishing  and  media  company,  “Cisco   provides  us  good  range  of  options  for  voice,  video,  conferencing,  and   messaging.”    Adds  the  IT  manager  for  a  regional  healthcare  firm,  “Cisco   means  reliability.”     ⇒ Then  and  Now:  Cisco  continues  to  demonstrate  continued  improvement;  raising   its  score  from  a  4.08  in  2010  (after  increasing  from  a  3.88  in  2009).      Cisco’s   increasing  scores  demonstrate  its  payoff  from  recent  acquisitions  to  broaden  its   UC  portfolio  including  Jabber  and  Tandberg.    Overall  Cisco  is  a  reliable  vendor   with  good  support  services,  says  the  IT  director  for  a  regional  manufacturing   firm.     ©Nemertes  Research  2011  ±  www.nemertes.com  ±  888-­‐241-­‐2685  ±  DN  1530   11  
  • 12.     ⇒ Future  Direction:  For  Cisco,  the  key  challenge  is  branching  out  beyond  voice  and   selling  its  customers  on  its  vision  of  video  emerging  as  the  key  component  of  a   collaboration  strategy.    “Cisco  envisions  presence  and  video  and  all  this  usability.   We  aren't  really  seeing  the  value  out  of  it  yet,”  says  the  telecom  manager  for  a   global  manufacturing  organization.    However  Cisco’s  push  to  drive  technology   change  is  resonating  with  its  customers,  its  4.23  technology  score  is  its  highest,   and  the  highest  of  all  Market  Leaders.   Technology   ⇒ As  noted,  Cisco  scores  a  4.23  on  technology,  highest  of  all  Market  Leaders,  and  a   solid  improvement  over  last  year’s  4.08.    Cisco  continues  to  differentiate  its  UC   offerings  through  support  for  ubiquitous  video  and  extensibility  beyond  the   enterprise  firewall,  an  effort  that  is  resonating  with  its  customers.   • “Cisco  makes  it  easier  for  us  to  stay  connected  to  the  outside  world,”  says  the   CTO  of  a  midsize  manufacturing  organization.   Customer  Service   ⇒ Cisco  scores  a  4.16  on  customer  service,  just  .02  below  overall  winner  Siemens,   but  ahead  of  its  fellow  Market  Leaders.    Here  again,  Cisco  improved  on  its  2009   score  of  3.99.    Customers  continue  to  praise  Cisco’s  service  and  support,  which  is   a  departure  from  recent  years  and  a  clear,  concerted  effort  on  Cisco’s  part.   • “With  Cisco  we  always  get  fast  response  times  whenever  we  need  support,”   says  the  CIO  of  a  small  manufacturing  company.   • “Cisco  provides  great  service  across  the  board,”  says  the  head  of  IT  for  a   midsize  education  organization  notes.   Value   ⇒ Cisco’s  value  score  of  4.12  was  its  lowest  overall  score  in  any  rating  area,  but   even  here  Cisco  beats  all  of  its  Market  Leader  competition  (and  improves  from  a   3.86  in  2009,  its  largest  improvement  in  any  area).    We’ve  often  heard  the   perception  that  Cisco  is  a  premium  brand,  more  expensive,  than  other  products,   but  that  “nobody  gets  fired  for  buying  Cisco.”  Cisco’s  improved  score  this  year  is   evidence  that  it  is  increasingly  able  to  demonstrate  the  value  of  its  products,   even  in  cases  where  the  actual  price  is  more  expensive  than  competitors.     Perhaps  more  importantly,  we  hear  praise  for  Cisco’s  cost  competitiveness,   something  we  rarely  heard  before.   • “We  looked  at  ShoreTel,  and  ShoreTel  was  more  expensive.  We  looked  at   video  this  year  with  Lifesize;  Cisco  was  cheaper  there,  too,”  says  the  senior   architect  of  a  global  manufacturing  company.         ©Nemertes  Research  2011  ±  www.nemertes.com  ±  888-­‐241-­‐2685  ±  DN  1530   12  
  • 13.     P ILOTHOUSE   F INALISTS :   M ARKET   L EADERS     Unified Communications: Market Leaders Cisco Avaya Microsoft IBM Lotus 4.30 4.23 4.20 4.17 4.19 4.16 4.11 4.13 4.10 4.12 4.10 4.06 4.05 4.03 4.00 4.03 3.96 3.93 3.90 3.90 3.88 3.80 Overall Technology Customer Service Value Chart  4:  PilotHouse  Market  Leaders:  Unified  Communications,  2011       ©Nemertes  Research  2011  ±  www.nemertes.com  ±  888-­‐241-­‐2685  ±  DN  1530   13  
  • 14.     Avaya Ratings 4.20 4.19 4.10 4.11 4.10 4.03 4.00 Overall Technology Customer Service Value   Chart  5:  Market  Leader:  Avaya,  Unified  Communications,  2011   Avaya   ⇒ Avaya  and  Overall  Scores   Avaya  closes  the  gap  between  it  and  Cisco  this  year,  narrowing  its  overall   disadvantage  to  just  .06,  versus  .13  in  2010.    It’s  impressive  that  it  improved  its   competitive  positioning  despite  the  upheaval  of  executing  on  its  integration  of   Nortel,  shifting  to  a  channel-­‐based  model  for  the  SME,  and  delivering  a  new  line  of   products  focused  around  its  Aura  SIP  session  manager.    Avaya’s  scores  are  marked   by  significant  disparity  between  its  technology  (4.19)  and  value  (4.03)  ratings,   reflecting  on  both  the  success  of  Avaya  in  using  Aura,  it’s  expanded  mobility   products,  new  video  offerings,  and  its  recently  launched  Flare  user  experience  to   portray  itself  as  a  technical  leader;  and  the  failure  of  Avaya  to  improve  its  value   score  as  much  as  its  other  scores  (value  is  the  one  ratings  category  where  Avaya   trails  the  average  within  its  market  classification).    For  comparison,  the  margin   between  Avaya’s  highest  and  lowest  score  in  2010  was  .13  where  this  year  it  grows   to  .16.    While  Avaya  improved  its  technology  score  from  a  3.95  to  4.19,  it  only   improved  value  from  3.82  to  4.03.     ⇒ The  average  overall  score  of  all  Market  Leaders  is  4.06;  Avaya’s  is  4.11.   ⇒ The  average  technology  score  of  all  Market  Leaders  is  4.11;  Avaya’s  is  4.19.   • “I  like  the  fact  that  Avaya  is  an  open  system  and  will  integrate  with  other   applications,”  says  the  IT  director  of  a  midsize  services  firm.   ©Nemertes  Research  2011  ±  www.nemertes.com  ±  888-­‐241-­‐2685  ±  DN  1530   14  
  • 15.     ⇒ The  average  customer  service  score  of  all  Market  Leaders  is  4.03;  Avaya’s  is  4.10.   • “We  have  great  account  team.  They  find  us  answers,  and  are  willing  to  work   with  us.  You  can't  ask  for  much  more  than  that,”  says  the  IT  manager  of  a   large  professional-­‐services  company   ⇒ The  average  value  score  of  all  Market  Leaders  is  4.06;  Avaya’s  is  4.03.   • “There  are  better  phones  out  there  at  a  lower  price  point,  but  Avaya’s   products  work,”  says  the  manager  of  IT  at  a  small  educational  institution.         ©Nemertes  Research  2011  ±  www.nemertes.com  ±  888-­‐241-­‐2685  ±  DN  1530   15  
  • 16.     Microsoft Ratings 4.20 4.10 4.13 4.06 4.05 4.00 3.96 3.90 3.80 3.70 3.60 3.50 Overall Technology Customer Service Value   Chart  6:  Market  Leader:  Microsoft,  Unified  Communications,  2011   Microsoft   ⇒ Microsoft  and  Overall  Scores   Microsoft’s  overall  score  is  a  4.05,  trailing  Avaya  by  only  .06,  narrowing  the  gap   between  it  and  Avaya  from  .12  in  2010.    Microsoft  continues  to  gain  tremendous   momentum  for  its  UC  offering,  with  more  companies  citing  Microsoft  as  their   strategic  vendor  for  UC  than  any  other  vendor.    With  the  introduction  of  Lync  in   2011,  Microsoft  has  taken  direct  aim  at  not  only  the  desktop  messaging  and   conferencing  portion  of  the  UC  market,  but  at  the  core  voice  services  largely  owned   by  Market  Leaders  Cisco  and  Avaya.    Microsoft  still  has  work  to  do  to  convince  IT   buyers  that  it  is  able  to  offer  reliable,  feature  rich  voice  services,  though  its  slightly-­‐ higher-­‐than-­‐average  technology  score  demonstrates  that  Microsoft  is  convincing  its   customers  that  it  is  driving  technology  change.    Microsoft’s  biggest  challenge  is  in   customer  service,  where  it  trails  the  mean  Market  Leader  score  by  .07.    As  in  2010,   customer  service  remains  Microsoft’s  Achilles  heel,  though  its  score  improved  from   3.66  last  year  to  3.96  this  year.    If  Microsoft  can  continue  to  improve  on  customer   service,  it  can  challenge  Market  Leaders  for  the  2012  PilotHouse  award.   ⇒ The  average  overall  score  of  all  Market  Leaders  is  4.06;  Microsoft’s  is  4.05.   • “We  haven't  had  many  problems;  their  overall  support  has  been  good.    They   are  proactive  in  including  us  in  demos/pilots,  have  allowed  us  to  collaborate   ©Nemertes  Research  2011  ±  www.nemertes.com  ±  888-­‐241-­‐2685  ±  DN  1530   16  
  • 17.     with  other  large  companies,”  says  the  director  of  telecom  for  a  global  energy   firm.   ⇒ The  average  technology  score  of  all  Market  Leaders  is  4.11;  Microsoft’s  is  4.13.   • “It  works  well  for  what  it  does,  but  the  voice  side  is  a  little  shaky,”  says  the   senior  architect  for  a  financial-­‐services  firm.   ⇒ The  average  customer-­‐service  score  of  all  Market  Leaders  is  4.03;  Microsoft’s  is   3.96.   • “They  are  very  confusing  and  difficult  to  work  with.  Sometimes  they  want  to   work  directly  with  you  and  sometimes  they  want  to  send  you  to  a  partner.   Sometimes  they  send  you  to  an  account  team  that  gives  you  wrong   information,”  says  the  IT  director  for  a  global  manufacturing  firm.   ⇒ The  average  value  score  of  all  Market  Leaders  is  4.06;  Microsoft’s  is  4.06.   • “The  fact  that  we've  been  able  to  take  this  product  and  offer  so  many   capabilities  is  huge  value;  never  thought  I'd  say  Microsoft  and  value  in  the   same  sentence,”  says  the  IT  manager  for  a  global  technology  company.       ©Nemertes  Research  2011  ±  www.nemertes.com  ±  888-­‐241-­‐2685  ±  DN  1530   17  
  • 18.     IBM Lotus Ratings 4.10 4.00 4.03 3.93 3.90 3.90 3.88 3.80 3.70 3.60 3.50 Overall Technology Customer Service Value   Chart  7:  Market  Leader:  IBM  Lotus,  Unified  Communications,  2011   IBM  Lotus   ⇒ IBM  and  Overall  Scores   IBM  improved  its  scores  across  the  board  from  2010,  with  the  greatest  gain  coming   in  value,  where  it  rose  from  a  3.56  in  2010  to  a  4.03  in  2011,  tying  Market  Leader   runner-­‐up  Avaya  though  still  trailing  Microsoft.    IBM  Lotus,  like  Microsoft,  comes  at   the  UC  market  from  a  history  of  providing  messaging  and  non-­‐real-­‐time   collaboration  applications.    Unlike  Microsoft,  Lotus  isn’t  competing  for  the   telephony  market,  rather  its  strategy  is  based  on  delivering  presence,  Web   conferencing,  instant  messaging  and  softphone/video  clients  that  integrate  with   standards-­‐based  voice/video  backend  platforms  from  others.    Given  its  more   narrow  focus,  it’s  not  surprising  that  technology  is  still  the  area  where  Lotus  greatly   trails  the  competition.    Customers  like  the  value  in  what  they  get  for  their  money,   but  they  don’t  perceive  IBM  Lotus  as  a  technical  leader.     ⇒ The  average  overall  score  of  all  Market  Leaders  is  4.06;  IBM’s  is  3.93.   ⇒ The  average  technology  score  of  all  Market  Leaders  is  4.11;  IBM’s  is  3.88.   • “Great  integration  with  Lotus  Notes,  good  IM,  but  lousy  integration  with   other  products,”  says  the  IT  architect  of  a  global  manufacturing  firm.     ⇒ The  average  customer-­‐service  score  of  all  Market  Leaders  is  4.03;  IBM’s  is  3.90.   ©Nemertes  Research  2011  ±  www.nemertes.com  ±  888-­‐241-­‐2685  ±  DN  1530   18  
  • 19.     “We  have  IBM  Lotus  UC  products,  and  they  work  well,  but  getting  good   • customer  support  is  challenging,”  says  the  IT  director  of  a  midsize   manufacturing  company.   ⇒ The  average  value  score  of  all  Market  Leaders  is  4.06;  IBM’s  is  4.03.   • “IBM  has  a  much  better  licensing  strategy,  and  is  less  aggressive  in  auditing   our  licensing,  especially  compared  to  Microsoft,”  says  the  director  of  telecom   for  a  global  manufacturing  firm.     ©Nemertes  Research  2011  ±  www.nemertes.com  ±  888-­‐241-­‐2685  ±  DN  1530   19  
  • 20.     P ILOT H OUSE   F INALISTS :   M ARKET   C HALLENGERS   Unified Communications: Market Challengers Siemens ShoreTel NEC Mitel Alcatel-Lucent 4.40 4.36 4.30 4.27 4.27 4.20 4.18 4.10 4.08 4.05 4.00 4.00 3.95 3.92 3.90 3.85 3.79 3.80 3.78 3.72 3.72 3.70 3.70 3.67 3.67 3.67 3.68 3.64 3.60 3.50 Overall Technology Customer Service Value  Chart  8:  PilotHouse  Market  Challengers:  Unified  Communications,  2011       ©Nemertes  Research  2011  ±  www.nemertes.com  ±  888-­‐241-­‐2685  ±  DN  1530   20  
  • 21.     ShoreTel Ratings 4.10 4.08 4.00 3.95 3.90 3.92 3.85 3.80 3.70 3.60 3.50 Overall Technology Customer Service Value  Chart  9:  Market  Challenger:  ShoreTel,  Unified  Communications,  2011   ShoreTel   ⇒ IBM  and  Overall  Scores   ShoreTel  comes  in  as  the  Market  Challenger  runner  up,  trailing  Siemens  in  all   categories,  but  leading  all  other  Market  Challengers  in  every  ratings  area  other  than   customer  service,  where  it  trails  Mitel.    As  a  relative  newcomer  in  the  UC  space   compared  with  Siemens,  ShoreTel  is  still  increasing  its  portfolio,  recently  improving   its  mobility  and  messaging  services.  ShoreTel’s  customers  largely  praise  the  value   and  feature  sets  of  what  they  are  buying,  but  ShoreTel  must  address  customer-­‐   service  concerns  (historically  a  strong  spot  for  it  in  the  IP  telephony  market)  to   increase  its  UC  success.   ⇒  The  average  overall  score  of  all  Market  Challengers  is  3.90;  ShoreTel’s  is  3.95.   ⇒ The  average  technology  score  of  all  Market  Challengers  is  3.84;  ShoreTel’s  is   3.92.   • “ShoreTel’s  solution  is  reliable,  and  provides  a  complete  feature  set,”  says  the   manager  of  a  small  software  firm.   ⇒ The  average  customer-­‐service  score  of  all  Market  Challengers  is  3.88  ShoreTel’s   was  3.85.   • “They  need  to  improve  on  their  software  engineering  practices.  It  seems  like   there  are  patches.  Fix  a  few,  break  a  few.  And  that's  caused  a  little  it  of   ©Nemertes  Research  2011  ±  www.nemertes.com  ±  888-­‐241-­‐2685  ±  DN  1530   21  
  • 22.     frustration  with  us.  You  put  in  a  patch,  and  a  new  issue.  I  just  patched  last   night  after  identifying  three  issues.  One,  receptionists  couldn't  get  to  certain   calls,”  says  the  CIO  of  a  midsize  professional-­‐services  company.   ⇒ The  average  value  score  of  all  Market  Challengers  is  3.97;  ShoreTel’s  is  4.08.   • “For  us  ShoreTel’s  cost  was  the  biggest  driver,”  says  the  head  of  IT  for  a  small   professional-­‐services  firm.       ©Nemertes  Research  2011  ±  www.nemertes.com  ±  888-­‐241-­‐2685  ±  DN  1530   22  
  • 23.     Mitel Ratings 4.10 4.05 4.00 3.90 3.80 3.79 3.70 3.68 3.64 3.60 3.50 Overall Technology Customer Service Value   Chart  10:  Market  Challenger:  Mitel,  Unified  Communications,  2011   Mitel   ⇒ Mitel  and  Overall  Scores   Save  not  for  the  second-­‐highest  score  among  Market  Challengers  in  customer   service,  Mitel  would  have  finished  last  among  all  others  in  its  category,  trailing  other   Challengers  in  value  and  technology.  Although  Mitel  improved  its  customer-­‐service   score  from  2010  (3.99)  to  this  year’s  4.05,  it’s  technology  score  dropped  from  a  4.05   to  3.64,  and  its  value  score  also  dropped;  going  from  3.86  in  2010  to  3.68  in  2011.   Despite  recent  turmoil  that  have  resulted  in  management  changes  at  the  top,  these   scores  suggest  Mitel  and  its  channel  partners  are  overall  improving  their  customer   service  and  support.    If  Mitel  addresses  technology  and  value  concerns,  it  will   position  itself  among  the  top  of  the  Market  Challengers.   ⇒ The  average  overall  score  of  all  Market  Challengers  is  3.90;  Mitel’s  is  3.79.   ⇒ The  average  technology  score  of  all  Market  Challengers  is  3.84;  Mitel’s  is  3.64.   • “Mitel’s  products  come  in  at  a  competitive  price,  but  their  technology  is   behind  the  competition,”  says  the  IT  director  for  a  small  education   organization   ⇒ The  average  customer-­‐service  score  of  all  Market  Challengers  is  3.88;  Mitel’s  is   4.05.   ©Nemertes  Research  2011  ±  www.nemertes.com  ±  888-­‐241-­‐2685  ±  DN  1530   23  
  • 24.     • “Our  voice  provider  pushed  us  to  go  with  Mitel,  Mitel  provides  great  Level  2   support,”  says  the  IT  manager  of  a  midsize  healthcare  firm.     ⇒ The  average  value  score  of  all  Market  Challengers  is  3.97;  Mitel’s  is  3.68.   • “Mitel  won  our  RFP,  coming  in  $15k  less  than  anyone  else,  and  with  more   functionality,”  says  the  IT  manager  of  a  small  healthcare  firm.           ©Nemertes  Research  2011  ±  www.nemertes.com  ±  888-­‐241-­‐2685  ±  DN  1530   24  
  • 25.     NEC Ratings 4.10 4.00 4.00 3.90 3.80 3.78 3.70 3.67 3.67 3.60 3.50 Overall Technology Customer Service Value   Chart  11:  Market  Challenger:  NEC,  Unified  Communications,  2011   Alcatel-Lucent Ratings 3.80 3.72 3.72 3.70 3.70 3.67 3.60 3.50 Overall Technology Customer Service Value   Chart  12:  Market  Challenger:  Alcatel-­‐Lucent,  Unified  Communications,  2011   ©Nemertes  Research  2011  ±  www.nemertes.com  ±  888-­‐241-­‐2685  ±  DN  1530   25  
  • 26.     O THER   M ARKET   C HALLENGERS     ⇒ NEC  and  Alcatel-­‐Lucent  finish  as  the  bottom  two  finalists  in  the  Market   Challenger  category;  NEC  did  post  a  strong  value  score,  good  for  third  overall   among  Market  Challengers;  Alcatel-­‐Lucent  struggles  in  all  categories.    Compared   to  2010,  NEC  sees  strong  improvements  in  technology  (3.43  to  3.67)  and   customer  service  (3.50  to  3.67),  while  its  value  score  improves  by  a  whopping   .64  (3.36  to  4.00).    If  NEC  can  continue  to  improve  in  both  technology  and   customer  service,  it  stands  a  chance  of  contending  for  next  year’s  Market   Challenger  award.     Alcatel-­‐Lucent’s  scores  fall  across  the  board  from  2010.    Technology  drops  from   a  3.80  to  3.72,  customer  service  falls  from  3.96  to  3.67,  and  value  declines  from  a   3.96  to  a  3.72.  The  overall  score  thus  declines  from  3.91  to  3.70,  representing  the   biggest  decline  of  any  UC  vendor.    Alcatel-­‐Lucent  has  its  work  cut  out  for  it  if  it   wishes  to  improve  its  market  position.     ©Nemertes  Research  2011  ±  www.nemertes.com  ±  888-­‐241-­‐2685  ±  DN  1530   26  
  • 27.     C ONCLUSION   ⇒ The  UC  market  continues  to  consist  of  largely  three  types  of  vendors:   • Enterprise  telephony  companies  (Avaya,  Cisco)  fighting  to  put  their   applications  onto  the  desktop.   • Desktop  IM/messaging  vendors  (IBM  Lotus,  Microsoft)  attempting  to  either   supplant,  or  exist  alongside  telephony  vendors.   • Market  Challengers  offering  integrated  solutions  aiming  to  grow  market   share,  serve  vertical  markets  and/or  size  segments,  and  challenge  their   larger  competitors.   ⇒ For  another  year,  the  telephony  centric  Market  Leaders  post  the  highest  overall   scores,  but  the  gap  is  narrowing  as  Microsoft  continues  to  show  improvement,   while  IBM  Lotus  again  trails  other  market  leaders  overall.    Continue  to  evaluate  a   wide  variety  of  UC  vendors,  paying  attention  to  areas  including  customer  service,   long-­‐term  vision,  value,  and  demonstrated  implementation  success.   ⇒ Vendor  Selection:  Based  on  the  outcome  of  the  PilotHouse  program,  here  is   Nemertes’  guidance  (with  vendors  listed  in  priority  order):   • Evaluate  a  minimum  of  four  providers.  Cisco,  Siemens,  Avaya  and  Microsoft   are  solid  options.    For  smaller  and  mid-­‐size  firms,  consider  ShoreTel,  as  well.       • If  technology  is  your  key  concern,  consider  Siemens,  Cisco,  Avaya,  Microsoft,   ShoreTel,  and  IBM  Lotus.   • If  customer-­‐service  is  your  key  decision  criteria,  consider  Siemens,  Cisco,   Avaya,  and  Microsoft.   • If  value  is  your  key  goal,  consider  Siemens,  Cisco,  ShoreTel,  Microsoft,  Avaya,   and  NEC.   ⇒ Differentiation  of  Leaders  and  Challengers:  Overall  Market  Leaders  outscored   Market  Challengers  across  the  board.    Challengers  may  offer  a  better  solution  for   small  and  midsize  businesses  concerned  with  getting  lost  among  many  larger   companies.  And,  based  on  other  services  you  may  have  with  the  Challengers,   they  may  be  a  good  fit  based  on  minimum  annual  revenue  commitments.     ⇒ Improvement  Outlook:  Watch  out  for  continued  development  of  new  services  as   vendors  expand  their  offerings  into  voice  and/or  desktop  collaboration.    Also   keep  your  eye  on  expanding  hosted  offerings  that  will  increasingly  provide  an   alternative  (or  complement)  to  the  on-­‐premises  solutions  evaluated  for  this   award.           ©Nemertes  Research  2011  ±  www.nemertes.com  ±  888-­‐241-­‐2685  ±  DN  1530   27  
  • 28.     M ETHODOLOGY   The  population  includes  individuals  primarily  from  U.S.  companies  (based  in  the   U.S.,  but  many  of  which  are  global  multinationals)  who  are  responsible  for  selecting,   or  influencing  the  selection  of,  suppliers  of  data-­‐center  and  communications   products  and  services.     Sample  Frame   In  selecting  the  sampling  frame,  Nemertes  has  asked  individuals  in  the  following   populations  to  rate  their  providers:   ± U.S.   business   subscriber   lists,   including   individuals   who   have   opted   to   participate   in   surveys   and   who   have   been   pre-­‐screened   to   determine   responsibility  for  selecting  or  influencing  relevant  products  and  services.   ± Nemertes   Research   IT   executive   database,   limited   to   individuals   who   meet   the   criteria   for   the   representative   population.   Individuals   from   this   list   represent   primarily   U.S.   companies,   but   also   include   companies   based   elsewhere   that   have   presence   in   North   America.   The   database   includes   individuals   who   have   participated   in,   or   who   have   expressed   interest   in   participating   in   our   research,   or   with   whom   Nemertes’   analysts   have   established  a  business  relationship. Individuals  participated  in  this  project  using  three  methods:   ± Web-­‐based  survey.  This  is  the  largest  percentage  of  the  respondents.  Those   who   meet   the   sample   frame   randomly   received   invitations   to   participate   in   the  survey.   ± Visitors   to   Nemertes’   Web   site,   and   recipients   of   Nemertes’   blogs   and   columns   in   third-­‐party   media   partners’   Web   sites.   They   must   meet   the   criteria  to  participate.     ± Benchmark   interviews.   This   is   a   smaller   percentage   of   the   respondents.   Nemertes’   analysts   asked   numerous   detailed   qualitative   questions   to   gauge   why   they   rated   their   service   providers   the   way   they   did,   as   well   as   gathering   other  information  about  their  usage  of  communications  services.       Benchmark  participants  spent  one  to  three  hours  on  the  phone  or  in  person  with  a   Nemertes  analyst  discussing  issues  relating  to  their  use  of  products  and  services.   The  Web-­‐based  survey  participants  answered  a  subset  of  the  benchmark  questions   that  focus  on  rating  the  providers,  stack-­‐ranking  important  criteria,  providing   financial  data,  open-­‐ended  comments,  and  demographics.   Planned  Sample  Size   According  to  U.S.  Census  Bureau  figures,  there  are  2,306,070  companies  with  five  or   more  employees.  Our  goal  was  to  receive  responses  from  a  minimum  of  1,000   ©Nemertes  Research  2011  ±  www.nemertes.com  ±  888-­‐241-­‐2685  ±  DN  1530   28  
  • 29.     individuals,  which  would  give  us  a  95%  confidence  level  and  3%  margin  of  error—if   every  individual  rated  every  vendor  in  every  technology  area  rated.  We  received   substantial  ratings  for  each  technology  category  (several  hundred  per  category),  but   each  vendor  in  each  area  did  not  receive  a  rating  from  every  research  participant.   About  4,000  individuals  accessed  the  survey  or  participated  in  a  benchmark   interview.  Of  those,  about  2,000  meet  Nemertes’  standards  to  be  considered  “valid.”   Our  survey  tool  automatically  exited  individuals  employed  by  IT  vendors  and   providers.  Analysts  reviewed  all  other  ratings  (survey  and  benchmark)  line  by  line,   and  categorized  as  “invalid”  those  who  demonstrated  inconsistencies  or   inaccuracies  in  their  responses  as  part  of  Nemertes’  complex  qualification   methodology.   We  achieved  validity  across  the  survey  and  interviews  by  ensuring  the  questions  we   asked  were  the  same  and  that  the  interview  group  and  survey  group  represent   discrete  samples  of  the  same  population.  Nemertes  achieves  survey  and  interview   consistency  through  the  use  of  pre-­‐scripted  interview  forms  and  peer  review  of   interview  protocols.  Analysts  also  relied  upon  their  own  knowledge  of  the   technology  areas,  natural  breakpoints  in  the  data,  and  interview  notes  from  the   survey  participants  to  further  validate  ratings.   Survey  Sub-­‐Groups/Stratification   Nemertes’  analysts  researched  which  providers  offer  products  and  services  in  each   category  and  created  lists  from  which  participants  identified  their  primary  service   providers.  Participants  also  were  able  to  select  “other,”  and  identify  a  service   provider  they  use  that  may  not  be  included  on  the  explicit  list  provided.   The  challenge  is  that  some  providers  (Market  Leaders)  have  thousands  of  business   customers  and  significant  market  share,  while  others  (Market  Challengers)  have  a   few  hundred  or  few  thousand  customers  and  smaller  market  share.  We  realized   some  providers  would  garner  a  relatively  large  number  of  ratings,  based  on  the   number  of  customers  they  have,  while  others  would  have  a  relatively  small  number   of  ratings.     Therefore,  we  created  the  two  distinct  categories  for  the  awards,  Market  Leaders   and  Market  Challengers,  and  compared  providers  within  each  category.  Nemertes   placed  providers  within  each  category  based  on  its  own  research  and  publicly   available  data.  Analysts  also  examined  natural  breakpoints  in  the  data.  Market   Leaders  typically  have  >10%  of  market  share,  based  on  these  analyses.  Market   Challengers  typically  have  smaller  market  shares.  In  some  categories,  there  were   not  enough  ratings  to  issue  an  award  in  the  Market  Challenger  category,  or  the   ©Nemertes  Research  2011  ±  www.nemertes.com  ±  888-­‐241-­‐2685  ±  DN  1530   29  
  • 30.     market  is  so  new  that  all  vendors  are  considered  Challengers.  In  these  cases,   Nemertes  issues  an  award  only  in  the  appropriate  category.     Nemertes  reserves  the  right  to  address  acquisitions  occurring  during  the   benchmark  and  survey  period  on  a  case-­‐by-­‐case  basis.    Unless  otherwise  noted,  an   acquisition  merging  two  companies  in  the  same  award  category  must  be  complete   before  the  start  of  the  survey  and  benchmark  interview  period  to  be  counted  as  one   company  in  the  ratings.   ©Nemertes  Research  2011  ±  www.nemertes.com  ±  888-­‐241-­‐2685  ±  DN  1530   30  
  • 31.     Awards   Nemertes  is  issuing  awards  in  the  following  categories:   Nemertes PilotHouse Awards, 2011 Award Category Market Leaders Market Challengers Advanced Communications Services MPLS Services ü ü Carrier Ethernet Services ü ü Internet Access Services ü ü SIP Trunking Services ü No award Managed Router Services ü ü Managed Internet Services ü ü Wireless and Mobility Wireless LANs ü ü Wireless Voice & Data Services ü ü Application Delivery Application Delivery Optimization ü ü Virtual Desktops ü ü Voice Communications IP Telephony ü ü Managed IP Telephony ü ü Hosted Voice Over IP ü No award Data-Center Technologies Servers for Virtualization ü ü Storage for Virtualization ü ü Data-Center Colocation ü ü Unified Communications Unified Communications ü ü IP Contact Centers ü ü Security Managed Firewall/IDS/IPS ü ü Data-Center Firewalls ü ü Small Branch Firewalls ü ü Cloud Software as a Service: Office ü No award   ©Nemertes  Research  2011  ±  www.nemertes.com  ±  888-­‐241-­‐2685  ±  DN  1530   31  
  • 32.     Timing   The  Web-­‐based  survey  was  conducted  between  March  and  May  2011.  The   benchmark  research  was  conducted  between  January  and  April  2011.   Incentives  to  Participate  &  Time  Commitment   Participants  of  the  Web-­‐based  survey  received  a  small  incentive  for  participating  in   the  survey.  Participants  from  Nemertes’  database  receive  the  findings  and  are   invited  to  participate  in  a  Webcast,  in  exchange  for  their  time.  The  Web-­‐based   survey  takes  about  15  minutes  to  complete;  the  benchmark  requires  one  to  three   hours  of  participants’  time.   Future  Plans   Nemertes  plans  to  conduct  its  PilotHouse  Awards  program  annually,  though  it   retains  the  right  to  cancel  the  project  at  any  time.       About  Nemertes  Research:     Nemertes  Research  is  a  research-­‐advisory  and  strategic-­‐consulting  firm  that   specializes  in  analyzing  and  quantifying  the  business  value  of  emerging   technologies.  You  can  learn  more  about  Nemertes  Research  at  our  Website:   http://www.nemertes.com           ©Nemertes  Research  2011  ±  www.nemertes.com  ±  888-­‐241-­‐2685  ±  DN  1530   32