Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Email/Calendar Microsoft Exchange 2003 End User Pilot Survey ...


Published on

Published in: Technology
  • Hi,

    You may also check how to send SMS from MS Exchange 2003 with Ozeki NG SMS Gateway:

    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

Email/Calendar Microsoft Exchange 2003 End User Pilot Survey ...

  1. 1. University Information Systems Central Administration IT (CAIT) Email & Calendaring Microsoft End User Pilot Survey Summary February 2, 2005 Author: Susan DeLellis, UIS Purpose: CAIT released a Request for Proposal (RFP) to vendors in early June, 2004 to evaluate and procure a new Email and Calendaring System for CAIT and its customers. Two vendor finalists were chosen out of eight to take to the next level of evaluation which included on-site back end testing as well as an in-depth end-user production pilot. These two finalists included IBM, with its Lotus Notes/Domino solution and Microsoft’s Outlook/Exchange 2003. The CAIT Email/Calendar project team solicited approximately 70 end users to participate in the Email/Calendar pilot for each vendor solution. The end user community represented staff and students from a number of CAIT departments and schools, including UIS, OHR, President’s Office, FAS, GSD, HSPH, Divinity School and the Law School. End user participants were asked to utilize the pilot email/calendar system as their production email system for a period of at least a week. An informal training and information session was provided to the pilot members prior to the go-live pilot date, with full documentation and installation instructions. In addition, a production help-desk service was provided by the UIS helpdesk to provide trouble ticketing work flow and additional help to pilot users. At the end of the formal pilot period, each end user was encouraged to fill out a twenty question on-line user survey to express the level of satisfaction with the Email/Calendar solution provided during the test. This document summarizes the results of the on-line survey for the Microsoft Exchange 2003 Pilot. A separate evaluation is also provided for the IBM Lotus Notes/Domino. For more information about this survey or the Email & Calendaring project, please contact Susan DeLellis at . Table Of Contents: Summary Statistics of the Survey………………. 2 - 10 Copy of the Survey (html Doc). …………………. 11 1
  2. 2. University Information Systems EMAIL/Calendar Microsoft Exchange 2003 End User Pilot Survey Summary Statistics Pilot Customer Base Surveyed: 70 Number of Respondents: 22 Response Rate: 31% Respondent Breakdown: Staff Members 81.8% Central Admin: 54.5% HSPH: 0.0% Students 18.2% FAS: 31.8% HDS: 0.0% Faculty 0.0% Law School: 9.1% GSD: 4.5% • Primary Email Client in Use Today o 45.5% Eudora o 18.2% Microsoft Outlook o 4.5% Mac OS X Client o 20.0% Microsoft Outlook Express o 0.0% Web Client o 31.8% Other – please specify- (5-Mozilla/Thunderbird, 1-forwards to gmail,1 -Pine) • Primary desktop/laptop operating System Environment in Use Today o 86.4% Windows 2000 or XP o 9.1% Mac OS X o 0.0% Linux o 0.0% Windows 98 or Older o 4.5% Other – please specify (Solaris) • Primary Calendar system/client in Use Today o 72.7% Meeting Maker Native Client o 4.5% Microsoft Exchange Outlook Client o 13.6% Do not currently use a Calendar System/Client o 9.1% Other (Outlook with PDA) 2
  3. 3. The following responses relate to the recent testing of the Microsoft Exchange 2003 Solution: • Type of Non-Web Client used in the pilot testing o 86.4% Microsoft Outlook 2003 for Windows o 0.0% Microsoft Outlook 2003 for Mac Os X o 13.6% Other (Thunderbird and SnapperMail (email), Microsoft Entourage and Apple iCal (calendaring UIs)3- used web client only) The following relate to the Microsoft Exchange 2003 solution features and functionality: • Level of Satisfaction with the functionality and use of the Microsoft Exchange 2003 System using the Native Client Very Satisfied Somewhat Dissatisfied Very Satisfied Satisfied Dissatisfied Customization of Client 10.0% 55.0% 30.0% 5.0% 0.0% Menus Email Message 40.9% 54.5% 4.5% 0.0% 0.0% Communication Message Formatting 36.4% 50.0% 13.6% 0.0% 0.0% Contact Lists 23.8% 61.9% 14.3% 0.0% 0.0% Search Functions 15.0% 50.0% 20.0% 15.0% 0.0% Message Filters 30.0% 40.0% 20.0% 10.0% 0.0% Folder Management 38.1% 47.6% 9.5% 4.8% 0.0% Task Management 15.0% 70.0% 10.0% 5.0% 0.0% Distribution Lists 5.3% 68.4% 26.3% 0.0% 0.0% Meetings and Apt. 10.0% 75.0% 15.0% 0.0% 0.0% Scheduling Calendar Proxy Mgt. & 5.3% 63.2% 21.1% 5.3% 5.3% Delegation Rights Calendar Views 15.0% 75.0% 5.0% 5.0% 0.0% Calendar Suggested 10.5% 68.4% 15.8% 5.3% 0.0% Meeting Times Encryption & Security 11.1% 61.1% 16.7% 5.6% 5.6% On-line Help 35.3% 29.4% 23.5% 11.8% 0.0% Spell Check 35.3% 41.2% 17.6% 5.9% 0.0% • Features/Functionality of the Microsoft Exchange solution found to be most Useful or liked the best (free text field) Summary: Overall, UI is very intuitive. This was the best feature. Receiving meeting invitations as email messages in Inbox; offline support; fast and responsive in pilot; keyboard commands and shortcuts made sense The calendar and task features were extremely easy to use, and I actually found myself using it. 3
  4. 4. calendar and mail in one I have used Outlook with Exchange in the past. I like the calendaring, task management and synch ability with pocketpc devices -Outlook 2003 strong UI -Good Exchange performance -Nice message flagging -Excellent parity with OWA on Internet Explorer -Search folders - very nice feature In the above questions, you need to add a selection for Did Not Try. There are several that I can't respond to because I didn't use them--calendaring, for instance. I have no use for calendaring, and therefore I have no feedback. The feature integration and intuitive design of the client made it very easy to use. n/r n/r I like the way Outlook is fully integrated into the Windows environment, including browser, o/s and MS office. I really liked the Web Client. It was very easy to use and when using IE, I liked the functionality of drag and drop. At times I forgot I was using the web client all together. Calendar announcements in email Outlook was easy to use on an intuitive level. The help was informative and usually solved whatever issue I had without having to involve the Helpdesk or another third party. In-windows dragging and dropping, fast response time and synchronized calendar - with Outlook alone, the calendar is only on one computer. Overall a good interface and easy to use; I think our customers will find it much easier than Domino. I liked how the Inbox was visually organized. Also liked how a new message resulted in a "pop up" in the bottom corner of the screen, which then faded away. Easy and intuitive to use. Task to-dos have more functionality than we have today. I like the in- line spell check I found it to be much more intuitive to use that Domino. I had no trouble figuring out how to organize and customize. I know I'm in the minority, but I like what MS has done with the UI in Office 2003 in general despite being generally critical of Microsoft. Perhaps owing to student familiarity with other MS products, my guess is undergrads would have a much easier time with Outlook than with Lotus. Outlook's pref settings are also more coherent, and the web client is impressive. The preview pane is nice, and I do like the ability to switch back and forth between mail, contacts, calendar etc. from the main screen. The calendar is very intuitive, and I can definitely see the advantages of an office having access to a system like this. The MacOS interfaces for calendaring through Exchange are much more usable and there are more options available. The IMAP email capabilities of Exchange aren't distinguishable from other IMAP servers, including the existing Harvard one. I used my regular email client to access the Exchange email server, so there was no difference in UI experience. • Features/Functionality of the Microsoft Exchange 2003 solution found to be most difficult to use or liked the least (free text field ) Summary: Entourage for the Mac does not have parity with Outlook for the PC. Non-industry standard alt- F4 to close windows. No integrated chat native to Exchange. Scheduling options are rather complicated; menus in full Outlook client were more complex than clients may be used to; more difficult (Web client only, perhaps?) to view full headers; special characters like ampersand, etc. cause messages not to be viewed Well, I use Gmail, and the sort of sorting that gmail does I feel is optimal. But for a standard conventional email client, this one is fair. it was kinda hard to customize everything to my satisfaction 4
  5. 5. It would not let me send emails as a different account name but I suspect that was just because the pilot addresses where on a different server in comparison to the other email accounts I manage Because the most important use for me is the ability to work on somebody else's email, I couldn't test the features that I really needed to test. It is critical that I be blind copied whenever xxxx sends mail from one of us computers, and this option was not available. Also, the ability to toggle with ease between xxxx’s name and my own when sending email seemed complicated. This is a problem. Delegation/Proxy wasn't as nice as MM. There is no listing of users who have granted right to you when trying to open a calendar. n/r n/r As an existing Outlook user I really didn't have trouble adapting to an Exchange back end. Nothing at all. Setup of multiple accounts I was only able to access one other person's calendar. I'm not sure if that was an issue with user preferences or the way that the pilot was set up. It would also have been nice if instant messaging was fully functional. As it was I could not access anyone else to instant message. The instant messaging function is not that intuitive to use. Web Access has limited functionality, and search folders or filters don't work for IMAP servers. Boo. Took up WAY too much screen real estate despite tweaking. I like to have email open in one window and do *work* in another window sometimes and this was difficult. Continue to have security concerns. n/r I'd like to be able to customize my dashboard more. I could not figure out how to make it automatically remember the order in which I like to display my incoming mail. Sometimes it would revert back to the default settings. I would have liked it to automatically display the contents of the next email after I've deleted the email message before it. It's a resource-intensive package even in light of its functional breadth. It's occasionally slow. It defaults to HTML messages. Its attachments are often not receivable for non-Outlook users (although I didn't personally hit this problem, winmail.dat). It seems like bad news if you want to check multiple email accounts. It should auto-delete deleted messages on exit, to conserve server space. I used this very little, so I can't comment on it too much, other than to say that I don't like not being able to restrict my email viewing to text-only (or at least I couldn't figure out how to do it.) That's a security issue for me. The Exchange IMAP service seems a bit slower than others that I use, including the current Harvard IMAP service. • Overall level of Satisfaction with the Microsoft Exchange 2003 service features and functionality o 27.3% Very Satisfied o 59.1% Satisfied o 4.5% Somewhat Satisfied o 9.1% Dissatisfied o 0.0% Very Dissatisfied 5
  6. 6. The following responses relate to the Microsoft Exchange 2003 Experience and Ease of Use: • Length of time taken to “learn” the Microsoft Exchange 2003 system & features o 59.1% Less than one hour o 13.6% About one hour o 22.7% About two hours o 4.5% Other – (already familiar) • Level of Intuitiveness or Ease of Use of the Microsoft Exchange 2003 system o 47.6% Very Intuitive o 19.0% Intuitive o 33.3% Somewhat Intuitive o 0.0% Not Intuitive at all • Experience with Microsoft Exchange 2003 system & client compared to current email/calendar system & client in use at Harvard today o 0.0% Much easier to use o 40.9% Easier to use o 40.9% About the same o 18.2% Not as easy to use • Overall Satisfaction with the Microsoft Exchange 2003 solution Experience & Ease of Use o 36.4% Very Satisfied o 45.5% Satisfied o 13.6% Somewhat Satisfied o 4.5% Dissatisfied o 0.0% Very Dissatisfied The following responses relate to the Microsoft Exchange 2003 Web Client called OWA: • Satisfaction Level with the features & functionality of the Microsoft Exchange 2003 Web Client o 28.6% Very Satisfied o 47.6% Satisfied o 9.5% Somewhat Satisfied o 0.0% Dissatisfied o 0.0% Very Dissatisfied o 14.3% Did not use the OWA Web Client during Pilot testing 6
  7. 7. • Features/Functionality of the Microsoft Exchange 2003 Web Client (OWA) found most useful or liked the best (free text question) Summary: Was very intuitive. Very fast and responsive; very good interface within IE. Overall a much better Web client than Lotus. The calendar and task features were extremely easy to use, and I actually found myself using it. interface was very nice Very similar to the actual Outlook client which is nice The experience was very similar to the native client which made it very easy to learn. I like the fact that OWA is almost exactly the same as Outlook. It was almost exactly like the full client itself. Very easy to use and navigate. Calendar To be able to easily access my calendar remotely from my Mac at home was the best feature. I like that it mostly resembled the native client. Look & Feel is very similar to the native client. I couldn't figure out how to view and change proxy delegations on the web client for Calendaring. This would be a big drawback of the web client. Spell check works nicely. On-line help is good too. I did not use the Web Client as much as the desktop version. However, I really liked that it seemed to look and behave very much like the desktop version. I didn't use it much, but I was impressed with its speed and the carryover of features. It also mirrors the interface of the software client, which will be helpful for student types. It seemed fairly instinctive, I mostly used the basic client via Firefox, and didn't have any trouble navigating or reading emails, other than the subject line restrictions. • Features/Functionality of the Microsoft Outlook Web Client (OWA) found most difficult to use or liked the least (free text question) Summary: Wasn't fully functional on the Mac. Initial page could sometimes take a while to load; compatibility with non-IE browsers not as good. I couldn't use premium with mozilla firefox it took eons to load (seriously) It would not let me change my password - pop up dialog would come up server not found no matter which web browser I used. Proxy/delegation Nothing in particular, other than in general I don't like Web clients. Nothing at all. 7
  8. 8. Set up of new account Because I have a Mac at home I really missed the drag and drop functionality available with the native client. Limited filtering was annoying, and the discrepancy between the web client's and native client's ability to filter was confusing. I normally use Mozilla Firefox, but I found the "basic" web client very limited. This meant I had to have two different browsers open for the duration of the beta test (I would never give up Firefox's extensive functionality and customizable security for IE, in gen Proxy delegation to calendar. nothing comes to mind. It doesn't support Mozilla at all, which is a huge sin. I can imagine talking to undergraduate users over the phone: User: "My internet explorer is broken in ways X, Y, and Z." Me: "I could fix it, but you'd be better off using Mozilla." User: "But I heard I can't check my OWA that way." Argh. (Perhaps it works with Firefox? I haven't tried.) The inability to log in via the Mozilla Suite. The reduced functionality of non-IE browsers (and the requirement to run Active X in an IE browser.) The security restrictions on subject lines were super ridiculous and very inconvenient. The security messages the browser flashes need a lot of fine tuning to avoid upsetting or confusing clients. • Experience with Microsoft Exchange 2003 Web Client compared to current email/calendar system Web Client or other Email web clients used at Harvard in the past o 33.3% Much easier to use o 28.6% Easier to use o 19.0% About the same o 14.3% Not as easy to use o 4.8% Never Used Email Web Clients in the past The following responses relate to the Importance of Aspects of Communication Services: • Importance of each of the following items as it relates to providing Collaboration and Communication Services 8
  9. 9. Absolutely Very Important Somewhat Not at All Critical Important Important Important New Value Added Services (IM, Web 9.1% 9.1% 27.3% 22.7% 31.8% Conf) Highly Reliable 81.8% 18.2% 0.0% 0.0% 0.0% Services More Cost Effective 13.6% 50.0% 13.6% 18.2% 4.5% Services Filesharing& Doc. 18.2% 27.3% 36.4% 9.1% 9.1% Mgt. Enhanced Security 36.4% 50.0% 9.1% 4.5% 0.0% Integrated Desktop 18.2% 50.0% 9.1% 9.1% 13.6% client for Services End User help desk 22.7% 40.9% 13.6% 18.2% 4.5% w/phone support Complete System 18.2% 31.8% 22.7% 18.2% 9.1% Documentation End User Training 13.6% 40.9% 22.7% 13.6% 9.1% Unified Messaging 4.5% 31.8% 18.2% 22.7% 22.7% (VM/Email Integr) Seamless Access to 4.5% 45.5% 27.3% 13.6% 9.1% Services via PDAs Robust Web Client 54.5% 36.4% 4.5% 4.5% 0.0% for Remote Access • Other Comments shared by Pilot Users (free text) Summary While I liked the product a lot, I'm concerned that many enhanced features require an additional software product. the web client was way too slow...otherwise, it was good Overall Outlook/Exchange was vastly superior to Domino/Notes. I really liked The outlook experience, partially because it is like the rest of the Office suite and very easy to support and use. Note: We primarily support Eudora but also have a lot of users using Silkymail or Squirrelmail web clients Did not test distribution lists. This survey needs a column for N/A or Not tested!!! Also we tested both web clients and the native client and there isn't much of a way to indicate this on this survey Very disappointed that support for the palm client was not part of this test. I came in expecting to be less than impressed with Outlook, having used Exchange in a previous workplace through Outlook 97, but the speed and usability improvements in Outlook 9
  10. 10. 2003 are quite impressive. If there's a way to check Exchange mail through IMAP/POP and the server maintenance/cost end of things isn't any worse than the existing Unix system, Outlook actually seems well worth looking into, at least from an undergrad perspective, provided we could give out free copies to undergrads. I would like to see more testing with non-MS clients with the Exchange backend to see if it would be possible to retain heterogeneity and reduce the security risk of relying on Microsoft software. Also, the browser restrictions are really prohibitive for students. And did anyone do testing from slow modem connections or off-campus locations? I worry about the ability of someone to access this from abroad. Also, the text limits on feedback are a bit stifling. Exchange calendaring interface (Entourage) is much more intuitive and friendly than the Lotus Notes client on the Mac. More options are available for using calendar services on Mac and with Palm devices, compared to Meeting Maker and Lotus. Email is fine, no real advantages to Exchange IMAP over existing service or Lotus Notes email service. • Interest in testing the IBM Outlook Connector which allows use of a Microsoft 2003 Exchange Client to access an IBM Lotus Domino back end messaging system o 59.1% Yes o 27.3% No o 13.6% No, but I would be interested in hearing about this option 10
  11. 11. Microsoft Exchange 2003 End User Pilot Email & Calendar Survey To view a copy of the Microsoft Exchange 2003 End User Pilot Survey questionnaire, click on the following link. Please note this is an html document. This survey was constructed using the iCommons Polling Tool. Not available at this time. 11