SurveyAnywhere: A Field data collection and Synchronization system for portable devices Amiangshu Shekhar Bosu Md. Ashfakul Islam firstname.lastname@example.org email@example.comIntroductionField data is important for any kind of research. Marketing researchers collect data from people toanalyze market and make their decisions about next marketing move. These decisions may run thecompany to entirely different direction. Software engineering researchers are interviewing designers,developers, quality tester and other stake-holders to gather their experiences, views and other data toanalyze them from different software engineering perspectives. Biological researchers go to variousnatural inhabitations to collect their required data, pictures of various wild lives. Researchers in eachfields depends on field survey data to continue their research. Even different organizations like anuniversity runs survey among students to know about their opinions regarding different issues ( e.g.course curriculum, course structure, instructor evaluation, transportation system, dining facilities, andlibrary services ) and make better decisions for students. At present surveys are very common.Lot of organizations and projects do not have required manpower to conduct a survey. Outsourcing asurvey is very popular in almost each sector. Lot of organizations out-source their surveys on behalf oftheir clients. Clients design their survey, target their population and make all decision related to conveythose surveys. Outsourced company make plan for execution, choose appropriate manpower anddistribute among workload among those surveyors. Surveyors go to field and communicate with orexamine their surveying subjects, fill all data collection forms and gather those data for the client.Portable devices are getting smarter with time. Laptops, PDAs, Net-books, tablets, and smart phones arebecoming richer in sense of computational power, storage and network resources. Especially smartphones are becoming very popular in these days. Smart phones can serve a variety of service to theirusers besides regular phone services. Different kinds of apps are developed to serve different needs ofusers. So, smart-phones can be used very conveniently to collect data for a survey. One thing a surveyorneeds to use their smart phones for data collection is an appropriate application, which is designed andbuilt for smart phones for that particular survey. But design and characteristic of data item in twodifferent surveys are not same. So surveyors need to install different applications for different surveys,which is very non-user friendly especially for the outsourced surveyor companies. They need acustomized application for surveying. Moreover if they want to change some form on their survey, theyhas to rebuild the application and redistribute. They also need a nice and structured framework forthese data collection process because uploading collected data directly from the portable devices is timeconsuming and infeasible sometimes.
We have designed and developed a flexible framework and an android application to manage surveydesign, survey distribution, data collection, and data synchronize between database server and smartphones.System Architecture:The system can be modeled in two parts: Server system and Client system. Researcher or owner of asurvey can create their data collection forms through the web interface easily. They can manage accessto their forms and collected data in database server. On the other hand surveyor can login to systemthrough android application and access to desired survey forms and upload data to server aftercompletion of a survey. Database Web Server Server Rest Web Service Frontend Figure 1: System Architecture
Server systemServer system will have the data storage. It will have a web-based front-end that will enableadministrator to define the data collection form. He can specify the field names, data types, attributesetc. Based on those definitions the system will customize its REST based web service. The server systemwill run on a cloud platform (SAAS- Software as A Service). The system will use database to store the textdata and storage (e.g. S3) for storing binary data (e.g. pictures, videos, sounds). Cloud platform willenable the application to be scalable. And the users will not need to worry about data backup. Thesystem will also allow the users to download the files easily and export the data in different formats (e.g.csv, xml)Client systemsThe client systems will run on different portable devices. Our initial target is to develop a clientapplication for Android devices. The application will connect to the server and retrieve the formattributes from the server. Using that definition the application will render the form to the user. Theusers will fill the form using collected data and submit the form to the server. The users can attachdifferent types of files (e.g. videos, pictures, sound) when they upload the data.As we are separating the two sub-components of our system using a defined interface (REST based webservice), the sub-components can be updated independently. And we can extend the scope of thesystem by creating clients for other types of devices.System ImplementationWe have implemented the system according to our design. A web based system was implemented toserve as Server system. And an android application was built to serve client system.PHP was used to implement web front end and MySQL was used as back end database server. Kohanaframework (http://kohanaframework.org ) was used for system. Android SDK was used to developandroid application. JSON was used for exchanging data between android application and the server.An administrator controls the web front end system. Password based user login was implemented. Anadministrator can create new users for survey or delete users. He can also design the data collectionform. A nice UI was developed for data collection form building. He can add five types of field for datacollection form: Text box, Checkbox, Dropdown and Password box, Location data field. He can alsomention must require data fields in the forms. He is able see the android preview of the form andreorganizes fields according his choice. After saving a form into database, a new table is created inMySQL database automatically corresponding to that form. When surveyor submits survey data, it willbe stored to that table of the database. Administrator can also manage user access to forms: grantaccess or revoke access.
A password based login system was implemented in client side. After login, surveyors as user of clientsystem can see available form list and description of forms. Available form list will be downloaded fromserver based on user. When user selects one form from the list, fields of the form will be downloadedfrom database and it will be built dynamically by android application. User can also able to change theirpassword. Figure 2: Database diagram of the system
Using the systemThe system is divided into three components. 1. Web based frontend for administrative users 2. Android application to access survey forms for mobile users 3. REST web service to manage interaction between Database and Android application1. Web based frontend for administrative usersThe web based frontend allows administrative users to: 1. Create add/edit/delete new mobile users (users that log in from portable devices and access forms) 2. Create add/edit/delete survey forms 3. Provide / Revoke access to a survey form to a mobile formsUsers login to the web-frontend using a public url. Our web frontend was hosted on Amazon WebService (AWS). AWS is cloud based and allows to dynamically scale the application. The URL of ourapplication was: http://ec2-107-22-45-207.compute-1.amazonaws.com/index.php (Figure. 3). The homepage contains brief motivation for the application and a login form for administrative users. Figure 3: Home page and login form for administrative usersa) Administrative User homepageAfter the administrative user has successfully logged in, the system shows a welcome message (Figure.4) and shows list of available actions to the user in top menu. Available actions are: 1. Form management: Edit / Delete/ Share existing survey forms 2. Create new form: Create a new survey form 3. User management: Add / Edit /Delete mobile users
4. Change password: Change password for the administrative user Figure 4: Homepage screen for administrative users after log inb) User ManagementThe user management section allows to create Add / Edit /Delete new mobile users. On the landing page of Usermanagement section there is a link to access the form to “Create a new user” (Figure. 5). Figure 5: Landing page of User management sectionThe “create new user” form has 5 fields (Figure. 6). To create a new user the First name, Last name,Email address and Password need to be filled. The confirm password field prevents accidental mistypingof password.
Figure 8: New user is listed in the user list of user management pageFor example, if the admin user clicks on the delete link the system will ask the admin user to confirm,whether he really wants to delete that user (Figure. 9). This confirmation prevents accidental deletion ofa mobile user. If admin clicks “Ok” on the confirmation screen the user will be deleted. Figure 9: Confirmation dialog to delete a user
After the user is created the system is redirected to the landing page of User management section. Thedeleted user is no longer listed in the userlist. Admin can now Add/ Edit/ Delete more users. Forexample, he might want to edit a user (Figure 10). Figure 10: Editing a userThe edit user form is similar to the “Create new user” form. Similar validation is also implemented in this form(Figure. 11). Figure 11: Form to edit an existing userAfter the user information is saved to the database. The form redirects to user listing page. It now shows that userwith modified information (Figure .12).
Figure 12: User list with edited user informationc) Create new formUser can access the section clicking on the “Create New Form” link from the top menu (Figure 10). Theform has two panels. On the left side user enters the name, description of the survey form. He candynamically add new fields to the form using the dropdown menu (Figure 13). On the right side there ispreview panel. Clicking on the “Preview on Android” button the user can see an approximate preview ofwhat the form might look like on an android device. Figure 13: Section to create a new survey form
The “create new form” section is the main part of the application. This form is very dynamic and weprovided special attention to usability in while designing this form. In any type of survey basically thereare three types of questions (Figure. 13). 1. Free form response: Presented using text input 2. Multiple choice: Presented using drop down list 3. Multiple response: Presented using checkboxWe can have also included a special field named “Location data”. If that field is included, android willsend the location of the user when submitting the form. The fields can be added to the form by selectinga new field type from the “Add New Field” from the drop down menu. Different fields have differenttypes of attributes. User can rearrange the fields, remove a field, or hide a field. User can mark a field asrequired or not (Figure 14). User can see the “Preview on Android” in the preview panel. When user issatisfied with the form he can save the form and the form will be created. Figure 14: Using the “Create new form” section to create new survey formThe system show popup dialog to indicate that a form has been created successfully (Figure. 15).
Figure 15: Confirmation of new form creationd) Form managementThe form management section allows the admin user to Edit / Delete and share form among mobileusers. This section can be accessed using “Form management” link from the top menu (Figure 16). If theuser clicks “Edit” the edit form is shown. The edit form is similar to the “Create new form”. If the userclicks “Delete” link, the system asks confirmation from the user. Figure 16: The form management section
When the user clicks the “Share form” link, the system shows the form to share with existing mobileusers. The right side shows the list of current users that has access to this form. There is “Revoke access”link on the right side of each user. Clicking this link will revoke access of this form from that user. In the“User name” textbox admin can type the name of a user to provide access. The system will show anautocomplete list to allow easy selection of a user that do not have access to the form (Figure 17).Admin selects a user from that autocomplete list and clicks on the “Allow acccess” button. The mobileuser will be added to the access list of that form. When that user now log in from an android device hewill see this form in the list of his forms. Figure 17: Sharing / Revoking the survey form from mobile users2. Android applicationA login page will be shown to the user after lunching the application on an android phone (Figure. 18 -left). An email id is required as login id. This login was created in the user management section of theweb frontend. First default password is set by administrator and informed via email to user. If the userprovides incorrect login id or password a pop up will be shown to the user informing an invalid login try(Figure. 18- right).
Figure 18: User login and validation on androidApplication will communicate will the server for user authentication. After successful authenticationmenu screen will be shown (Figure. 19). Figure 19: Main menu of the android applicationAfter clicking on “Show forms” icon a user will be directed to show forms page. Application will send aREST message to server with user authentication token and server runs a query to database for form listand send the list to application. A list view of forms will be showed up where first bold line shows nameof the forms and italic lines shows description of the forms. Those forms were created and accessgranted to current user in form management section of the web-frontend.
Figure 20: List of forms accessible to the userAfter selecting a form from the list, application will again send a REST message to server and server willreply with list consists of field name, type. Application will build a dynamic form based on that field list(Figure 21). Figure 21: A sample dynamic formAfter touching submit button, application will collect data from all fields, perform required validationand if all the validation is passed then submit those data to database server. A confirmation messagewill be shown to the user (Figure 22). The system will return to the main menu.
Figure 22: Confirmation of data submission to serverThe android application allows user to change the password. Change password icon will navigate tochange password window (Figure 23). Figure 23: Change password form on android
3. REST serviceThe rest service acts as bridge between the database and android application. Our implementation ofthe service is stateless. On first authentication, the service sends an authentication token to the client.For using other services a client needs to send that authentication token with their request. Themethods created by our service are described below.AuthenticationURL: http://SYSTEM_URL/index.php/webservice/loginParameter: username, passwordReturn: Authentication token on success, on failure returns 401 HTTP statusGetting form listURL: http://SYSTEM_URL/index.php/webservice/getformlistParameter: authentication tokenReturn: List of forms that user is allowed to accessGetting form fieldURL: http://SYSTEM_URL/index.php/webservice/getformfieldsParameter: authentication token, form_idReturn: Details of the form fieldsSaving a form dataURL: http://SYSTEM_URL/index.php/webservice/saveformParameter: authentication token, form_id, data of the formReturn: HTTP 200 on success, HTTP 500 on errorChanging passwordURL: http://SYSTEM_URL/index.php/webservice/changepasswordParameter: authentication token, current password, new passwordReturn: HTTP 200 on success, HTTP 500 on errorPossible Future EnhancementThe system functionalities can be enhanced further with some possible enhancements. Firstly, we canadd one more field type in form creation to handle uploading file from android device. This will allowthe users to upload images, audios and videos. Moreover, we can add video, audio or image data typefields directly to form. When user activates one of this type of form the application will accesscorresponding device to collect that type of data. For example, for image fields the application willaccess camera of the device to take capture an image.
On the web frontend, we can provide a data management section. From this section, user will be able tobrowse and download the data in various formats. The data management section can also show somesimple descriptive statistics of uploaded data.ConclusionPortable devices are very popular now. Using these portable devices in data collection in survey is a veryconvenient option. But problem of using portable devices is presentation of the data collection form ofdifferent forms in different devices. Our developed system is a flexible and convenient solution for thesescenarios. Designing the for is very flexible and user friendly. User can view preview of the form onandroid device from the web-frontend. This enhances the usability of the system greatly. The attributesof the forms are downloaded from server and build dynamically in portable devices. User interfaces areenhanced than consisting approach.