User-driven Technology Evaluation of eParticipation Systems

1. User-driven Technology Evaluation of eParticipation Systems Sotirios Koussouris, LoukasKipenis, George Gionis, FenaretiLampathaki, YannisCharalabidis, DimitriosAskounis Decision Support Systems Laboratory National Technical University of Athens Greece

2. WEB.DEP Project WEB.DEP – Western Balkans Democratic Participation FP6 Funded Project Duration: 1/1/2007 – 31/3/2009 http://www.web-dep.eu 7 Partners including: 2 universities (NTUA, NAPIER) 1 vendor/technology provider (ATC) 3 Balkan national news agencies (ATA, MIA and TANJUG) 1 association of journalists – a union (ESIEMTH)

3. Why News Agencies? As media organisations, news agencies act asintermediaries between Governments and Citizensand are regarded as “neutral news providers” In this case, using conscious code of ethics to…. Present the news “unedited” and “uncommented” (facts not opinions) Involve stakeholders (e.g. government and experts) Manage (moderate) forum New democratic context = big change for news agencies (relationship government & citizens)

4. WEB.DEP Outline 1/2 Information provision –content management system to share news more widely and efficiently E-Participation via shared forum across 3 countries Plus polling/questionnaires Content and moderation provided by news agencies

5. WEB.DEP Outline 2/2

6. Evaluation Needs eParticipation systems should satisfy multiple needs in order to be considered successfull. Rely not only on technology but (most importantly) on their “participative“ character and the users‘ engagement. These needs can be separated in two major categories IT driven category “Institutional” or “Decision Making” driven category

7. Methodology Used Two structured straight-forward questionnaires addressing the actual users of the platform Aiming at capturing two of the most important dimensions of eDemocracy systems the user perception of the usability and the expected impact of the system to the user‘s life Resulting in recommendations or guidelines that will strongly bind the following aspects system usability (user friendliness and straightforward functionalities) expected impact from the use of the system (increasing citizens’ participation in the decision making progress and of making the public opinion heard and considered by the decision makers)

8. Technology Acceptance Model 1/2 Extension of classical TAM with the introduction of a new construct is introduced, namely “External Factors”. The aim of this modification is to identify the level of impact of relevant external factors (prior experience, educational level and job/occupation relevance) to Perceived Usefulness, Perceived Ease of Use and Intention to Use. This addition is of outmost importance when dealing with systems deployed in converging regions, as issues like Digital Divide, low internet penetration and limited familiarization with technology become obstacles on the road towards the sustainable and beneficial implementation and operation of such a system.

9. Technology Acceptance Model2/2 For each construct, a group of questions was asked regarding the: Perceived Usefulness of the systems Perceived Ease of Use of the system Intention to Use the system External Factors regarding Relevant Skills Prior Experience Educational Level of the users.

10. Hypotheses drawn Various hypotheses were drawn to be tested. Based on the outcomes improvement scenarios will be build for maximizing usability and impact of the system. H1: Prior experience with similar technological tools and/or active citizenship will have a direct positive effect on system perceived usefulness. H2: Prior experience with similar technological tools and/or active citizenship will have a direct positive effect on system perceived ease of use. H3: Educational level will have a direct positive effect on system perceived usefulness. H4: Educational level will have a direct positive effect on system perceived ease of use. H5: Job/Occupation Relevance will have a direct positive effect on system perceived usefulness. H6: Prior experience with similar technological tools and/or active citizenship will have a direct positive effect on intention to use the system. H7: Job/Occupation Relevance will have a direct positive effect on intention to use the system. H8: Educational level will have a direct positive effect on intention to use the system. H9: Job/Occupation Relevance will have a direct positive effect on system perceived ease of use.

11. Single Construct Analysis Initial findings calculating the mean values for each construct The educational level of the users lies on “Degree or higher training” Perceived Usefulness, Perceived Ease of Use and Intention to Use are quite positive (lying on “maybe yes” level) Users’ occupation relation to IT technology and decision-making process and prior experience to discussion systems are slightly above the mean value of 3 which corresponds to “Yes and No” level.

12. TAM Findings Educational Level seems to have a strong positive effect to all main constructs that represent the grade of acceptance for WEB.DEP. There is a strong indication of positive effect of Occupation towards Perceived Usefulness, Perceived Ease of Use and Intention to Use. There is an indication of negative effect of Prior Experience to all 3 main constructs, which constitute a really interesting finding.

13. Hypotheses drawn Various hypotheses were drawn to be tested during the evaluation and based on the outcomes improvement scenarios will be build for maximizing usability and impact of the system. H1: Prior experience with similar technological tools and/or active citizenship will have a direct positive effect on system perceived usefulness. H2: Prior experience with similar technological tools and/or active citizenship will have a direct positive effect on system perceived ease of use. H3: Educational level will have a direct positive effect on system perceived usefulness. H4: Educational level will have a direct positive effect on system perceived ease of use. H5: Job/Occupation Relevance will have a direct positive effect on system perceived usefulness. H6: Prior experience with similar technological tools and/or active citizenship will have a direct positive effect on intention to use the system. H7: Job/Occupation Relevance will have a direct positive effect on intention to use the system. H8: Educational level will have a direct positive effect on intention to use the system. H9: Job/Occupation Relevance will have a direct positive effect on system perceived ease of use.

14. Conclusions – Next Steps Analyzeand investigate the “negative” relations revealed The proposed TAM should be tested on other systems and redesigned accordingly in order to move towards a one-fits-all solution Reduce the extend of the questionnaire in order to encourage more responses from users Re-evaluation of the WEB.DEP platform, as it matures.