Successfully reported this slideshow.
Your SlideShare is downloading. ×

ASSESSMENT OF THE USABILITY OF LATVIA’S OPEN DATA PORTAL OR HOW CLOSE ARE WE TO GAINING BENEFITS FROM OPEN DATA

Check these out next

1 of 25 Ad
1 of 25 Ad

ASSESSMENT OF THE USABILITY OF LATVIA’S OPEN DATA PORTAL OR HOW CLOSE ARE WE TO GAINING BENEFITS FROM OPEN DATA

Download to read offline

This presentation is a supplementary material for the following article -> Nikiforova, A. (2020). Assessment of the usability of Latvia’s open data portal or how close are we to gaining benefits from open data. In In IADIS 14th International Conference on Interfaces and Human Computer Interaction (pp. 51-28).
Nowadays, more and more countries are launching their own open data portals, seeking to provide their citizens with open data in a form that is useful and suitable for the original purpose of the open data, and Latvia is not an exception. Despite the fact that the Latvian open data portal was launched only in 2017, it is considered to be a fast-tracker. However, despite the overall high evaluations, critical voices,and many discussions about whether the Latvia’s open data portal is of sufficient quality to be appeared. Therefore, while previous studies deal with quality of open data, this study focuses on the analysis of the Latvian open data portal and aims to find the key challenges that may have a negative impact on user experience.The paper assesses the current situation and recommends corrective actions,highlighting the aspects to be considered when developing and improving open data portals.

This presentation is a supplementary material for the following article -> Nikiforova, A. (2020). Assessment of the usability of Latvia’s open data portal or how close are we to gaining benefits from open data. In In IADIS 14th International Conference on Interfaces and Human Computer Interaction (pp. 51-28).
Nowadays, more and more countries are launching their own open data portals, seeking to provide their citizens with open data in a form that is useful and suitable for the original purpose of the open data, and Latvia is not an exception. Despite the fact that the Latvian open data portal was launched only in 2017, it is considered to be a fast-tracker. However, despite the overall high evaluations, critical voices,and many discussions about whether the Latvia’s open data portal is of sufficient quality to be appeared. Therefore, while previous studies deal with quality of open data, this study focuses on the analysis of the Latvian open data portal and aims to find the key challenges that may have a negative impact on user experience.The paper assesses the current situation and recommends corrective actions,highlighting the aspects to be considered when developing and improving open data portals.

Advertisement
Advertisement

More Related Content

Slideshows for you (18)

Similar to ASSESSMENT OF THE USABILITY OF LATVIA’S OPEN DATA PORTAL OR HOW CLOSE ARE WE TO GAINING BENEFITS FROM OPEN DATA (20)

Advertisement
Advertisement

ASSESSMENT OF THE USABILITY OF LATVIA’S OPEN DATA PORTAL OR HOW CLOSE ARE WE TO GAINING BENEFITS FROM OPEN DATA

  1. 1. ASSESSMENT OF THE USABILITY OF LATVIA’S OPEN DATA PORTAL OR HOW CLOSE ARE WE TO GAINING BENEFITS FROM OPEN DATA 14th International Conference on Interfaces and Human Computer Interaction (part of 14th Multi Conference on Computer Science and Information Systems) 21 – 23 July 2020 Anastasija Nikiforova Faculty of Computing, University of Latvia Anastasija.Nikiforova@lu.lv
  2. 2.  in order to benefit from the open data, a sufficiently high-quality and user-friendly open data portal should be provided, allowing access to the data for all stakeholders, which would result in the transformation of the data into value and knowledge for the society;  despite many countries develop and launch their own OGD portals, they received a great deal of criticism from both, society and technical experts - Latvia is not an exception;  since the Latvian open data portal is relatively new, it is not included in other studies, while other portals of our neighbours (Lithuania, Estonia, Finland) have been investigated;  in order to bring added value at the international level, this study provides brief comparison with other portals, highlighting the aspects to be taken into account when developing and improving open data portals. RATIONALE FOR THE STUDY Aim: to explore Latvia’s Open Data portal, by evaluating it from different perspectives. Research questions: whether our Open Data Portal is of adequate quality? what are the main challenges to be addressed?
  3. 3.  the usability of open data portals is being studied even less frequently - over the last 5 years, only 6% of studies covering open data portal topic at least a little, mention the usability;  compared to the leading countries, the topic of Latvia’s open data portal is covered nearly 12 times rarely (1.33% of all studies, while for the leading countries this ratio exceeds 15%).;  the popularity of this topic in scientific literature may sometimes (but not necessarily) be linked to the state of the open data portals, as the number of studies demonstrates the public’s (at least the scientific part of society) interest in the subject. STATE OF THE ART despite the increase in popularity of the topics of the OGD and open data portals, the ratio of these researches to the researches dealing with open data does not exceed 7%
  4. 4.  Existing studies: ✘ cover their own national OGD initiatives, which in most cases turns to the an assessment of national open data portal(s), assessing them from different perspectives such as:  data, functionality, features,  stakeholder participation, stakeholder feedback;  the relevance of the data sets of a portal to the “5-stars” classification etc.. ✘ focus mainly on data delivery and the data environment, considering what data providers have done to facilitate users, but have not actually consulted users; ✘ lack a user perspective; ✘ lack a common methodology that would allow comparisons between studies and portals. STATE OF THE ART
  5. 5. STATE OF THE ART. BENCHMARKS AND INDEXES  Global Open Data Index follows the state of the OGD of 94 countries analysing 15 key datasets per country,  only 11% of the data set entries were open according to their open definition, ✘ data are currently available only for 2016 (the evaluation process is quite complex), their finding may be considered outdated;  Open Data Barometer provides a snapshot of OGD practices focusing on open data readiness, implementation, and emerging impacts. ✘ the most urgent, the 4th edition assessed these aspects for a sample of 30 countries in 2017 (113 countries in 2016);  EU Open Data portal assesses 4 key aspects, namely policy, portal, impact and quality. ✔ one of the most up-to-date assessments (is used in the study)  etc.
  6. 6. BRIEFLY ABOUT THE PORTAL Date of the launch: 2017 Owner: Latvian Ministry of Environmental Protection and Regional Development Rank (according to European Data Portal Maturity Report): 11 Number of datasets: 413* Number of publishers: 80* Number of categories: 14 *July 24
  7. 7. Latvia is one of 70 countries participating in the Open Government Partnership - an international platform for domestic reformers that committed to making their governments more open, accountable, and responsive to citizens; BRIEFLY ABOUT THE PORTAL at the moment of its launch 33 data sets from 13 data publishers in July of 2018 139 data sets from 41 publishers in July of 2020 413 data sets from 80 publishers. Latvian open data portal has the highest rate of open data maturity in comparison with neighbourhoods from Baltic States and Scandinavian countries. Languages: Latvian and English ✔ attempt to ensure multilingualism ✘ content uniformity/ homogenity in different languages is not ensured The most popular data formats: .CSV, .XLSX, .WMS ✔ data are machine-readable (is still a problem for many countries) ✔ the number of geospatial data increases significantly – the nature of data is up-to-date
  8. 8.  The impact and portal for Latvia are the worst aspects among impact, policy, portal, and quality (65 and 78%);  Maturity of Latvian open data portal: • in 2016 – 30th, • in 2017 – 21st, • in 2018 - 19th, • in 2018 - 11th, For more detail see Nikiforova, A. (2020) - Comparative analysis of national open data portals or whether your portal is ready to bring benefits from open data. In IADIS International Conference on International Conference on ICT, Society and Human Beings Rank Country 2019 2018 2017 2016 1 France 1 ↑+4 5 ↑+3 8 ↓-5 3 2 Spain 2 ↑+6 8 ↓-3 5 ↓-1 4 3 Ireland 3 3 3 ↑+7 10 4 Cyprus 4 ↓-2 2↑+22 24↓-6 18 5 Finland 5 ↓-4 1 ↑+5 6 ↑+3 9 6 Slovenia 6 ↑+1 7↑+13 20↓-7 13 7 Austria 7 ↑+3 10↓-1 9 ↓-4 5 8 Romania 8 ↓-4 4 4↑+11 15 9 Luxembourg 9 9 ↓-8 1 1 10 Netherlands 10↑+4 14↓-4 10↓-3 7 11 Latvia 11↑+8 19↑+2 21↑+9 30 12 Poland 12↓-1 11↑+12 23↑+1 24 13 Italy 13 13 13↑+6 19 14 Germany 14↑+9 25↓-23 2 ↑+4 6 15 Greece 15↓-9 6↑+20 26 26 16 Croatia 16↑+4 20↓-13 7 ↑+5 12 THE MATURITY OF OPEN DATA PORTALS ACCORDING TO EUROPEAN DATA PORTAL ↓-n – decrease in rank by n positions compared to the previous year ↑+n – improve in rank by n positions compared to the previous year ranking – below EU28+ average *The analysis is carried out by author ✓ Latvia demonstrates the most impressive result in terms of continuous development; ✓ only 4 countries improve their positions from year to year – Ireland, Latvia, Italy, and Malta
  9. 9. Why? (Máchová et al., 2018)*** usability evaluation framework ✔ reflects all the functionality of the portal and typical tasks normally performed by users; ✔ considers a user perspective; ✔ complies with the majority principles used or mentioned by other well-known researchers ((Charalabidis et al., 2018), (Attard et al., 2015), (Zuiderwijk et al., 2015)); ✔ well-cited by other researchers. What does it suppose?  3 categories (data specification, data set feedback, data set request)  14 aspects (see the next slide for detail)  3-point Likert scale (1 – not fullfilled, 2 – partly, 3 - fulfilled) ***Máchová, R. et al. (2018). Usability evaluation of open data portals. Aslib Journal of Information Management STAGE I: a questionnaire of Latvian citizens (105 participants) STAGE II: (Máchová et al., 2018)*** usability evaluation framework applied to the portal by 35 participants with IT background* *well-designed experiment requires at least 30 participants - ✔ STAGE III: additional aspects not covered in the applied methodology (or not detailed enough) analysed by addressing every data set (395 data sets) KEY POINTS
  10. 10.  the number of non-IT experts who are aware of the existence of Latvia’s open data portal is higher than the number of users with IT background by 13%,  this trend is not valid in the case of its use - only 7.7% of non-IT users use this portal regularly, while for IT-users this ratio is 12%.  the assessment received from IT users was lower by 1 point in the case of usability and 0.5 point in the case of quality.  105 voluntary participants representing three groups: (a) users with IT background, (b) without IT background, divided into (b1) undergraduated and (b2) graduated.  A list of simple question, including: (1) whether the participant knows that Latvia has its own national open data portal? (2) how he/ she assesses it (if «yes» for the 1st question) (2.1.) from a “usability” perspective, (2.2.) from the quality of the data.  only 53% of respondents are aware of the existence of Latvia’s open data portal, 13.5% suppose that this exists, however, are not sure,  74.3% of those, who know about the existence of the portal, have not used it, 18% used it only once and only 7.7% use it in a daily manner.  the overall [subjective] usability of the portal was assessed by 6.3 points out of 10, while the quality of published data – 6.2 out of 10. Despite the high results according to international assessment systems, Latvian users are not satisfied with this portal in the highest way. STAGE I: SURVEY
  11. 11. Category Aspect Description Open dataset specification a) Description of dataset Portal provides datasets together with their description and how and for what purpose they were collected b) Publisher of dataset Portal provides information about organization that published datasets c) Thematic categories and tags Portal provides thematic categories of datasets to address the main topics covered. It distinguishes categories (themes) from tags (keywords) d) Release date and up to date Datasets are associated with a time or period tag, that is, date published, date updated and its frequency e) Machine-readable formats Portal provides datasets formats that are machine-readable and allow easy re-use f) Open data licence Portal provides license information related to the use of the published datasets g) Visualization and statistics Portal provides visualization and analytics capabilities to gain information about a dataset, e.g. in charts or visualizations in maps. Open dataset feedback a) Documentation and tutorials Portal provides high quality of documentation and tutorials to help users b) Forum and contact form Portal provides an opportunity to submit feedback on a dataset from the users to providers and forum to discuss and exchange ideas among the users c) User rating and comments Portal provides capabilities allowing the collection of user ratings and comments d) Social media and sharing Portal provides the integration with social media technologies to create a distribution channel for open data and sharing feedback Open dataset request a) Request form Portal provides a form to request or suggest new type or format type of open data b) List of requests Portal provides a list of requests received from users, including the current state of request processing c) Involvement in the process Portal provides capabilities allowing the involvement in the same dataset STAGE II: ASPECTS COVERED IN THE FRAMEWORK TO BE APPLIED
  12. 12. * Categories • I category: open data specification • II category: open data set feedback • III category: open data set request  Result: 31.47 out of 42  Average result: 2.25 out of 3 points  Rank #17 out of 42 countries  Rank #11, if you look on the countries included in EDP rating  When recalculated as a percentage (where not fullfilled is 0), the average level of usability is 62,4% Not bad, BUT, we wish to be better For more detail see Nikiforova, A. (2020) - Comparative analysis of national open data portals or whether your portal is ready to bring benefits from open data. In IADIS International Conference on International Conference on ICT, Society and Human Beings # Country Languages I* II* III* TOTAL Rank (EDP) 1 Cyprus EL, EN 20,2 10,63 7,34 38,17 4 2 Russia RU, EN 16,61 11,39 8,49 36,49 N/A 3 France EN, FR, ES 20,2 9,38 6,59 35,93 1 4 Spain ES, CA, GL, EU, EN 17,97 9,58 8,01 35,56 2 5 Taiwan EN, TW, CN 17,09 10,84 7,06 34,99 N/A 6 Canada EN, FR 17,57 8,97 8,22 34,76 N/A 7 Austria AU 18,84 11,59 4,31 34,74 7 8 Colombia ES, EN 17,59 8,36 8,24 34,19 N/A 9 New Zealand EN 17,67 7,26 8,78 33,71 N/A 10 Ireland EN, GA 17,7 8,79 7,09 33,58 3 11 Portugal EN, FR, ES, PT 19,01 10,69 3,65 33,35 25 12 Finland FI, SV, EN 18,7 10,85 3,35 32,9 5 13 Lithuania LT, EN 15,27 8,19 8,75 32,21 27 14 Slovenia SL +Google Translate 18,71 9,15 4,28 32,14 6 15 India EN 18,39 8,16 5,22 31,77 N/A 16 Netherlands NL 15,97 8,41 7,16 31,54 10 17 Latvia LV, EN 16,43 8,28 6,76 31,47 11 ** 18 USA EN 17,54 8,97 4,63 31,14 N/A 19 Singapore EN 19,08 6,51 4,90 30,49 N/A 20 Estonia EE, EN 16,81 8,66 4,94 30,41 18 21 Slovakia SK, EN 16,08 9,48 4,58 30,14 29 n – competitive in a specific aspect, should be considered as an example n – very weak result, improvement needs to be made, it is worth looking at leaders (in green) RESULTS I
  13. 13. Category Aspect Points (1 to 3 points) Open dataset specification a) Description of dataset 2,07 b) Publisher of dataset 2,74 c) Thematic categories and tags 2,36 d) Release date and up to date 2,9 e) Machine-readable formats 2,1 f) Open data licence 3 g) Visualization and statistics 1,26 Open dataset feedback a) Documentation and tutorials 1,98 b) Forum and contact form 2,02 c) User rating and comments 1,76 d) Social media and sharing 2,52 Open dataset request a) Request form 2,83 b) List of requests 1,88 c) Involvement in the process 2,05 RESULTS I
  14. 14. 0 0,5 1 1,5 2 2,5 3 2,58 2,46 2,64 2,40 2,22 2,63 1,77 2,17 1,94 1,65 2,15 2,06 1,55 1,54 Average rating for 42 countries the aspect is assessed worse than in average for 42 countries the aspect is assessed better than in average for 42 countries good enough should be improved in the future, but can wait must be improved!!! RESULTS II
  15. 15. RESULTS III good enough should be improved in the future, but can wait must be improved!!! Overall OK, but far away from the excellent result and leaders Feedback must be improved NB! Feedback is the worst aspect for open data of all countries
  16. 16. For more detail see Nikiforova, A. (2020) - Comparative analysis of national open data portals or whether your portal is ready to bring benefits from open data. In IADIS International Conference on International Conference on ICT, Society and Human Beings RESULTS IV. COMPARISON WITH 41 COUNTRY
  17. 17. One of the most crucial aspects for open data, which affects users’ intention to reuse data and increase their value, however, is rarely ensured. the frequency of updates promised by the data publisher VS. the actual frequency of updates  3 parameters were analysed: “created”, “last updated” and “frequency” 12,0% 2,7% 14,7% 5,4% 0,5% 4,2% 6,1% 27,7% 0,2% 18,6% 1,0% 6,1% 0,7% 0% 20% 40% 60% 80% 100% % of datasets frequency of updates Values of "frequency of updates" STAGE III. FREQUENCY OF UPDATES 46% 25% 29% Whether the frequency of updates of the data is in line with real life? (%) yes no hard to say  the use of different names designating the same object, or its property was observed (e.g. “constantly” and “constantly updated”)  it is recommended to involve a list of allowable values or to develop and introduce a vocabulary,  values such as “unknown” and “other” should not be allowed since they do not provide supposed information.
  18. 18. STAGE III. WHETHER API IS AVAILABLE? data sharing in the form of machine-readable APIs is more useful for its further use and is one of the most critical success factors for open data initiatives, one of the most significant challenges for open data, and, in the context of Latvia, this is also valid - only 43.04% of datasets (170 out of 395) are supplemented with API feature. 43% 57% Whether API is available? (%) yes no 51% 49% Whether preview option is available? (%) yes no
  19. 19. STAGE III. WHETHER THE DESCRIPTION OF THE PARAMETERS IS AVAILABLE?  a description of parameters are aimed at clarifying, what the name of each parameter means;  only 20% of datasets are supplied with a description of parameters, where the textual description appears to be less popular, since 54.4% of the data sets are supplemented with the description available as a separate file. 21% 79% Whether the description of the parameters is available? (%) YES NO 45% 55% What is their form? text file 93% 7% Whether the description of dataset is provided? (%) YES NO
  20. 20.  Latvia’s open data is re-used;  the most popular topics demonstrate the potential of open data and opportunities to improve citizens’ lives by addressing topics that are crucial for Latvia:  tourism,  calculation queues to the hospitals or kindergarten,  finding parking,  finding the best place of life. !!!It is strongly recommended to publish this information on the portal to demonstrate the positive effects of the use of open data, thereby increasing public participation. STAGE III. USE-CASES OR SHOW- CASES  it is good practice to provide users with information on the number of applications based on open data that have been re-used – use-cases, which are able to attract more people to the portal and its data - these data are not provided.  71.43% of 42 analysed open data portals provide this feature, including our neighbours – Lithuania (9), and Poland (27).  France, Luxembourg or Portugal has even a “use case upload” feature,  18 portals offer a mapping between the use cases and the datasets they are based on, e.g. Ireland. ✘
  21. 21. BUT WE ARE GOOD ENOUGH IN … despite a list of the challenges mentioned above that has not yet been solved, Latvia’s open data portal holds sufficiently high positions in various ratings. Among the positive aspects, there are: ✔ the number of datasets and data publishers are provided and frequently updated; ✔ two languages, namely Latvian and English, are supported; ✔ the majority of datasets are downloadable and have at least basic set of [meta]data on it; ✔ search and filter by category, data format, tags, data publisher; ✔ a social media facility that can help create a social distribution channel for open data; ✔ a request form (appears to be a problem to a list of countries); ✔ a contact facility with data publisher; ✔ guidelines and news, etc. However, some aspects have not been checked on whether they are working as intended, thus, it is difficult to draw any conclusions on their quality - only the fact of their presence is highlighted.
  22. 22.  The study addresses the various researchers’ calls by dividing the analysis into three parts: (a) a survey which studies the knowledge of the Latvian society (105 participants), (b) the application of (Máchová et al., 2018) framework to the Latvia’s open data portal (40 participants), (c) an analysis of additional aspects inspecting each data set (395 datasets).  the current situation is being assessed, and corrective actions are being recommended;  the implementation of the guidelines defined should also improve user participation by encouraging the awareness and reuse of open data, which appears to be one of the most important but weakest aspects;  although the results of this study are more important mainly for open data portal holders, they could also be useful for users, as the results: (a) point to the weakest points to be taken into account using open data portal, (b) provide aspects to be considered when selecting the portal to be used, (c) potential open data publishers can use the results provided as a checklist when preparing their data for publishing. RESULTS
  23. 23.  This study will be continued to carry out an in-depth analysis of the identified challenges  to propose guidelines to address them,  to provide a detailed comparison with more successful open data portals,  a detailed literature review will also follow, In addition, it is planned to find foreign collaborators (preferably from the countries representing the leading portals) to jointly address challenges through exchange of experience. FUTURE WORK
  24. 24. I am grateful to the participants taking part in the experiment within my workshop “Open data and data quality”, which allowed to collect data and led to such results, and Latvian Open Technology Association for sharing information regarding open data hackathons. ACKNOWLEDGMENT
  25. 25. THANK YOU! For more information, see ResearchGate See also anastasijanikiforova.com For questions or any other queries, contact me via email - Anastasija.Nikiforova@lu.lv Article: Nikiforova, A. (2020). Assessment of the usability of Latvia’s open data portal or how close are we to gaining benefits from open data. In In IADIS 14th International Conference on Interfaces and Human Computer Interaction (pp. 51-28).

×