Privatization is the Symptom, Not the CureTom Tresser
Tom Tresser is a Chicago-based educator and activist who is dedicated to championing creativity, fighting privatization and defending the commons. http://www.tresser.com. Hey, SlideShare users - over 2,00 views in one year?! Send me some love, a comment, some acknowledgement that you are using this material - tom@civiclab.us - thanks!
Privatization is the Symptom, Not the CureTom Tresser
Tom Tresser is a Chicago-based educator and activist who is dedicated to championing creativity, fighting privatization and defending the commons. http://www.tresser.com. Hey, SlideShare users - over 2,00 views in one year?! Send me some love, a comment, some acknowledgement that you are using this material - tom@civiclab.us - thanks!
Target / case / переход от использования безадресной рекламы к директ мейлSergey Cherkasov
ARGET - программа развития клиентского портфеля покупок. UPSALE для высокооткликовой аудитории
Ситуация
TAEGET один из крупнейших ритейлеров США основной свой акцент в коммуникациях через почтовый ящик всегда делали на классическом инструменте безадресной рекламы (каталоги специальных предложений и купоны на скидки). Задача повысить в краткосрочный период средний чек и объем продаж.
Идея
Компания выделила наиболее активных клиентов , тех которые по своей природе более предрасположены покупать в Target ( в базу анализа были положены параметры чистоты покупки и среднего чека). Анализ CRM позволил выделить и сформировать индивидуальные предложения более чем 2 000 000 клиентам. В основу анализа покупок клиентов с чеком выше среднего было заложено более 20 параметров в результате чего компания смогла выделить группы товаров, которые могли бы быть интересны каждому из 2 млн. клиентов и сформировать индивидуальное предположение в каждой из 4х основных товарных групп. Персонализация была сделана не только на уровне того какой буклет/каталог получит тот или иной человек , но и в том какую скидку ему индивидуальной назначали на каждую группу товаров. Рекламный буклет имел 4 области, в которые наносились порядка 20 индивидуальных предложений специально для каждого человека (индивидуальный штрих код со скидкой на каждый продукт), наличие штрих кода на каждый из этих купонов позволило быстро оценивать эффективность и точность программы.
Результат
Отклик на рекламную компанию превысил аналогичные компании без индивидуализации предложения на 50%
На 30% сэкономили операционные издержки за счет печати более компактных предложений
Когда: 2015-16г
Где: США
Идустрия RETAIL
Директ-мейл для Политических партий. Political mail / USPSSergey Cherkasov
USPS провело фокус группы и онлайн опрос среди взрослых Американцев, старшего сегмента поколения Millennials.
Оригинал тут http://www.deliverthewin.com/millennials/
Согласно исследования люди этого поколения используют новостные агрегатов для того, чтобы найти новости и понять реальную текущую ситуацию. При этом эти люди воспринимают информацию поступивших им сообщений через директ мейл как более личные и что важно как информацию из первых рук не претерпевшей искажений журналистов, что фактически ценно , поскольку люди сами хотят выносить суждения относительно первоисточника. Люди очень хорошо реагируют на личностный контент (личные фотографии кандидата), которые отражают их повседневную жизнь, исходя из того, что люди все чаще пользуются Twitter и соц сетями с преимущественно коротким контентом, то наиболее хорошо воспринимается информация которая подается такими же короткими блоками и активного подкрепленная графическим рядом.
Если же говорить о цифрах то 82% респондентов ответили, что политически директ-мейл вызывает у ним больше доверия чем другие носители, 74% считают ДМ более безопасным инструментом получения информации о политических партиях и кандидатах чем информация полученная через интернет и другие цифровые каналы. Наиболее эффективными стали комбинации ДМ рассылок и использования оного их ЦИфровых каналов . (например в ДМ рассылки получение возможности подключения к личному кабинету и получения от кандидатов прямой информации по наиболее важным вопросам). Также ДМ остается важным каналом коммуникации в эпоху борьбы людей с цифровым шумом.
un aporte sobre sou fujimoto arquitecto contemporaneo la casa N una de sus obras mas sobresalientes por su forma y por su filosofia en donde prevalece el todo y sus partes
Relational branding – A New Paradigm for Modeling Marketplace Effects of CBRBlackBar Consulting
Presented by keynote speaker Max Blackston at Susan Fournier's Consumer Brand Relationships conference on May 17,2013 at Simmons College in Boston Mass.
The Small Pilot Evaluation - Feedback and Results
/
This report constitutes a background report to the Final report 3 – The Small Pilot Evaluation and the Use of the RD&I Information System for Evaluation. It focuses on the outcomes of the Small Pilot Evaluation (SPE) that was implemented in the context of this study from the month of September 2014 (launch of the preparatory activities) to the end of January 2015 (final panel reports).
This background report holds the guidelines and templates for the implementation of the National Evaluation of Research Organisations (NERO), for the use of the main actors involved (evaluation management, panel experts, referees and research organisations).
Target / case / переход от использования безадресной рекламы к директ мейлSergey Cherkasov
ARGET - программа развития клиентского портфеля покупок. UPSALE для высокооткликовой аудитории
Ситуация
TAEGET один из крупнейших ритейлеров США основной свой акцент в коммуникациях через почтовый ящик всегда делали на классическом инструменте безадресной рекламы (каталоги специальных предложений и купоны на скидки). Задача повысить в краткосрочный период средний чек и объем продаж.
Идея
Компания выделила наиболее активных клиентов , тех которые по своей природе более предрасположены покупать в Target ( в базу анализа были положены параметры чистоты покупки и среднего чека). Анализ CRM позволил выделить и сформировать индивидуальные предложения более чем 2 000 000 клиентам. В основу анализа покупок клиентов с чеком выше среднего было заложено более 20 параметров в результате чего компания смогла выделить группы товаров, которые могли бы быть интересны каждому из 2 млн. клиентов и сформировать индивидуальное предположение в каждой из 4х основных товарных групп. Персонализация была сделана не только на уровне того какой буклет/каталог получит тот или иной человек , но и в том какую скидку ему индивидуальной назначали на каждую группу товаров. Рекламный буклет имел 4 области, в которые наносились порядка 20 индивидуальных предложений специально для каждого человека (индивидуальный штрих код со скидкой на каждый продукт), наличие штрих кода на каждый из этих купонов позволило быстро оценивать эффективность и точность программы.
Результат
Отклик на рекламную компанию превысил аналогичные компании без индивидуализации предложения на 50%
На 30% сэкономили операционные издержки за счет печати более компактных предложений
Когда: 2015-16г
Где: США
Идустрия RETAIL
Директ-мейл для Политических партий. Political mail / USPSSergey Cherkasov
USPS провело фокус группы и онлайн опрос среди взрослых Американцев, старшего сегмента поколения Millennials.
Оригинал тут http://www.deliverthewin.com/millennials/
Согласно исследования люди этого поколения используют новостные агрегатов для того, чтобы найти новости и понять реальную текущую ситуацию. При этом эти люди воспринимают информацию поступивших им сообщений через директ мейл как более личные и что важно как информацию из первых рук не претерпевшей искажений журналистов, что фактически ценно , поскольку люди сами хотят выносить суждения относительно первоисточника. Люди очень хорошо реагируют на личностный контент (личные фотографии кандидата), которые отражают их повседневную жизнь, исходя из того, что люди все чаще пользуются Twitter и соц сетями с преимущественно коротким контентом, то наиболее хорошо воспринимается информация которая подается такими же короткими блоками и активного подкрепленная графическим рядом.
Если же говорить о цифрах то 82% респондентов ответили, что политически директ-мейл вызывает у ним больше доверия чем другие носители, 74% считают ДМ более безопасным инструментом получения информации о политических партиях и кандидатах чем информация полученная через интернет и другие цифровые каналы. Наиболее эффективными стали комбинации ДМ рассылок и использования оного их ЦИфровых каналов . (например в ДМ рассылки получение возможности подключения к личному кабинету и получения от кандидатов прямой информации по наиболее важным вопросам). Также ДМ остается важным каналом коммуникации в эпоху борьбы людей с цифровым шумом.
un aporte sobre sou fujimoto arquitecto contemporaneo la casa N una de sus obras mas sobresalientes por su forma y por su filosofia en donde prevalece el todo y sus partes
Relational branding – A New Paradigm for Modeling Marketplace Effects of CBRBlackBar Consulting
Presented by keynote speaker Max Blackston at Susan Fournier's Consumer Brand Relationships conference on May 17,2013 at Simmons College in Boston Mass.
The Small Pilot Evaluation - Feedback and Results
/
This report constitutes a background report to the Final report 3 – The Small Pilot Evaluation and the Use of the RD&I Information System for Evaluation. It focuses on the outcomes of the Small Pilot Evaluation (SPE) that was implemented in the context of this study from the month of September 2014 (launch of the preparatory activities) to the end of January 2015 (final panel reports).
This background report holds the guidelines and templates for the implementation of the National Evaluation of Research Organisations (NERO), for the use of the main actors involved (evaluation management, panel experts, referees and research organisations).
Spatial planners work in different contexts and have different objectives depending on which country is in question. That is something that became very clear during our collaboration. Therefore an evaluation framework has to be flexible, so that it can be adapted to different contexts.
The evaluation framework we developed presents a set of criteria and indicators that can be used in future evaluations. It gives methodological guidance for evaluating Maritime Spatial Planning processes, with an emphasis on transboundary aspects. The framework could also be used for designing effective planning processes.
Read more about the project www.balticscope.eu
Jocelyn Adkins of the U.S. Environmental Protection Agency Office of General Counsel's
International Environmental Law Practice Group presented information about the SEM Modernization Review Task Force's work.
Pilot test of new evaluation methodology of research organisationsMEYS, MŠMT in Czech
In 2015, the pilot test of a new evaluation methodology of research, development and innovation, in which twelve research organisations
active predominantly in two fields (Chemistry and History) participated, was carried out. Three main and nine subject panels, in which thirty-five
foreign and six local experts were present, prepared evaluation reports of thirty-one registered, field-specific research units. The feedback of the panellists and the institutions evaluated, which are useful for the preparation of a nationwide evaluation of research organisations in the Czech Republic, are the key output of the pilot test.
This report is one of the three Final reports of the study developing an evaluation methodology and institutional funding principles for the R&D system in the Czech Republic. It describes the methodology for the National Evaluation of Research Organisations (NERO) in the Czech Republic.
This report is the one of the three Final reports of a study developing an evaluation methodology and institutional funding principles for the R&D system in the Czech Republic. It describes the new principles for the institutional funding of research organisations (RO) in the Czech Republic.
The Small Pilot Evaluation and the Use of the RD&I Information System for Eva...MEYS, MŠMT in Czech
This report is one of the three Final reports for the study designing a new R&D evaluation methodology and funding principles for the Czech republic. It reports on the outcomes of two distinct strands of analyses, i.e. the Small Pilot Evaluation and the (potential) use of the RD&I Information System (IS) as a tool for information, in particular for evaluation purposes.
Summary Report / R&D Evaluation Methodology and Funding PrinciplesMEYS, MŠMT in Czech
This study has been undertaken under a contract to the Ministry of Education, Youth
and Sports of the Czech Republic by a team from Technopolis, the Technology Centre
ASCR, NIFU, and Infoscience Praha.
V tradičním pojetí jsou rizika chápána negativně, tedy jako hrozby. Tím, že jsou identifikována a vyhodnocena, vytvářejí ale současně příležitosti, které lze využít pro zlepšení výsledku projektu nebo pro snadnější uvádění jeho výsledků do života. Z veřejného projednávání dokumentů IPN Metodika, diskusí na konferencích a zpětné vazby výzkumných organizací, které se účastnily dvou pilotních ověřování nové metodiky hodnocení, byly získány některé podněty, které jsou považovány za rizika pro zavádění národního hodnocení výzkumných organizací (NERO).
Analýza má posloužit k podrobnějšímu rozboru možností a k doporučení a zdůvodnění výběru určitého řešení. Jednoznačná identifikace je důležitá nejen pro výzkumníky, aby mohli prezentovat výsledky své vědecké činnosti, ale využívá se též pro hodnocení autorů a pracovišť, vydavatelům a poskytovatelům finančních podpor zjednodušuje administrativu a usnadňuje organizaci databází. Zavedením trvalého jedinečného identifikátoru výzkumníka se vyřeší jednoznačné přiřazení výstupů a dalších profesních aktivit danému vědeckému pracovníkovi.
Dokument je zpracovaný na základě zkušeností z pilotního ověření navrhované metodiky hodnocení výzkumných organizací a z diskusí s odbornou veřejností během celé doby trvání projektu. Zaměřen je nejen na změny vnitřních předpisů veřejných vysokých škol, ale také na dlouhodobé záměry vysokých škol. Jsou formulována doporučení pro veřejné vysoké školy na jedné straně a MŠMT na straně druhé.
Podklady a doporučení pro zapracování do věcného záměru zákona nahrazujícího ...MEYS, MŠMT in Czech
Dokument analyzuje, zda vůbec a popřípadě v jaké míře jsou nutné změny regulatorního rámce v oblasti výzkumu, vývoje a inovací, které by si vyžádala realizace návrhů vzešlých z výstupů Individuálního projektu národního Efektivní systém hodnocení a financování výzkumu, vývoje a inovací. Jsou zohledněny i probíhající záměry úprav legislativních předpisů.
Harmonogram postupných kroků realizace návrhů nového hodnocení a financování ...MEYS, MŠMT in Czech
Tato implementační doporučení zpracoval expertní tým IPN Metodika v závěrečné fázi projektu na základě zkušeností získaných během úzké spolupráce se zpracovatelem studie „Metodika hodnocení ve výzkumu a vývoji a zásady financování“ společností Technopolis Group z Velké Británie dále také díky bohatým zkušenostem z realizace pilotního ověření navržené metodiky hodnocení týmem projektu, díky intenzivním diskusím se zainteresovanými subjekty, s odbornou veřejností a zahraničními experty po celou dobu trvání projektu. Doporučení jsou výsledkem podrobných diskusí realizačních aspektů v rámci širokého expertního týmu IPN Metodika.
Pilotní ověření návrhu nové metodiky hodnocení výzkumných organizacíMEYS, MŠMT in Czech
Dokumenty shrnují výsledky pilotního ověření metodiky NERO hodnocení výzkumné činnosti vycházející ze závěrů a doporučení Technopolisu, které bylo realizováno týmem projektu v průběhu roku 2015.
The French Revolution, which began in 1789, was a period of radical social and political upheaval in France. It marked the decline of absolute monarchies, the rise of secular and democratic republics, and the eventual rise of Napoleon Bonaparte. This revolutionary period is crucial in understanding the transition from feudalism to modernity in Europe.
For more information, visit-www.vavaclasses.com
Biological screening of herbal drugs: Introduction and Need for
Phyto-Pharmacological Screening, New Strategies for evaluating
Natural Products, In vitro evaluation techniques for Antioxidants, Antimicrobial and Anticancer drugs. In vivo evaluation techniques
for Anti-inflammatory, Antiulcer, Anticancer, Wound healing, Antidiabetic, Hepatoprotective, Cardio protective, Diuretics and
Antifertility, Toxicity studies as per OECD guidelines
Macroeconomics- Movie Location
This will be used as part of your Personal Professional Portfolio once graded.
Objective:
Prepare a presentation or a paper using research, basic comparative analysis, data organization and application of economic information. You will make an informed assessment of an economic climate outside of the United States to accomplish an entertainment industry objective.
Unit 8 - Information and Communication Technology (Paper I).pdfThiyagu K
This slides describes the basic concepts of ICT, basics of Email, Emerging Technology and Digital Initiatives in Education. This presentations aligns with the UGC Paper I syllabus.
Acetabularia Information For Class 9 .docxvaibhavrinwa19
Acetabularia acetabulum is a single-celled green alga that in its vegetative state is morphologically differentiated into a basal rhizoid and an axially elongated stalk, which bears whorls of branching hairs. The single diploid nucleus resides in the rhizoid.
How to Make a Field invisible in Odoo 17Celine George
It is possible to hide or invisible some fields in odoo. Commonly using “invisible” attribute in the field definition to invisible the fields. This slide will show how to make a field invisible in odoo 17.
Introduction to AI for Nonprofits with Tapp NetworkTechSoup
Dive into the world of AI! Experts Jon Hill and Tareq Monaur will guide you through AI's role in enhancing nonprofit websites and basic marketing strategies, making it easy to understand and apply.
Embracing GenAI - A Strategic ImperativePeter Windle
Artificial Intelligence (AI) technologies such as Generative AI, Image Generators and Large Language Models have had a dramatic impact on teaching, learning and assessment over the past 18 months. The most immediate threat AI posed was to Academic Integrity with Higher Education Institutes (HEIs) focusing their efforts on combating the use of GenAI in assessment. Guidelines were developed for staff and students, policies put in place too. Innovative educators have forged paths in the use of Generative AI for teaching, learning and assessments leading to pockets of transformation springing up across HEIs, often with little or no top-down guidance, support or direction.
This Gasta posits a strategic approach to integrating AI into HEIs to prepare staff, students and the curriculum for an evolving world and workplace. We will highlight the advantages of working with these technologies beyond the realm of teaching, learning and assessment by considering prompt engineering skills, industry impact, curriculum changes, and the need for staff upskilling. In contrast, not engaging strategically with Generative AI poses risks, including falling behind peers, missed opportunities and failing to ensure our graduates remain employable. The rapid evolution of AI technologies necessitates a proactive and strategic approach if we are to remain relevant.
Honest Reviews of Tim Han LMA Course Program.pptxtimhan337
Personal development courses are widely available today, with each one promising life-changing outcomes. Tim Han’s Life Mastery Achievers (LMA) Course has drawn a lot of interest. In addition to offering my frank assessment of Success Insider’s LMA Course, this piece examines the course’s effects via a variety of Tim Han LMA course reviews and Success Insider comments.
1. Pilotní ověření návrhu nové metodiky
hodnocení výzkumných organizací
Pilot Test of New Evaluation Methodology
of Research Organisations
Samostatný doplňující dokument 3:
Zápisy z kalibračních schůzek
Background document 3:
Minutes of Calibration Meetings
2. List of content
1 Calibration meeting of panel chairs, Disciplinary area Engineering & Technology ................................................3
2 Calibration meeting of panel chairs, Disciplinary area Humanities ........................................................................7
3 Calibration meeting of panel chairs, Disciplinary area Natural Sciences .............................................................12
Tento dokument byl zpracován v rámci Individuálního projektu národního pro oblast terciárního vzdělávání, výzkumu
a vývoje „Efektivní systém hodnocení a financování výzkumu, vývoje a inovací, CZ.1.07/4.1.00/33.0003“. Projekt byl
realizován Ministerstvem školství, mládeže a tělovýchovy a financován prostřednictvím Operačního programu
Vzdělávání pro konkurenceschopnost z Evropského sociálního fondu a státního rozpočtu České republiky.
This document has been prepared as a part of the Individual National Project for the area of Tertiary Education,
Research and Development „Effective System of Research Financing, Development and Innovation,
CZ.1.07/4.1.00/33.0003“). The project was realised by the Ministry of Education, Youth and Sports and financed by
the Operational Programme Education for Competitiveness of the European Social Fund and state budget of the Czech
Republic.
Dokument „Minutes of Calibration Meetings“ neprošel jazykovou korekturou.
3. 3
1 Calibration meeting of panel chairs, Disciplinary area
Engineering & Technology
Prague, National Technical Library, May 14, 2015
Approved by Vlastimil Růžička and Jonathan Seville
Participants list
Name Abbr. Panel
Prof. Jonathan Seville JS 2- Engineering and Technology
Prof. François Lapicque FL 2.4 Chemical Engineering
Prof. Tapio Salmi (replaces Prof. G. de With) TS 2.5 Materials Engineering
Prof.Wim Rulkens, excused WR 2.7 Environmental Engineering
Prof. Henryk Jeleń HJ 2.9 Industrial Biotechnology
prof. Jiří Hanika JH Main panel member
Ing. Ivan Souček IS Main panel member
Prof. Vlastimil Růžička VR Pilot testing project manager
Prof. Vladimír Majer VM Pilot testing project expert
Ing. Hana Bartková HB Pilot testing project expert
Ing. Tomáš Kopřiva TK Pilot testing project expert
Ing. Markéta Tiptová MT Secretary of main panel 2
Bc. Karolína Šedivcová KS Secretary of panel 2.7
Mgr. Kamila Gabrielová KG MEYS
Agenda:
Introduction: Vlastimil Růžička
Evaluation methodology, pilot testing: Vlastimil Růžička
Main and subject panel members: Vlastimil Růžička
Calibration exercise: Vlastimil Růžička plus panel members
Preparation of panels meeting, June 29 to July 3, Prague
Conclusions
4. 4
1. Calibration exercise
1.1. Research environment
Panel members discussed the description of criteria (as given in the presentation of VR), but hasn´t proposed
any modification.
Panel members agreed upon the following weighting of sub-criteria:
Sub-criteria
Weight/%
ScRO IBRO PSRO NatRes
The quality of the research management (including HR management) 65 80 80 80
The adequacy of the research strategy 35 20 20 20
1.2. Membership of the national and global research community
IS noted that for IBRO type the two sub-criteria are strongly influenced by the “mother company” (if there is
some) and its outreach (be in national or international).
Panel members agreed upon the following weighting of sub-criteria:
Sub-criteria
Weight/%
ScRO IBRO PSRO NatRes
Level of participation and recognition at the international level 60 50 50 50
Level of participation and recognition at the national level 40 50 50 50
Weights in PSRO category are identical to those in NatRes category
1.3. Research excellence
There was a common conclusion that terms originality, significance and rigour are not field specific, and they
have a general meaning.
VM presented a modification by the pilot testing project team of description of ranking of excellent outputs in
the template for referees.
1.4. Overall research performance
Panel members agreed upon the following weighting of sub-criteria:
Sub-criteria
Weight/%
ScRO IBRO PSRO NatRes
Research output 50 80 90 90
Competitiveness in research 50 20 10 10
Weights in PSRO category are identical to those in NatRes category
1.5. Societal relevance
There was a common conclusion that terms reach and significance are widely applicable, and are not field
specific.
5. 5
1.6. Calibration for different types of RO
VR presented a table from the Second Interim Report (see Fig.16, Weights in the default scenario) and explained its
meaning, i.e. its use in decision about funding allocation. Panel members suggested modifying some figures
(original figures are crossed out, new figures in red).
Assessment criterion ScRO IBRO PSRO NatRes
Research environment 15 20/ 15 20 20
Membership of the national and global research
community
10 5 5 5
Scientific research excellence 20 5/ 10 5 5
Overall research performance 50/ 45 50 40 40
Societal relevance 5/ 10 20 30 30
1.7. General discussion
Research Organisations (“ROs”) were free to decide to which subject panel they will register its Research Unit/s
(“RU/RUs”). In some cases the panel members may see a more suitable subject panel for evaluation of the RO/RU.
For the future evaluation some more guidance may be needed.
It was suggested that at the beginning of the subject panel meeting it would be more appropriate to start the
discussion of results of individual remote ranking of all RUs together rather than taking RUs one by one in
sequential order. The suggested procedure will facilitate a mutual comparison among RUs.
JS brought up a confidentiality issue. In RAE/REF in the UK ranking of outputs by referees is not disclosed.
VR presented a suggestion of the IPN Metodika team1 to make the membership in main as well as subject panels
public prior to the meeting of panels in Prague at the end of June. Names of referees will not be disclosed. VR also
agreed with the proposal of JS to not disclose the ranking of excellent outputs as this might result in some
undesirable consequence.
2. Preparation of panels meeting, June 29 to July 3, Prague
2.1. Due to time limitations of prof. Jonathan Seville the meeting will start on Tuesday June 30 by the
morning session of the main and all subject panels’ members. The panel chairs, their deputies and panel
members are kindly asked to arrive in Prague on Monday June 29 in the evening. As the number of RUs in
each subject panel is three or below it was agreed to finish the meeting on Thursday July 2 in the evening.
2.2. The morning session of all panel members will be immediately followed by subject panel meetings. In the
meetings subject panel members will discuss their individual draft evaluation reports with the objective to
reach a consensual final assessment of each RU. The subject panel meetings should be concluded on
Wednesday July 1 in the evening.
2.3. On Thursday July 2 all panels will meet together to discuss drafts of subject panel reports.
2.4. Site-visits, their appropriateness was discussed. The panel members agreed that they might visit some RUs,
depending upon time availability. Some expressed their scepticisms in the value of site-visits, and also of
their fairness. Some considered the site-visits important in particular visiting the facilities.
2.5. Preparation of consolidated report on the EvU level was discussed. It was agreed that on the last day,
Thursday July 2, the panel chairs will meet for a short meeting. They will draft a consolidated EvU report,
1 IPN Metodika team is the team that runs the EU financed project to develop an “Effective Evaluation and Financing System for Research,
Development and Innovation”
6. 6
then return back to the panels and debate it also within the panel. Finally the consolidated EvU report will be
finalised by the main panel chair prof. Jonathan Seville.
2.6. It was agreed that preparing a conclusive subject field report as well as disciplinary area report was not
possible in the pilot testing.
2.7. Panel members agreed to prepare the draft evaluation report for each RU by the end of July. JS will review
them. JS will also draft comments to the Evaluation methodology.
2.8. Subject panel chairs will receive from the IPN Metodika team the list of names of members, their contact
details, and their C.V. These lists will also be sent to prof. Jonathan Seville.
2.9. All subject panel members as well as the chair of the disciplinary area main panel (prof. Jonathan Seville)
will receive the following information:
2.9.1. User name and password to log on into the on-line system storing all relevant information for RUs to be
evaluated by the corresponding panel (self-evaluation report, bibliometric report, integrated report on
excellent outputs with referees´ scores, template for panel report, etc.)
2.9.2. Guidelines to facilitate evaluation process, namely understanding self-evaluation report and related
items.
2.9.3. Self-evaluation reports of relevant RUs as well as bibliometric reports should be available by June 5 the
latest.
2.9.4. Integrated report on excellent outputs should be available by June 21 the latest.
3. Administrative remarks
3.1. All international panel chairs and members present at the meeting are asked to send invoice for the air ticket
plus boarding passes, invoice for hotel accommodation to the respective panel assistant (see the participants
list on page 1) for preparing the reimbursement of cost.
3.2. Hotel International will be booked by the administration team for the meeting from June 29 till July 3.
3.3. Panel members are asked to use public transport on their way to and from the airport when possible.
3.4. The administration team will help panel members in filling in the time-sheets. These have to be handed over to
the administration team at the beginning of next month, based on the notification. The remuneration of experts
work will be taxed according to the Czech legislation. By the first quarter of 2016 panel members will receive
income reports containing the amount of deducted taxes in the Czech Republic.
4. List of tasks
Who Description Deadline
Panel chairs/members-foreign
Send your travel plans, in particular the air travel plan, to
Andrea Weinbergerová (andrea.weinbergerova@msmt.cz)
for approval. Please plan to arrive in Prague on Monday
June 29 in the late afternoon/evening.
asap
Jitka Pošvová, Andrea
Weinbergerová
Send to subject panel chairs list of names of members, their
contact details, and their C.V.
May 31
IPN Metodika team
User name and password to log on into the on-line system
for all panel members
June 5
Subject panel chairs/members
Prepare individually quality level scores and draft evaluation
report for each RU (remote work).
June 29
7. 7
2 Calibration meeting of panel chairs, Disciplinary area
Humanities
Prague, National Technical Library, May 20, 2015
Prepared by: David Pavlorek, Matěj Petráček, Gabriela Strádalová, Ludmila Štěpánová
Modified and approved by: Vlastimil Růžička
Approved by: Michael North
Participants list
Name Abbr. Panel
Prof. Michael North MN 6 Main panel Humanities
prof. Leoš Müller LM 6.1 History and Archeology
Prof. Frank Hadler FH 6.1 History and Archeology
Prof. Dr. Jakob De Roover JR 6.3 Philosophy, Ethics and Religion
Prof. Jiří Pešek JP Main panel member
Assoc.Prof. Václav Ledvinka, excused VL Main panel member
Prof. Vlastimil Růžička VR Pilot testing project manager
Prof. Petr Vorel PV Pilot testing project expert
Ing. Tomáš Kopřiva TK Pilot testing project expert
Assoc.Prof. Daniel Münich DM Methodology project guarantor
Mgr. Jitka Pošvová JP MEYS2
Mgr. Kamila Gabrielová KG MEYS
Bc. Ludmila Štěpánová LS Main panel assistant
Mgr. David Pavlorek DP Assistant of panel History and Archeology
Matěj Petráček MP Assistant of panel Languages and Literature
Gabriela Strádalová GS Assistant of panel Philosophy, Ethics and Religion
Agenda:
Introduction: Vlastimil Růžička
Evaluation methodology, pilot testing: Vlastimil Růžička
Main and subject panel members: Vlastimil Růžička
Calibration exercise: Vlastimil Růžička plus panel members
2
Ministry of Education, Youth and Sports
8. 8
Preparation of panels meeting, July 6 to July 10, Prague
Conclusions
1. Calibration exercise
1.1. Research environment
Panel members discussed the description of criteria (as given in the presentation of VR), and haven´t
proposed any modification.
Panel members agreed upon the following weighting of sub-criteria:
Sub-criteria
Weight/%
ScRO3 IBRO PSRO NatRes
The quality of the research management (including HR management) 40 n.a. 40 60
The adequacy of the research strategy 60 n.a. 60 40
1.2. Membership of the national and global research community
Panel members agreed upon the following weighting of sub-criteria:
Sub-criteria
Weight/%
ScRO IBRO PSRO NatRes
Level of participation and recognition at the international level 70 n.a.
Level of participation and recognition at the national level 30 n.a.
For PSRO and NatRes it was suggested to use examples from similar R&D evaluations from other countries. All panel
chairs agreed they will not consider sub-criteria as field specific and will use them only in the explanatory part of the
evaluation report.
1.3. Research excellence
There was a common conclusion that terms originality, significance and rigour are not field specific.
VR presented a modification by the pilot testing project team of description of excellent outputs ranking in the
template for referees.
It was noted that not originality, but innovative approach is important. Adapting something from previous cases.
Originality from different point of view – it could be original from the national point of view, but not
internationally. Significance can be viewed nationally and internationally.
3 ScRO=Scientific Research Organizations; IBRO=Industry and Business services Research Organizations; PSRO=Public Service Research
Organizations; NatRes=Natural Resources. This classification is slightly different from that presented in the First Interim Report, and is currently
presented in the document “Research Organizations and the Effects of the EM thresholds”
9. 9
1.4. Overall research performance
Panel members agreed upon the following weighting of sub-criteria:
Sub-criteria
Weight/%
ScRO IBRO PSRO NatRes
Research output 40 n.a. 40 40
Competitiveness in research 60 n.a. 60 60
1.5. Societal relevance
There was a common conclusion that terms reach and significance are widely applicable, and are not field
specific.
1.6. General discussion
1.6.1. Panel chairs and members agreed to follow the suggestion from the Engineering and Technology main
panel calibration meeting held on May 14, 2015 that at the beginning of the subject panel meeting it
would be more appropriate to start the discussion of results of individual remote ranking of all RUs
together rather than taking RUs one by one in sequential order. The suggested procedure will facilitate a
mutual comparison among RUs.
1.6.2. VR presented a suggestion of the IPN Metodika team4 to make the list of members in main as well as
subject panels public prior to the meeting of panels in Prague at the end of June. Names of referees will
not be disclosed. VR also presented a proposal to not disclose the ranking of excellent outputs as this
might result in some undesirable consequence.
1.6.3. Present subject panel chairs were asked by VR whether they would agree to become the second referee
of excellent outputs in the case of need. Currently Vladimír Majer, Petr Vorel and Hana Bartková (pilot
testing project experts) are organizing the selection and communication with referees. Those panel
members eventually agreeing to undertake the job would get a new contract and remuneration for
refereeing.
1.6.4. Present panel chairs emphasized the need of having a list of names of all researchers in the research
unit being evaluated as being particularly important for Humanities. Also, a list of titles and authors of all
books published by authors from the research unit in the evaluation period should be available for panel
experts. FH asked for supplying international reviews of books submitted for research excellence criterion
evaluation.
1.6.5. There was a discussion on the role of Ph.D. students in Czech research organizations. VR explained that
we are not evaluating individuals but research units and event. Evaluated units. Basically PhD students
are included in the current evaluation; there are some parts in the self-evaluation report related to Ph.D.
students. FH noted that in Germany once the Ph.D. student completed his/her research, he/she has to
publish a book, otherwise will not get the degree.
1.6.6. A need of receiving a list of abbreviations was stated.
4 IPN Metodika team is the team that runs the EU financed project to develop an “Effective Evaluation and Financing System for Research,
Development and Innovation”
10. 10
2. Preparation of panels meeting, July 6 to July 10, Prague
2.1. All meetings will start on Monday July 6. The panel chairs, their deputies and panel members are kindly
asked to arrive in Prague on Sunday July 5 in the evening.
2.2. VR asked the subject panel chairs to kindly carry out within the panel a calibration exercise, similar to that
carried out on May 21 in Prague. They will probably use a teleconference, a Skype conference or similar
ways of mutual communication. The presentation of VR in PPT format will be provided. The calibration
exercise should be completed within the first half of June, prior to remote assessment of RUs.
2.3. The morning session of all panels will be immediately followed by subject panel meetings. In the meetings
subject panel members will discuss their individual draft evaluation reports with the objective to reach a
consensual final assessment of each RU.
2.4. On Wednesday July 8 in the afternoon the main and the subject panel chairs should convene for a
discussion of the drafts of subject panel reports in each subject panel.
2.5. Site-visits, their appropriateness was discussed. The panel members were keen to visit some RUs,
depending upon time availability. The IPN Metodika team will come up with a proposal for site-visits
obtaining also prior consent of the visited EvU/RU.
2.6. Consolidated report on the EvU level was discussed. It was suggested that on Thursday July 9, the panel
chairs will meet for a short meeting. They will debate whether it is feasible to draft a consolidated EvU
report. If agreed the consolidated EvU report would be finalised by the main panel chair prof. Michael North.
2.7. It was agreed that preparing a conclusive subject field report as well as disciplinary area report was not
possible in the pilot testing.
2.8. Panel members agreed to prepare the draft evaluation report for each RU by the end of July. MN will review
them. MN will also draft comments to the Evaluation methodology.
2.9. Subject panel chairs will receive from the IPN Metodika team the list of names of members, their contact
details, and their C.V. These lists will also be sent to prof. Michael North.
2.10. All subject panel members as well as the chair of the disciplinary area main panel (prof. Michael North) will
receive the following information:
2.10.1. User name and password to log on into the on-line system storing all relevant information for RUs to be
evaluated by the corresponding panel (self-evaluation report, bibliometric report, integrated report on
excellent outputs with referees´ scores, template for panel report, etc.)
2.10.2. Guidelines to facilitate evaluation process, namely understanding self-evaluation report and related
items.
2.10.3. Self-evaluation reports of relevant RUs as well as bibliometric reports should be available by June 5 the
latest.
2.10.4. Integrated report on excellent outputs should be available by June 21 the latest.
3. Administrative remarks
3.1. All international experts present at the meeting were asked to send invoice for the air ticket plus boarding
passes, invoice for hotel accommodation to the respective panel assistant (see the participants list on page 1)
for preparing the reimbursement of cost.
3.2. Hotel International will be booked by the administration team for the meeting from July 5 to July 10.
3.3. Panel members are asked to use public transport on their way to and from the airport when possible.
3.4. The administration team will help panel members in filling in the time-sheets. These have to be handed over to
the administration team at the beginning of next month, based on the notification. The remuneration of experts
work will be taxed according to the Czech legislation. By the first quarter of 2016 panel members will receive
income reports containing the deducted taxes in the Czech Republic.
11. 11
4. List of tasks
Who Description Deadline
Panel chairs/members-foreign
Send your travel plans, in particular the air travel plan, to
Andrea Weinbergerová (andrea.weinbergerova@msmt.cz)
for approval. Please plan to arrive in Prague on Sunday
July 5 in the late afternoon/evening.
asap
Jitka Pošvová, Andrea
Weinbergerová
Send to subject panel chairs list of names of members,
their contact details, and their C.V.
June 5
IPN Metodika team
User name and password to log on into the on-line system
for all panel members
June 5
Subject panel chairs/members
Prepare individually quality level scores and draft
evaluation report for each RU (remote work).
July 5
IPN Metodika team
Provide overhead projectors into meeting rooms for subject
panels
June 29
12. 12
3 Calibration meeting of panel chairs, Disciplinary area
Natural Sciences
Prague, National Technical Library, May 21, 2015
Prepared by: Ondřej Dvouletý, Helena Kvačková, Alexandra Riva
Modified and approved by: Vlastimil Růžička
Approved by: Erik Thulstrup
Participants list
Name Abbr. Panel
Professor Erik W. Thulstrup ET 1 Main panel Natural Sciences
Dr. Philippe Hapiot PH 1.4 Chemical Sciences
Prof. Arnold JM Driessen AD 1.6 Biological Sciences
Ing. Martin Matějka - excused MM Main panel member
Prof. Zdeněk Němeček ZN Main panel member
Ing. Václav Rejholec VRe Main panel member
Prof. Vlastimil Růžička VRu Pilot testing project manager
Prof. Vladimír Majer VM Pilot testing project expert
Ing. Hana Bartková HB Pilot testing project expert
Mgr. Jitka Pošvová JP MEYS5
Mgr. Kamila Gabrielová KG MEYS
Bc. Ondřej Dvouletý OD Main panel assistant
RNDr. Helena Kvačková HK Chemical panel assistant
Alexandra Riva AR Biological panel assistant
Agenda:
Introduction: Vlastimil Růžička
Evaluation methodology, pilot testing: Vlastimil Růžička
Main and subject panel members: Vlastimil Růžička
Calibration exercise: Vlastimil Růžička plus panel members
Preparation of panels meeting, June 29 to July 3, Prague
Conclusions
5
Ministry of Education, Youth and Sports
13. 13
1. Calibration exercise
1.1. Research environment
Panel members discussed the description of criteria (as given in the presentation of VR), and haven´t
proposed any modification.
Panel members agreed upon the following weighing of sub-criteria:
Sub-criteria
Weight/%
ScRO6 IBRO PSRO NatRes
The quality of the research management (including HR management) 50 50 50 50
The adequacy of the research strategy 50 50 50 50
1.2. Membership of the national and global research community
Panel members agreed upon the following weighing of sub-criteria:
Sub-criteria
Weight/%
ScRO IBRO PSRO NatRes
Level of participation and recognition at the international level 60 40 10 10
Level of participation and recognition at the national level 40 60 90 90
1.3. Research excellence
There was a common conclusion that terms originality, significance and rigour are not field specific.
VRu presented a modification by the pilot testing project team of description of ranking of excellent outputs in
the template for referees.
1.4. Overall research performance
Panel members agreed upon the following weighing of sub-criteria:
Sub-criteria
Weight/%
ScRO IBRO PSRO NatRes
Research output 60-70 30-40 80-90 80-90
Competitiveness in research 40-30 70-60 20-10 20-10
1.5. Societal relevance
There was a common conclusion that terms reach and significance are widely applicable, and are not field
specific.
6
ScRO=Scientific Research Organizations; IBRO=Industry and Business services Research Organizations; PSRO=Public Service Research
Organizations; NatRes=Natural Resources. This classification is slightly different from that presented in the First Interim Report, and is currently
presented in the document “Research Organizations and the Effects of the EM thresholds”
14. 14
1.6. General discussion
1.6.1. Panel chairs and members agreed to follow the suggestion from the Engineering and Technology main
panel calibration meeting held on May 14, 2015 that at the beginning of the subject panel meeting it
would be more appropriate to start the discussion of results of individual remote ranking of all RUs
together rather than taking RUs one by one in sequential order. The suggested procedure will facilitate a
mutual comparison among RUs.
1.6.2. VRu presented a suggestion of the IPN Metodika team7 to make the list of members in main as well as
subject panels public prior to the meeting of panels in Prague at the end of June. Names of referees will
not be disclosed. VRu also presented a proposal to not disclose the ranking of excellent outputs as this
might result in some undesirable consequence.
1.6.3. Present subject panel chairs were asked by VRu whether they would agree to become the second
referee of excellent outputs in the case of need. Currently VM, Petr Vorel (pilot testing project expert in
Humanities) and HB are organizing the selection and communication with referees. Those panel
members eventually agreeing to undertake the job would get a new contract and remuneration for
refereeing.
1.6.4. Potential bias resulting from wrong registrations of RUs into subject panels by EvUs: Panel members
mentioned that there are some RUs which are cross-disciplinary as some EvUs are very diverse. It could
have been recommended putting them to another panel. However, due to limited time available a
recommendation for improving guidelines leading to better understanding of registration into a subject
panel will be included in the final feedback to the evaluation methodology principles.
2. Preparation of panels meeting, June 29 to July 3, Prague
2.1. All meetings will start on Monday June 29. It will be decided in a joint discussion of panel chairs whether a
joint morning session of the main and all subject panels’ members will be held, mainly due to severe time
limitation of the Chemical Sciences panel having nine research units for evaluation. The panel chairs, their
deputies and panel members are kindly asked to arrive in Prague on Sunday June 28 in the evening.
2.2. VRu asked both subject panel chairs to kindly carry out within the panel a calibration exercise, similar to that
carried out on May 21 in Prague. They will probably use a teleconference, a Skype conference or similar
ways of mutual communication. The presentation of VR in PPT format will be provided. The calibration
exercise should be completed within the first half of June, prior to the remote assessment of RUs.
2.3. The morning session of all panels (if it will be organized) will be immediately followed by subject panel
meetings. In the meetings subject panel members will discuss their individual draft evaluation reports with
the objective to reach a consensual final assessment of each RU.
2.4. The Biological Sciences subject panel meeting should be concluded on Wednesday July 1 in the evening.
The Chemical Sciences subject panel meeting will go on until Friday July 3.
2.5. On Wednesday July 1 in the afternoon the main and the subject panel chairs should convene for a
discussion of the progress and drafts of subject panel reports in the Biological Sciences subject panel, and
discussion of the progress in the Chemical Sciences subject panel.
2.6. Site-visits, their appropriateness was discussed. The panel members agreed that they might visit some RUs,
depending upon time availability. Importance of setting up the agenda of the site-visit, provision of the draft
evaluation report to the EvU/RU management prior to the visit was also discussed. The draft evaluation
report will not be finished before the site-visit. Two site-visits were deemed better than just one, expert
panels may be divided into smaller groups. The IPN Metodika team will come up with a proposal for site-
visits obtaining also prior consent of the visited EvU/RU.
2.7. Preparation of a consolidated report on the EvU level was discussed. It was suggested that on Wednesday
July 1, the panel chairs will meet for a short meeting. They will debate whether it is feasible to draft a
7 IPN Metodika team is the team that runs the EU financed project to develop an “Effective Evaluation and Financing System for Research,
Development and Innovation”
15. 15
consolidated EvU report, in particular as some EvUs registered their RUs to subject panels belonging to two
disciplinary areas, viz. Natural Sciences and Engineering and Technology. Inclusion of prof. Jonathan
Seville, the Engineering and Technology main panel chair, in the discussion will be necessary. If agreed the
consolidated EvU report would be finalised by the main panel chair prof. Erik Thulstrup.
2.8. It was agreed that preparing a conclusive subject field report as well as disciplinary area report was not
possible in the pilot testing.
2.9. Panel members agreed to prepare the draft evaluation report for each RU by the end of July. ET will review
them. ET will also draft comments to the Evaluation methodology.
2.10. Subject panel chairs will receive from the IPN Metodika team the list of names of members, their contact
details, and their C.V. These lists will also be sent to prof. Erik Thulstrup.
2.11. All subject panel members as well as the chair of the disciplinary area main panel (prof. Erik Thulstrup) will
receive the following information:
2.11.1. User name and password to log on into the on-line system storing all relevant information for RUs to be
evaluated by the corresponding panel (self-evaluation report, bibliometric report, integrated report on
excellent outputs with referees´ scores, template for panel report, etc.)
2.11.2. Guidelines to facilitate evaluation process, namely understanding self-evaluation report and related
items.
2.11.3. Self-evaluation reports of relevant RUs as well as bibliometric reports should be available by June 5 the
latest.
2.11.4. Integrated report on excellent outputs should be available by June 21 the latest.
3. Administrative remarks
3.1. Erik Thulstrup, Arnold Driessen, and Philipe Hapiot were asked to send invoice for the air ticket plus boarding
passes, invoice for hotel accommodation to the respective panel assistant (see the participants list on page 1)
for preparing the reimbursement of cost.
3.2. Hotel International will be booked by the administration team for the meeting from June 28 to July 3.
3.3. Panel members were asked to use public transport on their way to and from the airport when possible.
3.4. The administration team will help panel members in filling in the time-sheets. These have to be handed over to
the administration team at the beginning of next month, based on the notification. The remuneration of experts
work will be taxed according to the Czech legislation. By the first quarter of 2016 panel members will receive
income reports containing the amount of deducted taxes in the Czech Republic.
4. List of tasks
Who Description Deadline
Panel chairs/members-foreign
Send your travel plans, in particular the air travel plan, to Andrea
Weinbergerová (andrea.weinbergerova@msmt.cz) for approval.
Please plan to arrive in Prague on Sunday June 28 in the late
afternoon/evening.
asap
Jitka Pošvová, Andrea
Weinbergerová
Send to subject panel chairs list of names of members, their
contact details, and their C.V.
May 31
IPN Metodika team
User name and password to log on into the on-line system for all
panel members
June 5
Subject panel chairs/members
Prepare individually quality level scores and draft evaluation report
for each RU (remote work).
June 29
IPN Metodika team Provide overhead projectors into meeting rooms for subject panels June 29
16. Pilotní ověření návrhu nové metodiky hodnocení výzkumných organizací
Samostatný doplňující dokument 3
Zápisy z kalibračních schůzek
Vydává Ministerstvo školství, mládeže a tělovýchovy, Karmelitská 7, Praha 1
Individuální projekt národní pro oblast terciárního vzdělávání, výzkumu a vývoje:
Efektivní systém hodnocení a financování výzkumu, vývoje a inovací (IPN Metodika)
www.metodika.reformy-msmt.cz
Praha 2015