The document summarizes feedback from a town hall meeting at the 2011 International Semantic Web Conference (ISWC). Key discussion points included feedback on new events like the Outrageous Ideas track, the balance of workshops and tutorials, improving the paper reviewing process, metrics for identifying influential papers, and ideas for improving future ISWC conferences. Attendees were encouraged to provide any other thoughts or feedback to the Semantic Web Science Association.

1. ISWC 2011 Town Hall

2. What is this meeting all about?
• Main goal: get feedback from the conference
participants (you!) on what we should do
better or different next year
• Focus: the scientific content of the
conference, not the logistics

3. New events this year
• Should we keep them? Should we do them
differently?
– Outrageous ideas track
– Minute madness
– Meet the editors
– Linked Data-a-thon
– Future funding by EU and beyond
• Ideas for special sessions for next year?

4. Tracks
• Do we have too many tracks? Too few?
– Research track
– In-use track
– Industry track
• Do you know where you should submit?

5. Workshops and Tutorials
• Balance different this year: more workshops
(16), fewer tutorials (5)
• Is this balance better than 50-50?
• Ideas for workshops and tutorials next year
– What would you like to see?

6. The Panels
• 2 Panels this year
– Meet Editors
(Informational)
– Deathmatch
(Debate)
• Do we want panels?
– More or less?
– What kind?

7. Events for students
• How can we continue to increase the prestige
of the Doctoral Consortium? Should we try?
• Other events for students
– Ideas?

8. Conference attendance
• What makes it hard for people to attend the
conference? (Please, limit to the things that
we can change!)
– Tight schedule?
– Not enough venues to present something?
– Other?

9. Outrageous Ideas
Research Papers

10. Outrageous Ideas Track
• 24 Submissions
• 5 selected by PC members for presentation
• 19 available online for public vote
• PC & Chairs
• Criteria
– Is the idea outrageous?
– Is it something new?
– Are the reasons it is outrageous made clear?
– Are concrete use cases given?
– Are the challenges to realizing the idea made clear?
– Is this idea important for the development of our community?

11. Outrageous Ideas Stats (18p)
• Challenges – 2.4 (0.58)
• Importance to field – 2.2 (0.57)
• Use cases – 2.1 (0.48)
• Presentation of idea – 1.9 (0.58)
• Outrageousness - 1.6 (0.73)
• Novelty – 1.5 (0.66)

12. The Reviewing

13. Reviews Process
• Author provide title, keywords, abstract
• Reviewers bid (primarily on title)
– not all reviewers bid (and/or don’t indicate topics)
• Author provide full papers
– 1/3 abstracts are not submitted as full papers
• Review assignment based on bids (3 rev. per paper)
• Initial notification of authors & rebuttals
• Reviews discussion and editing, considering rebuttals
• Meta-reviews
• Special Emergency Review Team (SERT)
• SPC Meeting

14. Quality of Reviews
• Consistency problems between reviewers
– Numerical scoring
– Review quality
• Rebuttal consideration
• Summer period
– Late reviews
– Participation in discussion
• Proactiveness in SPC members
– Meta-reviews not always reflect all three reviews
– Comments not integrated in meta-reviews
– Chasing missing reviews
• Maybe a reviewer training program?
– Quantitatively the same outcome

15. Rebuttals
– Useful?
– Infuriating?
– Your thoughts as an author?
– Your thoughts as a reviewer?
– Should other tracks (in-use) have it?

16. SWSA 10-Year Award
• We used Google Scholar citations for papers
from SWWS 2001 + sanity check
• There is no perfect way to assess impact
• Is there a better proxy than citation counts?
What is it? Does it need to be “objective”?

17. What’s an
Influential Paper?
what metadata do
we need to
identify reliably
influential papers?

18. Top cited papers from 2001 SWWS
• 338: Anupriya Ankolekar, Mark H. Burstein, Jerry R. Hobbs, Ora Lassila,
David L. Martin, Sheila A. McIlraith, Srini Narayanan, Massimo Paolucci,
Terry R. Payne, Katia P. Sycara, Honglei Zeng "DAML-S: Semantic Markup
for Web Services"
• 273: Harold Boley, Said Tabet, Gerd Wagner "Design Rationale for
RuleML: A Markup Language for Semantic Web Rules”
• 226: Diego Calvanese, Giuseppe De Giacomo, Maurizio Lenzerini "A
Framework for Ontology Integration”
• 225: Michel C. A. Klein, Dieter Fensel "Ontology versioning on the
Semantic Web”
• 220: Jane Hunter "Adding Multimedia to the Semantic Web: Building an
MPEG-7 ontology”

19. Past ISWC Best Papers
• ISWC-2009: Chris Welty & Lora Aroyo: On the Semantic Web [5] Rank 30/53,
max 43, 5 papers > 20.
• ISWC-2008: Chris Welty & Lora Aroyo Large Data on the Semantic Web [66].
Rank 3/57, max 70, 4 papers > 50.
• ISWC-2007: Chris Welty & Lora Aroyo. Data on the Semantic Web [27]
• ISWC-2006: Chris Welty & Lora Aroyo. Queries on the Semantic Web [212]. Rank
1/72, 6 papers > 100. bad data
• ISWC-2005: Chris Welty & Lora Aroyo. Social Networks on the Semantic Web.
[447] Rank 1/72, 8 papers>100. bad data
• ISWC-2004: Chris Welty & Lora Aroyo. Knowledge Bases on the Semantic Web
[170] Rank 8/54, max 489. 17 papers > 100.
• ISWC-2003: Chris Welty & Lora Aroyo. Reasoning on the Semantic Web. [70]
Rank 23/54, max 498. 20 papers>100
• ISWC-2002: Chris Welty & Lora Aroyo. Ontologies on the Semantic Web. [36]
Rank 23/41, max 1890. 12 papers > 100.

20. Past ISWC Best Papers
• ISWC-2009: Ugur Kuter & Jennifer Golbeck: Semantic Web Service Composition
in Social Environments [5] Rank 30/53, max 43, 5 papers > 20.
• ISWC-2008: Matthew Horridge, Bijan Parsia & Ulrike Sattler for Laconic and
Precise Justifications in OWL [66]. Rank 3/57, max 70, 4 papers > 50.
• ISWC-2007: Dimitris Zeginis, Yannis Tzitzikas and Vassilis Christophides. On the
Foundations of Computing Deltas between RDF models [27]
• ISWC-2006: Marcelo Arenas, Jorge Perez and Claudio Gutierrez. Semantics and
Complexity of SPARQL [212]. Rank 1/72, 6 papers > 100. bad data
• ISWC-2005: Peter Mika. Ontologies are us: A unified model of social networks
and semantics. [447] Rank 1/72, 8 papers>100. bad data
• ISWC-2004: Y. Guo, Z. Pan, and J. Heflin. An Evaluation of Knowledge Base
Systems for Large OWL Datasets [170] Rank 8/54, max 489. 17 papers > 100.
• ISWC-2003: Aimilia Magkanaraki, Val Tannen, Vassilis Christophides, Dimitris
Plexousakis. Viewing the Semantic Web through RVL Lenses. [70] Rank 23/54,
max 498. 20 papers>100
• ISWC-2002: Peter F. Patel-Schneider, Jérôme Siméon. Building the Semantic Web
on XML. [36] Rank 23/41, max 1890. 12 papers > 100.

21. What is SWSA?
• Semantic Web Science Association (SWSA)
– A committee of 12-15 members (iswsa.org)
– Main role: manage the ISWC conference series
• Decide on the location of each conference
– Calls for bids are out ~2.5 years before the conference
– Rotates location Americas/Europe/Asia-Pacific
• Appoint and approve general chair and program chairs
for the conference
• If you have feedback on ISWC in general and ideas
for future conferences, email
swsa-feedback@lists.uni-karlsruhe.de

22. Any other thoughts? Gripes?
• What can we do better next year?
Email to: swsa-feedback@lists.uni-karlsruhe.de