This is a slightly modified version of the slides presented at AACL 2018, Atlanta, Georgia. All the graphs on the slides are created by CasualConc using R.

1. Going beyond simple word-list
creation using CasualConc
Yasuhiro IMAO
Osaka University, Japan
casualconc@gmail.com
AACL 2018 at Georgia State University, Atlanta GA

2. A few questions
How many of you are Mac users?
How many of you have used CasualConc?

3. A few observations
Through attending presentations / reading papers
Methods of analysis
Use of statistics
Hugely depend on
the access to the resources
the tools one uses
specialized application
programing skills
someone who can write scripts

4. To advance the field
more easy-to-use and accessible tools are necessary

5. Current Situation
AntConc and Antxxx
and other more small specialized application
WordSmith Tools / Monoconc Pro
The gold standard?

6. Introducing CasualConc

7. A little bit of background
I started developing a concordancer around 2005
I released the first, limited version around 2008
It is a Mac native app!
KWIC, Word/n-gram Lists, Collocation

9. Before going into more detail

10. Small Scale Corpus Research
Building your own specialized corpus
Possibly adding annotations (POS, syntactic, etc.)
Which tools to use?

11. A suggestion (not the answer)
I have developed few companion apps
CasualTranscriber (transcription helper)
CasualTextractor (text extractor/editor)
CasualTagger (tagging helper)
CasualPConc (parallel concordancer)

12. CasualTagger
KWIC search + short-cut tag insertion
Batch Processing
Tagging (TreeTagger, STF CoreNLP, MeCab)
Tokenizing (macOS built-in tagger)
Sentence Splitting (macOS built-in tagger, STF CoreNLP)
Misspelling detection (non-dictionary words)
compound-2-word variation detection

13. CasualTagger

16. Finally, the main point!

17. Today’s highlights
Corpus file management - going beyond loading files
More informative word/n-gram lists
Individual file word/n-gram lists
More keyword extraction functions - going beyond LL/χ2
Visualizing frequency data - incl. multivariate analyses
Some other niche functions

18. Corpus File Management

19. Corpus File Management
Accept Drag & Drop!

20. Where were those files I want to use?
Of course, you can use Spotlight to search them, but…
If the application remembers where they are…

21. Corpus File Management

22. Corpus File Management

23. More Informative Word/n-gram List

24. Proportion, # of Files, % of Files

25. Proportion, # of Files, % of Files

26. 3-grams / 4-grams comparison

27. p-frames

28. p-frames

29. Lemma / Spelling variations/ multi-word

30. Lemma / Spelling Variation

31. Multi-word Recognition

32. Multi Filters

33. Search Word List

34. Individual File Word/n-gram Lists

35. Individual File/Corpus Word/n-gram Lists

36. More keyword extraction functions

37. Sample
ICNALE - Writing
Written learner English corpus
College students in Asian countries/regions
JPN, CHN, HKG, IDN, KOR, PAK, PHI, SIN, THA, TWN
Two topics
Ave. 220-230 words

38. Simple Keyword Extraction (LL)

39. Multiple Keyword Statistics

40. Key-2-gram Extraction

41. Key-3-gram Extraction

42. Going Beyond Simple Keyness

43. Multiple Keyness Comparisons

44. TF-IDF
(term-frequency / inverse document frequency)

45. TF-IDF
(term-frequency / inverse document frequency)

46. Mann-Whitney-U (using multiple files)

47. Random Forest (1000 words)

48. Visualizing Frequency Data

49. CasualConc integrates

50. Word Cloud

51. Let’s go back to keyword analysis

52. Simple Keyword Extraction (LL)

53. Keyword analysis is done
How can you check the validity?
Let’s see how well they separate the groups
Use the LL result above 6.63 (p < .01)

54. Principle Component Analysis (PCA)

55. Cluster Analysis

56. What about different ability levels?

57. Principle Component Analysis (PCA)

58. Cluster Analysis

59. Heat Map

60. By the way, do you check raw data?
Not a lot of people really look at the data…
Let’s check the use of “I think”

61. Box Plot

62. Histogram

63. What about a word
that are too common to be looked at?

64. Box Plot

65. Histogram

66. Some other misc features…

67. Relative Position Plot (paragraph)
therefore / however
Japanese
Non-Japanese
ENS

68. Relative Position Polot (sentence)
therefore / however
Japanese
Non-Japanese
ENS

69. Vocab Profiler

70. Tokenizer

71. Tokenizer

72. Proper Name Extraction

73. The Way Forward
Utilizing Stanford CoreNLP - dependency information

74. Grammatical Relationship Search

75. CasualConc
I just released version 2.1.0 with the updated manual
The manual is full of screenshots with over 250 pages
It is a FREEWARE
Downloadable from
https://sites.google.com/site/casualconc
Or just google ‘casualconc’