Despite the existence of data analysis tools such as R, SQL, Excel and others, it is still insufficient to cope with today's big data analysis needs.
The author proposes a CUI (Character User Interface) toolset with dozens of functions to neatly handle tabular data in TSV (Tab Separated Values) files.
It implements many basic and useful functions that have not been implemented in existing software with each function borrowing the ideas of Unix philosophy and covering the most frequent pre-analysis tasks during the initial exploratory stage of data analysis projects.
Also, it greatly speeds up basic analysis tasks, such as drawing cross tables, Venn diagrams, etc., while existing software inevitably requires rather complicated programming and debugging processes for even these basic tasks.
Here, tabular data mainly means TSV (Tab-Separated Values) files as well as other CSV (Comma Separated Value)-type files which are all widely used for storing data and suitable for data analysis.
Despite the existence of data analysis tools such as R, SQL, Excel and others, it is still insufficient to cope with today's big data analysis needs.
The author proposes a CUI (Character User Interface) toolset with dozens of functions to neatly handle tabular data in TSV (Tab Separated Values) files.
It implements many basic and useful functions that have not been implemented in existing software with each function borrowing the ideas of Unix philosophy and covering the most frequent pre-analysis tasks during the initial exploratory stage of data analysis projects.
Also, it greatly speeds up basic analysis tasks, such as drawing cross tables, Venn diagrams, etc., while existing software inevitably requires rather complicated programming and debugging processes for even these basic tasks.
Here, tabular data mainly means TSV (Tab-Separated Values) files as well as other CSV (Comma Separated Value)-type files which are all widely used for storing data and suitable for data analysis.
Theory to consider an inaccurate testing and how to determine the prior proba...Toshiyuki Shimono
I presented a mathematical theory on a medical testing method. This fundamental theory can be taken account of both cases when the resource of the testing is limited or not. One implication is that "negative proof" may not function well, and another implication is that excessively high specificity and accuracy are required for meaningful diagnosis unless the careful usage of the diagnosis is considered.
To Make Graphs Such as Scatter Plots Numerically Readable (PacificVis 2018, K...Toshiyuki Shimono
Different-sized discrete crosses placed in an organized lattice pattern can assist the human eyes to read numerical values on statistical graphs, enabling more precise interpretation and enlarging the utility of statistical graphs that visually represent numerical quantities. This paper presents a novel graph-plotting method that places roughly ten thousand of separated grids on a graph, providing human data analysis with an easy access to arbitrary numerical readouts from a statistical graph. At present, this functionality has been lacking in the existing graph-plotting softwares.
To Make Graphs Such as Scatter Plots Numerically Readable (PacificVis 2018, K...Toshiyuki Shimono
Different-sized discrete crosses placed in an organized lattice pattern can assist the human eyes to read numerical values on statistical graphs, enabling more precise interpretation and enlarging the utility of statistical graphs that visually represent numerical quantities. This paper presents a novel graph-plotting method that places roughly ten thousand of separated grids on a graph, providing human data analysis with an easy access to arbitrary numerical readouts from a statistical graph. At present, this functionality has been lacking in the existing graph-plotting softwares.
Make Accumulated Data in Companies Eloquent by SQL Statement Constructors (PDF)Toshiyuki Shimono
Presented at IEEE BigData 2017, Boston, on Dec 11, 2017
in the Workshop of "3rd International Workshop on Methodologies to Improve Big Data projects".
The author is Toshiyuki Shimono, Digital Garage, Inc.
(This is PDF format instead of MS Powerpoint format for the sake of significantly smaller file size.)