Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
What Parnas72

Means for D
Luís Marques, DConf 2016, May 4th 2016

luis@luismarques.eu
Reproducing the Classics
• My experience growing
artificial societies
Growing Artificial Societies
• Sugarscape
Growing Artificial Societies
• Sugarscape
Growing Artificial Societies
• Sugarscape
SettingsSettings
World speed:
Agents:
World resources:
Minimum vision:
Maximum vi...
Growing Artificial Societies
• Sugarscape
Animation B-3. WealthHistogramEvolutionunderRules({GI), {M,
R[60,IOO]}) from aRan...
What Parnas72 Means for D
• Who’s Parnas?
• What’s Parnas72?
• Electrical engineer
• PhD student of Alan Perlis
• Applies traditional engineering
principles to software design
• Criti...
Parnas72
On the Criteria
To Be Used in
Decomposing
Systems into
Modules
Programming R. Morris
Techniques Editor
On the Cri...
Parnas72(b)
• Popularizes information-hiding modules
• See Parnas72a: “Information Distribution Aspects of Design
Methodol...
Parnas72(b)
• The documentation story
• Philips Computer Industry
• A manager asked for help on creating work assignments
...
Modularization
• A module is a work/responsibility assignment
• A class, a D module, a component, etc.
• “The modularizati...
Example
• KWIC Index program
• First, two Parnas modularizations
• Then, an idiomatic D modularization
KWIC
• “The KWIC index system accepts an ordered
set of lines, each line is an ordered set of
words, and each word is an o...
KWIC
KWIC
KWIC
Descent of Man
The Ascent of Man
The Old Man and The Sea
A Portrait of The Artist As a Young Man
(Key Word In Context...
KWIC
a portrait of the ARTIST as a young man
the ASCENT of man
DESCENT of man
a portrait of the artist as a young MAN
desc...
KWIC
a portrait of the ARTIST as a young man
the ASCENT of man
DESCENT of man
a portrait of the artist as a young MAN
desc...
KWIC
a portrait of the ARTIST as a young man
the ASCENT of man
DESCENT of man
a portrait of the artist as a young MAN
desc...
KWIC
A Portrait of The Artist As a Young Man
The Ascent of Man
Descent of Man
A Portrait of The Artist As a Young Man
Desc...
KWIC
Artist As a Young Man, A Portrait of The
Ascent of Man, The
Descent of Man
Man, A Portrait of The Artist As a Young
M...
KWIC
Artist As a Young Man, A Portrait of The
Ascent of Man, The
Descent of Man
Man, A Portrait of The Artist As a Young
M...
KWIC
Artist As a Young Man A Portrait of The
Ascent of Man The
Descent of Man
Man A Portrait of The Artist As a Young
Man ...
KWIC
Artist As a Young Man A Portrait of The
Ascent of Man The
Descent of Man
Man A Portrait of The Artist As a Young
Man ...
KWIC
A Portrait of The Artist As a Young Man
a Young Man A Portrait of The Artist As
and The Sea The Old Man
Artist As a Y...
KWIC
A Portrait of The Artist As a Young Man
a Young Man A Portrait of The Artist As
and The Sea The Old Man
Artist As a Y...
KWIC
• Input: a sequence of lines
• Line: a sequence of words
• Word: a sequence of characters
• Circular shift:
• foo bar...
A Tale of 2 Decompositions
• Parnas’ two decompositions reimplemented in D
• <https://github.com/luismarques/parnas72>
Decomposition 1 (D1)
• Idea:
• The flowchart method
• The classic method
• Data-oriented design
• Module == collection of p...
Decomposition 1 (D1)
canonical representation of input
all circular shifts of input lines

(no order requirements)
all cir...
D1 - Input
• Module 1: Input. This module reads the data lines
from the input medium and stores them in core for
processin...
D1 - Input
Foo
import std.utf;
public:
// change these definitions, for different performance
alias LineNum = ptrdiff_t;
a...
D1 - Input
• Input:
Descent␣of␣Man↵The⇥Ascent⇥␣of␣Man␣␣↵The␣
Old␣Man␣and␣The␣Sea↵␣␣↵A␣Portrait␣of␣The␣Ar
tist␣As␣a␣Young␣M...
// change these definitions, for different performance/capability trade-offs:
alias LineNum = ptrdiff_t;
alias WordNum = p...
// Reads the lines from the input medium, and stores each word separated by a
// `wordSeparator` character constant. The L...
data = readText(filename)
.lineSplitter
// normalize the spacing between words
.map!(line => line
.splitter!isWhite
.filte...
// normalize the spacing between words
.map!(line => line
.splitter!isWhite
.filter!(c => !c.empty)
.joiner([wordSeparator...
// normalize the spacing between words
.map!(line => line
.splitter!isWhite
.filter!(c => !c.empty)
.joiner([wordSeparator...
// normalize the spacing between words
.map!(line => line
.splitter!isWhite
.filter!(c => !c.empty)
.joiner([wordSeparator...
// remove empty lines
.filter!(line => !line.empty)
// keep an index of where each line starts
.tee!((line) {
lineIndex ~=...
// remove empty lines
.filter!(line => !line.empty)
// keep an index of where each line starts
.tee!((line) {
lineIndex ~=...
D1 - Circular Shift
• Module 2: Circular Shift. This module is called
after the input module has completed its work. It
pr...
D1 - Circular Shift
import std.algorithm;
import std.range;
import std.typecons;
import std.utf;
import one.input;
public:...
D1 - Circular Shift
Circular
shifter
Input
.map!(a => a.value.byCodeUnit
.enumerate // (0, ((0, char1), (1, char2
.splitte...
D1 - Circular Shift
{
LineNum lineNum;
CharNum firstChar;
}
ShiftIndexEntry[] shiftIndex;
void setup()
{
shiftIndex = iota...
shiftIndex = iota(lineIndex.length)
.map!(lineNum => line(lineNum))
.enumerate
.map!(a => a.value.byCodeUnit
.enumerate
.s...
shiftIndex = iota(lineIndex.length)
.map!(lineNum => line(lineNum))
.enumerate
.map!(a => a.value.byCodeUnit
.enumerate
.s...
shiftIndex = iota(lineIndex.length)
.map!(lineNum => line(lineNum))
.enumerate
.map!(a => a.value.byCodeUnit
.enumerate
.s...
shiftIndex = iota(lineIndex.length)
.map!(lineNum => line(lineNum))
.enumerate
.map!(a => a.value.byCodeUnit
.enumerate
.s...
shiftIndex = iota(lineIndex.length)
.map!(lineNum => line(lineNum))
.enumerate
.map!(a => a.value.byCodeUnit
.enumerate
.s...
(0, [[Descent][of]
(1, [[The][Ascent]
shiftIndex = iota(lineIndex.length)
.map!(lineNum => line(lineNum))
.enumerate
.map!...
(0, [[Descent][of][Man]])

(1, [[The][Ascent][of][Man]])
shiftIndex = iota(lineIndex.length)
.map!(lineNum => line(lineNum...
(0, [0, 8, 11])
(1, [14, 18, 25, 28])
[0, 8, 11]

[14, 18, 25, 28]
shiftIndex = iota(lineIndex.length)
.map!(lineNum => li...
shiftIndex = iota(lineIndex.length)
.map!(lineNum => line(lineNum))
.enumerate
.map!(a => a.value.byCodeUnit
.enumerate
.s...
shiftIndex = iota(lineIndex.length)
.map!(lineNum => line(lineNum))
.enumerate
.map!(a => a.value.byCodeUnit
.enumerate
.s...
D1 - Alphabetizing
• Module 3: Alphabetizing. This module takes as
input the arrays produced by modules 1 and 2. It
produc...
D1 - Alphabetizing
import std.typecons;
import std.utf;
import one.input;
public:
struct ShiftIndexEntry
{
LineNum lineNum...
[Ascent␣of␣Man][The]
[The␣Ascent␣of␣Man]
[Ascent␣of␣Man␣The]
private:
auto line(ShiftIndexEntry entry)
{
auto a = entry.fi...
D1 - Alphabetizing
module one.alphabetizer;
import std.algorithm;
import std.math;
import std.range;
import std.typecons;
...
D1 - Output
• Module 4: Output. Using the arrays produced by
module 3 and module 1, this modules produces a
nicely formatt...
D1 - Output
void printLines()
{
shiftIndex.map!(entry => entry.line).each!writeln;
}
private:
auto line(ShiftIndexEntry en...
D1 - Outputmodule one.output;
import std.algorithm;
import std.range;
import std.stdio;
import one.input;
import one.circu...
D1 - Master Control
• Module 5: Master Control. This module does little
more than control the sequencing among the other
f...
D1 - Master Control
module one.control;
import one.input;
import one.circularshifter;
import one.alphabetizer;
import one....
D1 - Result
A Portrait of The Artist As a Young Man
a Young Man A Portrait of The Artist As
and The Sea The Old Man
Artist...
Decomposition 2 (D2)
• Based on:
• Difficult design decisions, or design decisions
which are likely to change (the secret)
...
Decomposition 2 (D2)
control
Output
Alphabetizer
Circular
shifter
Input
Line Storage
D2 - Line Storage
• Module 1: Line Storage. This module consists of a number of
functions(…).
• WORD
• SETWRD
• WORDS
• LI...
D2 - Line Storage
Definition of a "Line Storage" Module
Introduction: This definition specifies a mechanism which
hold up ...
D2 - Line Storage
call
call
call
call
call
call
ERLWEL
ERLWNL
ERLWEW
ERLWNW
ERLWEC
ERLWNC
if
if
if
if
if
if
1 <
1 >
w <
w ...
D2 - Line Storage
Definition of a "Line Storage" Module
Introduction: This definition specifies a mechanism which
hold up ...
D2 - Line Storage
• The function call CHAR(r,w,c) will have as value an
integer representing the cth character in the rth ...
D2 - Line Storage
• Functions
• CHAR
• SETCHAR
• WORDS
• LINES
• DELWRD
• DELLINE
• CHARS
D2 - Line Storage
module two.linestorage;
import std.algorithm;
import std.range;
import std.utf;
public:
// change these ...
D2 - Line Storage
void removeLine(LineNum lineNum)
{
assert(lineNum >= 0 && lineNum < numLine
assert(false, "not implement...
D2 - Line Storage
// --
private:
enum wordSeparator = ' ';
char[] data;
CharNum[] lineIndex;
auto line(LineNum lineNum)
{
...
D2 - Line Storage
alias CharNum = ptrdiff_t;
enum maxLines = LineNum.max; /// (original name: p1)
enum maxWordsPerLine = W...
/// Returns one character from a given word, from line `lineNum`.
/// (original name: WORD)
char wordChar(LineNum lineNum,...
/// Returns one character from a given word, from line `lineNum`.
/// (original name: WORD)
char wordChar(LineNum lineNum,...
/// Returns one character from a given word, from line `lineNum`.
/// (original name: WORD)
char wordChar(LineNum lineNum,...
/// Returns one character from a given word, from line `lineNum`.
/// (original name: WORD)
char wordChar(LineNum lineNum,...
/// Returns one character from a given word, from line `lineNum`.
/// (original name: WORD)
char wordChar(LineNum lineNum,...
/// Returns one character from a given word, from line `lineNum`.
/// (original name: WORD)
char wordChar(LineNum lineNum,...
/// Sets the next character for the current or the next word.
/// The current word is the last word present at the last li...
D2 - Line Storage
/// Sets the next character for the current or the next word.
/// The current word is the last word pres...
D2 - Input
• Module 2: Input. This module reads the original
lines from the input media and calls the line storage
module ...
D2 - Inputmodule two.input;
import std.stdio;
import std.uni;
import two.linestorage;
public:
void readLines(string inputF...
void readLines(string inputFile)
{
LineNum lineNum;
foreach(line; File(inputFile).byLine)
{
WordNum wordNum;
CharNum charN...
D2 - Circular Shifter
• Module 3: Circular Shifter. The principal functions provided by this
module are analogs of functio...
D2 - Circular Shifter
private:
WordNum storageWordNum = (entry.firstWor
storage.numWords(entry.lineNum);
return storage.nu...
D2 - Circular Shifter
public:
enum maxLines = LineNum.max; /// (original name: p4)
alias LineNum = storage.LineNum;
alias ...
/// (original name: CSSTUP)
void setup()
{
shiftIndex = iota(storage.numLines)
.map!(a => storage.numWords(a).iota
.map!(b...
/// (original name: CSSTUP)
void setup()
{
shiftIndex = iota(storage.numLines)
.map!(a => storage.numWords(a).iota
.map!(b...
/// (original name: CSSTUP)
void setup()
{
shiftIndex = iota(storage.numLines)
.map!(a => storage.numWords(a).iota
.map!(b...
/// (original name: CSSTUP)
void setup()
{
shiftIndex = iota(storage.numLines)
.map!(a => storage.numWords(a).iota
.map!(b...
/// (original name: CSSTUP)
void setup()
{
shiftIndex = iota(storage.numLines)
.map!(a => storage.numWords(a).iota
.map!(b...
/// (original name: CSSTUP)
void setup()
{
shiftIndex = iota(storage.numLines)
.map!(a => storage.numWords(a).iota
.map!(b...
D2 - Circular Shifter
/// (original name: CSWRDS)
LineNum numWords(LineNum lineNum)
{
auto entry = shiftIndex[lineNum];
re...
D2 - Circular Shifter
void setup()
{
shiftIndex = iota(storage.numLines)
.map!(a => storage.numWords(a).iota
.map!(b => Sh...
D2 - Circular Shifter
/// (original name: CSCHRS)
CharNum numCharacters(LineNum lineNum, WordNum wordNum)
{
auto entry = s...
D2 - Alphabetizing
• Module 4: Alphabetizing. (…) ITH(i) will give the
index of the circular shift which comes ith in the
...
auto alphabeticIndex()
{
auto indexOffsets = new LineNum[numLines];
makeIndex!((a, b) => a.icmp(b) < 0)(numLines.iota.map!...
D2 - Alphabetizing
private:
ReturnType!alphabeticIndex index;
auto alphabeticIndex()
{
auto indexOffsets = new LineNum[num...
D2 - Alphabetizing
import two.circularshifter;
public:
/// (original name: ALPH)
void setup()
{
index = alphabeticIndex;
}...
D2 - Output
• Module 5: Output. This module will give the
desired printing of the set of lines or circular shifts.
D2 - Output
.line)
.each!writeln;
}
// --
private:
auto line(LineNum lineNum)
{
return numWords(lineNum)
.iota
.map!(wordN...
D2 - Output
import std.stdio;
import std.utf;
import two.circularshifter;
import two.alphabetizer;
public:
void printLines...
D2 - Result
A Portrait of The Artist As a Young Man
a Young Man A Portrait of The Artist As
and The Sea The Old Man
Artist...
Comparing Decompositions
• The runtime representation of both decompositions
might be the same
• D2 might have a performan...
Comparing Decompositions
• Amenability to change
• Comprehensibility
• Testability
• Parallel Development
Changeability
• Input format
• Store all data in memory
• Pack the characters
• Create an index of the shifts vs

store th...
Changeability
Module D1 D2
Input ✔ ✔
Line Storage —
Circular
Shifter
Alphabetizer
Output
• Input format
• Store all data i...
Changeability
Module D1 D2
Input ✔
Line Storage — ✔
Circular
Shifter
✔
Alphabetizer ✔
Output ✔
• Input format
• Store all ...
Changeability
Module D1 D2
Input ✔
Line Storage — ✔
Circular
Shifter
✔
Alphabetizer ✔
Output ✔
• Input format
• Store all ...
Changeability
Module D1 D2
Input
Line Storage —
Circular
Shifter
✔ ✔
Alphabetizer ✔
Output ✔
• Input format
• Store all da...
Changeability
Module D1 D2
Input
Line Storage —
Circular
Shifter
Alphabetizer ✔ ✔
Output ✔
• Input format
• Store all data...
Comprehensibility
• Example: understanding the output module
Comprehensibility
• Example: understanding the output module
• Decomposition 1
module one.output;
import std.algorithm;
im...
Comprehensibility
• Example: understanding the output module
import one.circularshifter;
void printLines()
{
shiftIndex.ma...
Comprehensibility
• Example: understanding the output module
• Decomposition 2
import std.algorithm;
import std.range;
imp...
Comprehensibility
• Example: understanding the output module
• Decomposition 2
{
numLines
.iota
.map!(lineNum => lineNum
....
Comprehensibility
• Example: understanding the output module
• Decomposition 2
// --
private:
auto line(LineNum lineNum)
{...
Testability
• Parnas disputes the idea that information hiding is an
empirical result
• It’s a mathematical theorem:
1.You...
Testability
Module D1 D2
Input ✔
Line Storage — ✔
Circular
Shifter
✔
Alphabetizer ✔
Output ✔
Parallel Development
• Decomposition 1
 • Decomposition 2
M1

System design



M2
 …

M1

System design


M2
 …

Evaluation
• Information hiding: Yay or Nay?
• Fred Brook’s The Mythical
Man Month
Evaluation — MMM
"Show me your flowcharts and conceal your tables,
and I shall continue to be mystified. Show me your
tables...
Evaluation — MMM
"Parnas (…) has proposed a still more radical
solution. His thesis is that the programmer is most
effecti...
Evaluation — MMM
"Parnas Was Right, and I Was Wrong about Information
Hiding



(…) I am now convinced that information hi...
Evaluation
• Information hiding: Yay or Nay?
• Fred Brooks 👍
• Name
• Is decomposition 2 sufficiently good?
Master, teach me the true ways of information hiding
Accidental Complexity
• We already removed some accidental complexity:
• Byte-oriented vs word-oriented
• Unicode vs weird...
Accidental Complexity
• Yet, D2 still has a lot of issues:
• Global state
• Lack of constructors / initializers
• Each fun...
Sequence Interface
• Input: a sequence of lines
• Line: a sequence of words
• Word: a sequence of characters
Sequence Interface
f o o ␣ b a r b a z
• Input: a sequence of lines
• Line: a sequence of words
• Word: a sequence of char...
Sequence Interface
• Input: a sequence of lines
• Line: a sequence of words
• Word: a sequence of characters
f o o
b a z
b...
Sequence Interface
• Functions
• CHAR
• SETCHAR
• WORDS
• LINES
• DELWRD
• DELLINE
• CHARS
f o o ␣ b a r b a z
Sequence Interface
• Functions
• CHAR
• SETCHAR
• WORDS
• LINES
• DELWRD
• DELLINE
• CHARS
f o o ␣ b a r b a z
E.g. CHAR(r...
Sequence Interface
• Functions
• DELLINE
• LINES
• DELWRD
• WORDS
• CHARS
• CHAR
• SETCHAR
Sequence Interface
• Functions
• DELLINE
• LINES
• DELWRD
• WORDS
• CHARS
• CHAR
• SETCHAR
f o o
b a z
b a r
line
line
wor...
Sequence Interface
• Functions
• DELLINE
• LINES
• DELWRD
• WORDS
• CHARS
• CHAR
• SETCHAR
f o o
b a z
b a r
pointer
point...
Sequence Interface
• Functions
• DELLINE
• LINES
• DELWRD
• WORDS
• CHARS
• CHAR
• SETCHAR
Sequence Interface
• Functions
• DELLINE
• LINES
• DELWRD
• WORDS
• CHARS
• CHAR
• SETCHAR
• Input: a sequence of lines
• ...
Push vs Pull
Algorithm 1
Target
Algorithm 2
Target
Algorithm 3
Target
Algorithm 1 Algorithm 2 Algorithm 3 Target
Push vs Pull
control
Output
Alphabetizer
Circular
shifter
Input
Line Storage
Decomposition 2
Decomposition 2
Push vs Pull
control
Output
Alphabetizer
Circular
shifter
Input
Line Storage
algorithm
target
Idiomatic Decomposition
• Based on hierarchical abstract interfaces
• Consistent sequence interface
• Pull based
• Efficien...
Idiomatic Decomposition
control
Output
Alphabetizer
Circular
shifter
Input
Idiomatic Decomposition
control
Output
Alphabetizer
Circular
shifter
Input
Idiomatic Decomposition
control
Output
Alphabetizer
Circular
shifter
Input
Idiomatic Decomposition
control
Output
Alphabetizer
Circular
shifter
Input
Range
Interface
sequence<T>
ID - Input
.alphabetized // alphabetizer
.each!writeln; // output
}
// --
private:
/// Performs "foo bar n baz" -> [["foo"...
ID - Circular Shift
{
return range
.lineSplitter
.map!(line => line
.splitter!(chr => chr.isWhite)
.filter!(word => !word....
ID - Circular Shift
/// Performs [["foo", "bar"], ["baz"]] ->
/// [["foo", "bar"], ["bar", "foo"], ["baz"]]
auto withCircu...
/// Performs ["foo", "bar"] -> [["foo", "bar"], ["bar", "foo"]]
auto rotations(Range)(Range range)
{
auto len = range.walk...
/// Performs ["foo", "bar"] -> [["foo", "bar"], ["bar", "foo"]]
auto rotations(Range)(Range range)
{
auto len = range.walk...
/// Performs ["foo", "bar"] -> [["foo", "bar"], ["bar", "foo"]]
auto rotations(Range)(Range range)
{
auto len = range.walk...
/// Performs ["foo", "bar"] -> [["foo", "bar"], ["bar", "foo"]]
auto rotations(Range)(Range range)
{
auto len = range.walk...
/// Performs ["foo", "bar"] -> [["foo", "bar"], ["bar", "foo"]]
auto rotations(Range)(Range range)
{
auto len = range.walk...
/// Performs ["foo", "bar"] -> [["foo", "bar"], ["bar", "foo"]]
auto rotations(Range)(Range range)
{
auto len = range.walk...
/// Performs ["foo", "bar"] -> [["foo", "bar"], ["bar", "foo"]]
auto rotations(Range)(Range range)
{
auto len = range.walk...
ID - Alphabetizing
/// Performs [["foo", "bar"], ["baz"]] -> ["baz", "foo bar"]
auto alphabetized(Range)(Range range)
{
re...
ID - Output
/// Performs [["foo", "bar"], ["baz"]] -> ["f
auto alphabetized(Range)(Range range)
{
return range
.map!(line ...
ID - Control
import std.range;
import std.stdio;
import std.string;
import std.uni;
public:
void run(string inputFile)
{ /...
void run(string inputFile)
{ // Original module:
readText(inputFile) // input
.asWordLists // input
.withCircularShifts //...
ID - Result
A Portrait of The Artist As a Young Man
a Young Man A Portrait of The Artist As
and The Sea The Old Man
Artist...
Conclusion
• The idiomatic D decomposition naturally reflects
the problem statement
• The decomposition that was begging to...
Conclusion
• So, what does Parnas72 mean for D?
• D has strong modelling power. What was otherwise
complex become, simple,...
Upcoming SlideShare
Loading in …5
×

DConf 2016: What Parnas72 Means for D by Luis Marques

763 views

Published on

DConf 2016 talk

Published in: Software
  • Be the first to comment

DConf 2016: What Parnas72 Means for D by Luis Marques

  1. 1. What Parnas72
 Means for D Luís Marques, DConf 2016, May 4th 2016
 luis@luismarques.eu
  2. 2. Reproducing the Classics • My experience growing artificial societies
  3. 3. Growing Artificial Societies • Sugarscape
  4. 4. Growing Artificial Societies • Sugarscape
  5. 5. Growing Artificial Societies • Sugarscape SettingsSettings World speed: Agents: World resources: Minimum vision: Maximum vision: Resource types: Start Sex Death Sickness Sharing WikipedianTrade Metabolism: Grow rate: Spacing: Agent resources:
  6. 6. Growing Artificial Societies • Sugarscape Animation B-3. WealthHistogramEvolutionunderRules({GI), {M, R[60,IOO]}) from aRandomInitial Distributionof Agents 4 39 39 12 15 18 21 24 27 : I ) 7 3 11 22 33 44 55 66 77 88 99 110 0 20 40 60 80 100 120 140 160 la : J 200 1:1)-.122
  7. 7. What Parnas72 Means for D • Who’s Parnas? • What’s Parnas72?
  8. 8. • Electrical engineer • PhD student of Alan Perlis • Applies traditional engineering principles to software design • Critic of SDI • Known for introducing the concept of “Information hiding” David Lorge Parnas
  9. 9. Parnas72 On the Criteria To Be Used in Decomposing Systems into Modules Programming R. Morris Techniques Editor On the Criteria To Be Used in Decomposing Systems into Modules D.L. Parnas Carnegie-Mellon University This paper discusses modularization as a mechanism for improving the flexibility and comprehensibility of a system while allowing the shortening of its development time. The effectiveness of a "modularization" is dependent upon the criteria used in dividing the system into modules. A system design problem is presented and both a conventional and unconventional decomposition are described. It is shown that the unconventional decompositions have distinct advantages for the goals outlined. The criteria used in arriving at the decom- positions are discussed. The unconventional decomposi- tion, if implemented with the conventional assumption that a module consists of one or more subroutines, will be less efficient in most cases. An alternative approach to implementation which does not have this effect is sketched. Key Words and Phrases: software, modules, modularity, software engineering, KWIC index, software design CR Categories: 4.0 Introduction A lucid statement of the philosophy of modular programming can be found in a 1970 textbook on the design of system programs by Gouthier and Pont [1, ¶I0.23], which we quote below: 1 A well-defined segmentation of the project effort ensures system modularity. Each task forms a separate, distinct program module. At implementation time each module and its inputs and outputs are well-defined, there is no confusion in the intended interface with other system modules. At checkout time the in- tegrity of the module is tested independently; there are few sche- duling problems in synchronizing the completion of several tasks before checkout can begin. Finally, the system is maintained in modular fashion; system errors and deficiencies can be traced to specific system modules, thus limiting the scope of detailed error searching. Usually nothing is said about the criteria to be used in dividing the system into modules. This paper will discuss that issue and, by means of examples, suggest some criteria which can be used in decomposing a system into modules. Copyright @ 1972,Association for Computing Machinery, Inc. General permission to republish, but not for profit, all or part of this material is granted, provided that reference is made to this publication, to its date of issue, and to the fact that reprinting privileges were granted by permission of the Association for Com- puting Machinery. Author's address: Department of Computer Science, Carnegie- Mellon University, Pittsburgh, PA 15213. 1053 A Brief Status Report The major advancement in the area of modular programming has been the development of coding techniques and assemblers which (l) allow one module to be written with little knowledge of the code in another module, and (2) allow modules to be reas- sembled and replaced without reassembly of the whole system. This facility is extremely valuable for the production of large pieces of code, but the systems most often used as examples of problem systems are highly- modularized programs and make use of the techniques mentioned above. 1Reprinted by permission of Prentice-Hall, Englewood Cliffs, N.J. Communications December 1972 of Volume 15 the ACM Number 12
  10. 10. Parnas72(b) • Popularizes information-hiding modules • See Parnas72a: “Information Distribution Aspects of Design Methodology" • Topic: • Architecture? Well… • Work assignments! • Documentation! • (Parnas72a)
  11. 11. Parnas72(b) • The documentation story • Philips Computer Industry • A manager asked for help on creating work assignments • How to create specifications so that modules would integrate successfully • Difficult because the modules had to know a lot about each other • How to properly decompose into modules?
  12. 12. Modularization • A module is a work/responsibility assignment • A class, a D module, a component, etc. • “The modularizations include the design decisions which must be made before the work on independent modules can begin” • “Architecture is the set of design decisions that must be made early in a project” (Fowler, “Who Needs an Architect?”, 2003)
  13. 13. Example • KWIC Index program • First, two Parnas modularizations • Then, an idiomatic D modularization
  14. 14. KWIC • “The KWIC index system accepts an ordered set of lines, each line is an ordered set of words, and each word is an ordered set of characters. Any line may be “circularly shifted” by repeatedly removing the first word and appending it at the end of the line. The KWIC index system outputs a listing of all circular shifts of all the lines in alphabetical order”
  15. 15. KWIC KWIC
  16. 16. KWIC Descent of Man The Ascent of Man The Old Man and The Sea A Portrait of The Artist As a Young Man (Key Word In Context) https://users.cs.duke.edu/~ola/ipc/kwic.html
  17. 17. KWIC a portrait of the ARTIST as a young man the ASCENT of man DESCENT of man a portrait of the artist as a young MAN descent of MAN the ascent of MAN the old MAN and the sea the OLD man and the sea a PORTRAIT of the artist as a young man the old man and the SEA a portrait of the artist as a YOUNG man
  18. 18. KWIC a portrait of the ARTIST as a young man the ASCENT of man DESCENT of man a portrait of the artist as a young MAN descent of MAN the ascent of MAN the old MAN and the sea the OLD man and the sea a PORTRAIT of the artist as a young man the old man and the SEA a portrait of the artist as a YOUNG man
  19. 19. KWIC a portrait of the ARTIST as a young man the ASCENT of man DESCENT of man a portrait of the artist as a young MAN descent of MAN the ascent of MAN the old MAN and the sea the OLD man and the sea a PORTRAIT of the artist as a young man the old man and the SEA a portrait of the artist as a YOUNG man
  20. 20. KWIC A Portrait of The Artist As a Young Man The Ascent of Man Descent of Man A Portrait of The Artist As a Young Man Descent of Man The Ascent of Man The Old Man and The Sea The Old Man and The Sea A Portrait of The Artist As a Young Man The Old Man and The Sea A Portrait of The Artist As a Young Man
  21. 21. KWIC Artist As a Young Man, A Portrait of The Ascent of Man, The Descent of Man Man, A Portrait of The Artist As a Young Man, Descent of Man, The Ascent of Man and The Sea, The Old Old Man and The Sea, The Portrait of The Artist As a Young Man, A Sea, The Old Man and The Young Man, A Portrait of The Artist As a
  22. 22. KWIC Artist As a Young Man, A Portrait of The Ascent of Man, The Descent of Man Man, A Portrait of The Artist As a Young Man, Descent of Man, The Ascent of Man and The Sea, The Old Old Man and The Sea, The Portrait of The Artist As a Young Man, A Sea, The Old Man and The Young Man, A Portrait of The Artist As a
  23. 23. KWIC Artist As a Young Man A Portrait of The Ascent of Man The Descent of Man Man A Portrait of The Artist As a Young Man Descent of Man The Ascent of Man and The Sea The Old Old Man and The Sea The Portrait of The Artist As a Young Man A Sea The Old Man and The Young Man A Portrait of The Artist As a
  24. 24. KWIC Artist As a Young Man A Portrait of The Ascent of Man The Descent of Man Man A Portrait of The Artist As a Young Man and The Sea The Old Man Descent of Man The Ascent of Old Man and The Sea The Portrait of The Artist As a Young Man A Sea The Old Man and The Young Man A Portrait of The Artist As a
  25. 25. KWIC A Portrait of The Artist As a Young Man a Young Man A Portrait of The Artist As and The Sea The Old Man Artist As a Young Man A Portrait of The As a Young Man A Portrait of The Artist Ascent of Man The Descent of Man Man A Portrait of The Artist As a Young Man and The Sea The Old Man Descent of Man The Ascent of of Man Descent of Man The Ascent of The Artist As a Young Man A Portrait Old Man and The Sea The Portrait of The Artist As a Young Man A Sea The Old Man and The The Artist As a Young Man A Portrait of The Ascent of Man The Old Man and The Sea The Sea The Old Man and Young Man A Portrait of The Artist As a
  26. 26. KWIC A Portrait of The Artist As a Young Man a Young Man A Portrait of The Artist As and The Sea The Old Man Artist As a Young Man A Portrait of The As a Young Man A Portrait of The Artist Ascent of Man The Descent of Man Man A Portrait of The Artist As a Young Man and The Sea The Old Man Descent of Man The Ascent of of Man Descent of Man The Ascent of The Artist As a Young Man A Portrait Old Man and The Sea The Portrait of The Artist As a Young Man A Sea The Old Man and The The Artist As a Young Man A Portrait of The Ascent of Man The Old Man and The Sea The Sea The Old Man and Young Man A Portrait of The Artist As a
  27. 27. KWIC • Input: a sequence of lines • Line: a sequence of words • Word: a sequence of characters • Circular shift: • foo bar baz → baz foo bar • Output: all circular shifts of all lines, in alphabetical order
  28. 28. A Tale of 2 Decompositions • Parnas’ two decompositions reimplemented in D • <https://github.com/luismarques/parnas72>
  29. 29. Decomposition 1 (D1) • Idea: • The flowchart method • The classic method • Data-oriented design • Module == collection of procedures
  30. 30. Decomposition 1 (D1) canonical representation of input all circular shifts of input lines
 (no order requirements) all circular shifts of input lines
 in alphabetical order pretty printed index control Output Alphabetizer Circular shifter Input input data
  31. 31. D1 - Input • Module 1: Input. This module reads the data lines from the input medium and stores them in core for processing by the remaining modules. The characters are packed four to a word, and an otherwise unused character is used to indicate the end of a word. An index is kept to show the starting address of each line.
  32. 32. D1 - Input Foo import std.utf; public: // change these definitions, for different performance alias LineNum = ptrdiff_t; alias WordNum = ptrdiff_t; alias CharNum = ptrdiff_t; enum wordSeparator = ' '; string data; CharNum[] lineIndex; // Reads the lines from the input medium, and stores e // `wordSeparator` character constant. The Lines are s // and a separate index of where the lines start is ke void readLines(string filename) { F o o ␣ D 0 0 0 F o o ␣ D L u í s { 1 char / 1 dchar { 2 char / 1 dchar •••• •••• • •••• ••••
  33. 33. D1 - Input • Input: Descent␣of␣Man↵The⇥Ascent⇥␣of␣Man␣␣↵The␣ Old␣Man␣and␣The␣Sea↵␣␣↵A␣Portrait␣of␣The␣Ar tist␣As␣a␣Young␣Man • Output: Descent␣of␣ManThe␣Ascent␣of␣ManThe␣Old␣Man␣ and␣The␣SeaA␣Portrait␣of␣The␣Artist␣As␣a␣You ng␣Man
 [0, 14, 31, 54]
  34. 34. // change these definitions, for different performance/capability trade-offs: alias LineNum = ptrdiff_t; alias WordNum = ptrdiff_t; alias CharNum = ptrdiff_t; enum wordSeparator = ' '; string data; CharNum[] lineIndex; // Reads the lines from the input medium, and stores each word separated by a // `wordSeparator` character constant. The Lines are stored without a separator, // and a separate index of where the lines start is kept in `lineIndex`. void readLines(string filename) { size_t lineStart; data = readText(filename) .lineSplitter // normalize the spacing between words .map!(line => line .splitter!isWhite .filter!(c => !c.empty) .joiner([wordSeparator])) // remove empty lines .filter!(line => !line.empty) // keep an index of where each line starts .tee!((line) { lineIndex ~= lineStart; lineStart += line.byChar.walkLength; }) // join the lines .joiner .to!string; } D1 - Input
  35. 35. // Reads the lines from the input medium, and stores each word separated by a // `wordSeparator` character constant. The Lines are stored without a separator, // and a separate index of where the lines start is kept in `lineIndex`. void readLines(string filename) { size_t lineStart; data = readText(filename) .lineSplitter // normalize the spacing between words .map!(line => line .splitter!isWhite .filter!(c => !c.empty) .joiner([wordSeparator])) // remove empty lines .filter!(line => !line.empty) // keep an index of where each line starts .tee!((line) { lineIndex ~= lineStart; lineStart += line.byChar.walkLength; }) // join the lines .joiner .to!string; }
  36. 36. data = readText(filename) .lineSplitter // normalize the spacing between words .map!(line => line .splitter!isWhite .filter!(c => !c.empty) .joiner([wordSeparator])) // remove empty lines .filter!(line => !line.empty) // keep an index of where each line starts .tee!((line) { lineIndex ~= lineStart; lineStart += line.byChar.walkLength; }) Descent␣of␣Man↵The⇥Ascent⇥␣of␣Man␣␣↵The␣Old␣M an␣and␣The␣Sea↵␣␣↵A␣Portrait␣of␣The␣Artist␣As␣a␣ Young␣Man 
 [Descent␣of␣Man][The⇥Ascent⇥␣of␣Man␣␣] [The␣Old␣Man␣and␣The␣Sea][␣␣] [A␣Portrait␣of␣The␣Artist␣As␣a␣Young␣Man]
  37. 37. // normalize the spacing between words .map!(line => line .splitter!isWhite .filter!(c => !c.empty) .joiner([wordSeparator])) // remove empty lines .filter!(line => !line.empty) // keep an index of where each line starts .tee!((line) { lineIndex ~= lineStart; lineStart += line.byChar.walkLength; }) // join the lines .joiner .to!string; [Descent␣of␣Man][The⇥Ascent⇥␣of␣Man␣␣] [The␣Old␣Man␣and␣The␣Sea][␣␣] [A␣Portrait␣of␣The␣Artist␣As␣a␣Young␣Man] [[Descent][of][Man]][[The][Ascent][][of] [Man][][]][[The][Old][Man][and][The][Sea]] [[][][]][[A][Portrait][of][The][Artist] [As][a][Young][Man]]
  38. 38. // normalize the spacing between words .map!(line => line .splitter!isWhite .filter!(c => !c.empty) .joiner([wordSeparator])) // remove empty lines .filter!(line => !line.empty) // keep an index of where each line starts .tee!((line) { lineIndex ~= lineStart; lineStart += line.byChar.walkLength; }) // join the lines .joiner .to!string; [[Descent][of][Man]][[The][Ascent][][of] [Man][][]][[The][Old][Man][and][The][Sea]] [[][][]][[A][Portrait][of][The][Artist] [As][a][Young][Man]] [[Descent][of][Man]][[The][Ascent][of] [Man]][[The][Old][Man][and][The][Sea]][] [[A][Portrait][of][The][Artist][As][a] [Young][Man]]
  39. 39. // normalize the spacing between words .map!(line => line .splitter!isWhite .filter!(c => !c.empty) .joiner([wordSeparator])) // remove empty lines .filter!(line => !line.empty) // keep an index of where each line starts .tee!((line) { lineIndex ~= lineStart; lineStart += line.byChar.walkLength; }) // join the lines .joiner .to!string; [[Descent][of][Man]][[The][Ascent][of] [Man]][[The][Old][Man][and][The][Sea]][] [[A][Portrait][of][The][Artist][As][a] [Young][Man]] [Descent␣of␣Man][The␣Ascent␣of␣Man] [The␣Old␣Man␣and␣The␣Sea][] [A␣Portrait␣of␣The␣Artist␣As␣a␣Young␣Man]
  40. 40. // remove empty lines .filter!(line => !line.empty) // keep an index of where each line starts .tee!((line) { lineIndex ~= lineStart; lineStart += line.byChar.walkLength; }) // join the lines .joiner .to!string; } [Descent␣of␣Man][The␣Ascent␣of␣Man] [The␣Old␣Man␣and␣The␣Sea][] [A␣Portrait␣of␣The␣Artist␣As␣a␣Young␣Man] 
 [Descent␣of␣Man][The␣Ascent␣of␣Man] [The␣Old␣Man␣and␣The␣Sea] [A␣Portrait␣of␣The␣Artist␣As␣a␣Young␣Man]
  41. 41. // remove empty lines .filter!(line => !line.empty) // keep an index of where each line starts .tee!((line) { lineIndex ~= lineStart; lineStart += line.byChar.walkLength; }) // join the lines .joiner .to!string; } Descent␣of␣ManThe␣Ascent␣of␣ManThe␣Old␣Man␣ and␣The␣SeaA␣Portrait␣of␣The␣Artist␣As␣a␣You ng␣Man
  42. 42. D1 - Circular Shift • Module 2: Circular Shift. This module is called after the input module has completed its work. It prepares an index which gives the address of the first character of each circular shift, and the original index of the line in the array made up by module 1. It leaves its output in core with words in pairs (original line number, starting address).
  43. 43. D1 - Circular Shift import std.algorithm; import std.range; import std.typecons; import std.utf; import one.input; public: struct ShiftIndexEntry { LineNum lineNum; CharNum firstChar; } ShiftIndexEntry[] shiftIndex; void setup() { shiftIndex = iota(lineIndex.length) // 0
  44. 44. D1 - Circular Shift Circular shifter Input .map!(a => a.value.byCodeUnit .enumerate // (0, ((0, char1), (1, char2 .splitter!(b => b.value == wordSeparator) .map!(b => b.front.index + lineIndex[a.index] .enumerate .map!(a => a.value .map!(b => ShiftIndexEntry(a.index, b))) .joiner .array; } private: auto line(LineNum lineNum) { auto lineStart = lineIndex[lineNum]; auto lineEnd = lineNum+1 >= lineIndex.length ? data.length : lineIndex[lineNum+1]; return data[lineStart .. lineEnd]; }
  45. 45. D1 - Circular Shift { LineNum lineNum; CharNum firstChar; } ShiftIndexEntry[] shiftIndex; void setup() { shiftIndex = iota(lineIndex.length) .map!(lineNum => line(lineNum)) .enumerate .map!(a => a.value.byCodeUnit .enumerate .splitter!(b => b.value == wordSeparator) .map!(b => b.front.index + lineIndex[a.index])) .enumerate .map!(a => a.value .map!(b => ShiftIndexEntry(a.index, b))) .joiner .array; } private:
  46. 46. shiftIndex = iota(lineIndex.length) .map!(lineNum => line(lineNum)) .enumerate .map!(a => a.value.byCodeUnit .enumerate .splitter!(b => b.value == wordSeparator) .map!(b => b.front.index + lineIndex[a.index])) .enumerate .map!(a => a.value .map!(b => ShiftIndexEntry(a.index, b))) .joiner .array; vate: o line(LineNum lineNum)
  47. 47. shiftIndex = iota(lineIndex.length) .map!(lineNum => line(lineNum)) .enumerate .map!(a => a.value.byCodeUnit .enumerate .splitter!(b => b.value == wordSeparator) .map!(b => b.front.index + lineIndex[a.index])) .enumerate .map!(a => a.value .map!(b => ShiftIndexEntry(a.index, b))) .joiner .array; vate: o line(LineNum lineNum) 0, 1, … [Descent␣of␣Man]
 [The␣Ascent␣of␣Man] …
  48. 48. shiftIndex = iota(lineIndex.length) .map!(lineNum => line(lineNum)) .enumerate .map!(a => a.value.byCodeUnit .enumerate .splitter!(b => b.value == wordSeparator) .map!(b => b.front.index + lineIndex[a.index])) .enumerate .map!(a => a.value .map!(b => ShiftIndexEntry(a.index, b))) .joiner .array; vate: o line(LineNum lineNum) (0, [Descent␣of␣Man])
 (1, [The␣Ascent␣of␣Man]) [Descent␣of␣Man]
 [The␣Ascent␣of␣Man]
  49. 49. shiftIndex = iota(lineIndex.length) .map!(lineNum => line(lineNum)) .enumerate .map!(a => a.value.byCodeUnit .enumerate .splitter!(b => b.value == wordSeparator) .map!(b => b.front.index + lineIndex[a.index])) .enumerate .map!(a => a.value .map!(b => ShiftIndexEntry(a.index, b))) .joiner .array; vate: o line(LineNum lineNum) [0, 8, 11]
 [14, 18, 25, 28] (0, [Descent␣of␣Man])
 (1, [The␣Ascent␣of␣Man])
  50. 50. shiftIndex = iota(lineIndex.length) .map!(lineNum => line(lineNum)) .enumerate .map!(a => a.value.byCodeUnit .enumerate .splitter!(b => b.value == wordSeparator) .map!(b => b.front.index + lineIndex[a.index])) .enumerate .map!(a => a.value .map!(b => ShiftIndexEntry(a.index, b))) .joiner .array; vate: o line(LineNum lineNum) (0, [Descent␣of␣Man])
 (1, [The␣Ascent␣of␣Man]) 0 1 2 3 4 5 6 7 8 9 10 11 12 13
  51. 51. (0, [[Descent][of] (1, [[The][Ascent] shiftIndex = iota(lineIndex.length) .map!(lineNum => line(lineNum)) .enumerate .map!(a => a.value.byCodeUnit .enumerate .splitter!(b => b.value == wordSeparator) .map!(b => b.front.index + lineIndex[a.index])) .enumerate .map!(a => a.value .map!(b => ShiftIndexEntry(a.index, b))) .joiner .array; vate: o line(LineNum lineNum) 0 1 2 3 4 5 6 8 9 (0, [Descent␣of␣Man])
 (1, [The␣Ascent␣of␣Man]) 0 1 2 3 4 5 6 7 8 9 10 11 12 13
  52. 52. (0, [[Descent][of][Man]])
 (1, [[The][Ascent][of][Man]]) shiftIndex = iota(lineIndex.length) .map!(lineNum => line(lineNum)) .enumerate .map!(a => a.value.byCodeUnit .enumerate .splitter!(b => b.value == wordSeparator) .map!(b => b.front.index + lineIndex[a.index])) .enumerate .map!(a => a.value .map!(b => ShiftIndexEntry(a.index, b))) .joiner .array; vate: o line(LineNum lineNum) 0 1 2 3 4 5 6 8 9 11 13 [0, 8, 11]
 [14, 18, 25, 28]
  53. 53. (0, [0, 8, 11]) (1, [14, 18, 25, 28]) [0, 8, 11]
 [14, 18, 25, 28] shiftIndex = iota(lineIndex.length) .map!(lineNum => line(lineNum)) .enumerate .map!(a => a.value.byCodeUnit .enumerate .splitter!(b => b.value == wordSeparator) .map!(b => b.front.index + lineIndex[a.index])) .enumerate .map!(a => a.value .map!(b => ShiftIndexEntry(a.index, b))) .joiner .array; vate: o line(LineNum lineNum)
  54. 54. shiftIndex = iota(lineIndex.length) .map!(lineNum => line(lineNum)) .enumerate .map!(a => a.value.byCodeUnit .enumerate .splitter!(b => b.value == wordSeparator) .map!(b => b.front.index + lineIndex[a.index])) .enumerate .map!(a => a.value .map!(b => ShiftIndexEntry(a.index, b))) .joiner .array; vate: o line(LineNum lineNum) (0, [0, 8, 11]) (1, [14, 18, 25, 28]) [ ShiftIndexEntry(0, 0),
 ShiftIndexEntry(0, 8), 
 ShiftIndexEntry(0, 11) ] [ ShiftIndexEntry(1, 14), ShiftIndexEntry(1, 18), ShiftIndexEntry(1, 25), ShiftIndexEntry(1, 28) ]
  55. 55. shiftIndex = iota(lineIndex.length) .map!(lineNum => line(lineNum)) .enumerate .map!(a => a.value.byCodeUnit .enumerate .splitter!(b => b.value == wordSeparator) .map!(b => b.front.index + lineIndex[a.index])) .enumerate .map!(a => a.value .map!(b => ShiftIndexEntry(a.index, b))) .joiner .array; vate: o line(LineNum lineNum) ShiftIndexEntry(0, 0)
 ShiftIndexEntry(0, 8) 
 ShiftIndexEntry(0, 11) ShiftIndexEntry(1, 14) ShiftIndexEntry(1, 18) ShiftIndexEntry(1, 25) ShiftIndexEntry(1, 28)
  56. 56. D1 - Alphabetizing • Module 3: Alphabetizing. This module takes as input the arrays produced by modules 1 and 2. It produces an array in the same format as that produced by module 2. In this case, however, the circular shifts are listed in another order (alphabetically).
  57. 57. D1 - Alphabetizing import std.typecons; import std.utf; import one.input; public: struct ShiftIndexEntry { LineNum lineNum; CharNum firstChar; } ShiftIndexEntry[] shiftIndex; void setup() { shiftIndex = iota(lineIndex.length) // 0, 1, 2 ... .map!(lineNum => line(lineNum)) // first line, se .enumerate // (0, first line), (1, secon .map!(a => a.value.byCodeUnit [The␣Ascent␣of␣Man] [Ascent␣of␣Man␣The] [Ascent␣of␣Man][The] [The␣Ascent␣of␣Man] [Ascent␣of␣Man␣The]
  58. 58. [Ascent␣of␣Man][The] [The␣Ascent␣of␣Man] [Ascent␣of␣Man␣The] private: auto line(ShiftIndexEntry entry) { auto a = entry.firstChar; auto b = entry.lineNum+1 >= lineIndex.length ? data.length : lineIndex[entry.lineNum+1]; auto c = lineIndex[entry.lineNum]; auto d = (entry.firstChar-1).max(0).max(c); auto x = data[a .. b]; auto y = data[c .. d]; return joiner(only(x, y).filter!(a => !a.empty), " "); } c d a b y x
  59. 59. D1 - Alphabetizing module one.alphabetizer; import std.algorithm; import std.math; import std.range; import std.typecons; import std.uni; import std.utf; import one.input; import one.circularshifter; public: void setup() { shiftIndex.sort!((a, b) => icmp(line(a), line(b)) < 0); } private: auto line(ShiftIndexEntry entry) { auto a = entry.firstChar; auto b = entry.lineNum+1 >= lineIndex.length ? data.length : lineIndex[entry.lineNum+1];
  60. 60. D1 - Output • Module 4: Output. Using the arrays produced by module 3 and module 1, this modules produces a nicely formatted output listing all of the circular shifts. In a sophisticated system the actual start of each line will be marked, pointers to further information may be inserted, and the start of the circular shift may actually not be the first word in the line, etc.
  61. 61. D1 - Output void printLines() { shiftIndex.map!(entry => entry.line).each!writeln; } private: auto line(ShiftIndexEntry entry) { auto a = entry.firstChar; auto b = entry.lineNum+1 >= lineIndex.length ? data.length : lineIndex[entry.lineNum+1]; auto c = lineIndex[entry.lineNum]; auto d = (entry.firstChar-1).max(0).max(c); auto x = data[a .. b]; auto y = data[c .. d]; return joiner(only(x, y).filter!(a => !a.empty), " "); }
  62. 62. D1 - Outputmodule one.output; import std.algorithm; import std.range; import std.stdio; import one.input; import one.circularshifter; void printLines() { shiftIndex.map!(entry => entry.line).each!writeln; } private: auto line(ShiftIndexEntry entry) { auto a = entry.firstChar; auto b = entry.lineNum+1 >= lineIndex.length ? data.length : lineIndex[entry.lineNum+1];
  63. 63. D1 - Master Control • Module 5: Master Control. This module does little more than control the sequencing among the other four modules. It may also handle error messages, space allocation, etc.
  64. 64. D1 - Master Control module one.control; import one.input; import one.circularshifter; import one.alphabetizer; import one.output; public: void run(string inputFile) { readLines(inputFile); one.circularshifter.setup(); one.alphabetizer.setup(); printLines(); }
  65. 65. D1 - Result A Portrait of The Artist As a Young Man a Young Man A Portrait of The Artist As and The Sea The Old Man Artist As a Young Man A Portrait of The As a Young Man A Portrait of The Artist Ascent of Man The Descent of Man Man A Portrait of The Artist As a Young Man and The Sea The Old Man Descent of Man The Ascent of of Man Descent of Man The Ascent of The Artist As a Young Man A Portrait Old Man and The Sea The Portrait of The Artist As a Young Man A Sea The Old Man and The The Artist As a Young Man A Portrait of The Ascent of Man The Old Man and The Sea The Sea The Old Man and Young Man A Portrait of The Artist As a
  66. 66. Decomposition 2 (D2) • Based on: • Difficult design decisions, or design decisions which are likely to change (the secret) • Modules are designed to hide these decisions from the others • Abstract interface • Efficient work partition
  67. 67. Decomposition 2 (D2) control Output Alphabetizer Circular shifter Input Line Storage
  68. 68. D2 - Line Storage • Module 1: Line Storage. This module consists of a number of functions(…). • WORD • SETWRD • WORDS • LINES • DELWRD • DELLINE • CHARS
  69. 69. D2 - Line Storage Definition of a "Line Storage" Module Introduction: This definition specifies a mechanism which hold up to pi lines, each line consisting of up to p2 word may be up to p3 characters. Function WORD possible values: integers initial values: undefined parameters: l»w,c all effect: j . call call call call call call ERLWEL ERLWNL ERLWEW ERLWNW ERLWEC ERLWNC integer if if if if if if 1 < 1 > w < w > c < c > 1 or 1 > LINES 1 or w > WORDS(1) 1 or c > CHARS(l,w) P2 P3 Function SETWRD possible values: initial values: none not applicable
  70. 70. D2 - Line Storage call call call call call call ERLWEL ERLWNL ERLWEW ERLWNW ERLWEC ERLWNC if if if if if if 1 < 1 > w < w > c < c > 1 or 1 > LINES 1 or w > WORDS(1) 1 or c > CHARS(l,w) P2 P3 Function SETWRD possible values: initial values: parameters: effect: none not applicable l,w,c,d all integers call call call call call call call call if 1 if w ERLSLE ERLSBL ERLSBL ERLSWE ERLSBW ERLSBW ERLSCE ERLSBC = 'LINES' 1 or 1 > pl 'LINES' +1 'LINES' 1 or w > p2 'WORDS'(1) + 1 'WORDS'(1) 1 or c > p3 if c .noteq. 'CHARS'(l,w)+l +1 then LINES = 'LINES* + 1 if 1 < if 1 > if 1 < if w < if w > if w < if c < = 'WORDS*(1) +1 then WORDS(1) CHARS(l,w) = c WORDQ,w,c) = d Function WORDS
  71. 71. D2 - Line Storage Definition of a "Line Storage" Module Introduction: This definition specifies a mechanism which hold up to pi lines, each line consisting of up to p2 word may be up to p3 characters. Function WORD possible values: integers initial values: undefined parameters: l»w,c all effect: j . call call call call call call ERLWEL ERLWNL ERLWEW ERLWNW ERLWEC ERLWNC integer if if if if if if 1 < 1 > w < w > c < c > 1 or 1 > LINES 1 or w > WORDS(1) 1 or c > CHARS(l,w) P2 P3 Function SETWRD possible values: initial values: none not applicable CHAR r
  72. 72. D2 - Line Storage • The function call CHAR(r,w,c) will have as value an integer representing the cth character in the rth line, wth word • A call such as SETCHAR(r,w,c,d) will cause the cth character in the wth word of the rth line to be the character represented by d (i.e., CHAR(r,w,c) = d) • WORDS(r) returns the number of words in line r • Etc.
  73. 73. D2 - Line Storage • Functions • CHAR • SETCHAR • WORDS • LINES • DELWRD • DELLINE • CHARS
  74. 74. D2 - Line Storage module two.linestorage; import std.algorithm; import std.range; import std.utf; public: // change these definitions, for different performance/capabili alias LineNum = ptrdiff_t; alias WordNum = ptrdiff_t; alias CharNum = ptrdiff_t; enum maxLines = LineNum.max; /// (original name: p1) enum maxWordsPerLine = WordNum.max; /// (original name: p2) enum maxCharsPerWord = CharNum.max; /// (original name: p3) /// Returns one character from a given word, present at a given /// (original name: WORD) char wordChar(LineNum lineNum, WordNum wordNum, CharNum charNum
  75. 75. D2 - Line Storage void removeLine(LineNum lineNum) { assert(lineNum >= 0 && lineNum < numLine assert(false, "not implemented"); } // -- private: enum wordSeparator = ' '; char[] data; CharNum[] lineIndex; auto line(LineNum lineNum) { auto lineStart = lineIndex[lineNum]; auto lineEnd = lineNum+1 >= lineIndex.le private:
  76. 76. D2 - Line Storage // -- private: enum wordSeparator = ' '; char[] data; CharNum[] lineIndex; auto line(LineNum lineNum) { auto lineStart = lineIndex[lineNum]; auto lineEnd = lineNum+1 >= lineIndex.length ? data.length : lineIndex[lineNum+1]; return data[lineStart .. lineEnd].byCodeUnit; } /// Returns a range of words for a given line auto wordsForLine(LineNum lineNum) { return line(lineNum).splitter(wordSeparator); } private:
  77. 77. D2 - Line Storage alias CharNum = ptrdiff_t; enum maxLines = LineNum.max; /// (original name: p1) enum maxWordsPerLine = WordNum.max; /// (original name: p2) enum maxCharsPerWord = CharNum.max; /// (original name: p3) /// Returns one character from a given word, from line `lineNum`. /// (original name: WORD) char wordChar(LineNum lineNum, WordNum wordNum, CharNum charNum) { assert(lineNum >= 0 && lineNum < maxLines); assert(lineNum < numLines); assert(wordNum >= 0 && wordNum < maxWordsPerLine); assert(wordNum < numWords(lineNum)); assert(charNum >= 0 && charNum < maxCharsPerWord); assert(charNum < numCharacters(lineNum, wordNum)); return wordsForLine(lineNum) .dropExactly(wordNum) .front .dropExactly(charNum) .front; }
  78. 78. /// Returns one character from a given word, from line `lineNum`. /// (original name: WORD) char wordChar(LineNum lineNum, WordNum wordNum, CharNum charNum) { assert(lineNum >= 0 && lineNum < maxLines); assert(lineNum < numLines); assert(wordNum >= 0 && wordNum < maxWordsPerLine); assert(wordNum < numWords(lineNum)); assert(charNum >= 0 && charNum < maxCharsPerWord); assert(charNum < numCharacters(lineNum, wordNum)); return wordsForLine(lineNum) .dropExactly(wordNum) .front .dropExactly(charNum) .front; } /// Sets the next character for the current or the next word. /// The current word is the last word present at the last line. /// (original name: SETWRD) void setWordChar(LineNum lineNum, WordNum wordNum, CharNum charNum, ch { assert(lineNum >= 0 && lineNum < maxLines);
  79. 79. /// Returns one character from a given word, from line `lineNum`. /// (original name: WORD) char wordChar(LineNum lineNum, WordNum wordNum, CharNum charNum) { assert(lineNum >= 0 && lineNum < maxLines); assert(lineNum < numLines); assert(wordNum >= 0 && wordNum < maxWordsPerLine); assert(wordNum < numWords(lineNum)); assert(charNum >= 0 && charNum < maxCharsPerWord); assert(charNum < numCharacters(lineNum, wordNum)); return wordsForLine(lineNum) .dropExactly(wordNum) .front .dropExactly(charNum) .front; } /// Sets the next character for the current or the next word. /// The current word is the last word present at the last line. /// (original name: SETWRD) void setWordChar(LineNum lineNum, WordNum wordNum, CharNum charNum, ch { assert(lineNum >= 0 && lineNum < maxLines); f o o a rb
  80. 80. /// Returns one character from a given word, from line `lineNum`. /// (original name: WORD) char wordChar(LineNum lineNum, WordNum wordNum, CharNum charNum) { assert(lineNum >= 0 && lineNum < maxLines); assert(lineNum < numLines); assert(wordNum >= 0 && wordNum < maxWordsPerLine); assert(wordNum < numWords(lineNum)); assert(charNum >= 0 && charNum < maxCharsPerWord); assert(charNum < numCharacters(lineNum, wordNum)); return wordsForLine(lineNum) .dropExactly(wordNum) .front .dropExactly(charNum) .front; } /// Sets the next character for the current or the next word. /// The current word is the last word present at the last line. /// (original name: SETWRD) void setWordChar(LineNum lineNum, WordNum wordNum, CharNum charNum, ch { assert(lineNum >= 0 && lineNum < maxLines); f o o a rb
  81. 81. /// Returns one character from a given word, from line `lineNum`. /// (original name: WORD) char wordChar(LineNum lineNum, WordNum wordNum, CharNum charNum) { assert(lineNum >= 0 && lineNum < maxLines); assert(lineNum < numLines); assert(wordNum >= 0 && wordNum < maxWordsPerLine); assert(wordNum < numWords(lineNum)); assert(charNum >= 0 && charNum < maxCharsPerWord); assert(charNum < numCharacters(lineNum, wordNum)); return wordsForLine(lineNum) .dropExactly(wordNum) .front .dropExactly(charNum) .front; } /// Sets the next character for the current or the next word. /// The current word is the last word present at the last line. /// (original name: SETWRD) void setWordChar(LineNum lineNum, WordNum wordNum, CharNum charNum, ch { assert(lineNum >= 0 && lineNum < maxLines); a rb
  82. 82. /// Returns one character from a given word, from line `lineNum`. /// (original name: WORD) char wordChar(LineNum lineNum, WordNum wordNum, CharNum charNum) { assert(lineNum >= 0 && lineNum < maxLines); assert(lineNum < numLines); assert(wordNum >= 0 && wordNum < maxWordsPerLine); assert(wordNum < numWords(lineNum)); assert(charNum >= 0 && charNum < maxCharsPerWord); assert(charNum < numCharacters(lineNum, wordNum)); return wordsForLine(lineNum) .dropExactly(wordNum) .front .dropExactly(charNum) .front; } /// Sets the next character for the current or the next word. /// The current word is the last word present at the last line. /// (original name: SETWRD) void setWordChar(LineNum lineNum, WordNum wordNum, CharNum charNum, ch { assert(lineNum >= 0 && lineNum < maxLines); a rb
  83. 83. /// Returns one character from a given word, from line `lineNum`. /// (original name: WORD) char wordChar(LineNum lineNum, WordNum wordNum, CharNum charNum) { assert(lineNum >= 0 && lineNum < maxLines); assert(lineNum < numLines); assert(wordNum >= 0 && wordNum < maxWordsPerLine); assert(wordNum < numWords(lineNum)); assert(charNum >= 0 && charNum < maxCharsPerWord); assert(charNum < numCharacters(lineNum, wordNum)); return wordsForLine(lineNum) .dropExactly(wordNum) .front .dropExactly(charNum) .front; } /// Sets the next character for the current or the next word. /// The current word is the last word present at the last line. /// (original name: SETWRD) void setWordChar(LineNum lineNum, WordNum wordNum, CharNum charNum, ch { assert(lineNum >= 0 && lineNum < maxLines); a r
  84. 84. /// Sets the next character for the current or the next word. /// The current word is the last word present at the last line. /// (original name: SETWRD) void setWordChar(LineNum lineNum, WordNum wordNum, CharNum charNum, char charValue) { assert(lineNum >= 0 && lineNum < maxLines); assert(lineNum == numLines-1 || lineNum == numLines); assert(wordNum >= 0 && wordNum < maxWordsPerLine); assert(charNum >= 0 && charNum < maxCharsPerWord); if(lineNum < numLines) { auto nw = numWords(lineNum); assert(wordNum == nw-1 || wordNum == nw); if(wordNum < nw) { assert(charNum == numCharacters(lineNum, wordNum)); } } if(lineNum == numLines) { lineIndex ~= data.length; } else { if(wordNum == numWords(lineNum)) { data ~= wordSeparator; } } data ~= charValue; }
  85. 85. D2 - Line Storage /// Sets the next character for the current or the next word. /// The current word is the last word present at the last line. /// (original name: SETWRD) void setWordChar(LineNum lineNum, WordNum wordNum, CharNum charNum, char charValue) { if(lineNum == numLines) { lineIndex ~= data.length; } else { if(wordNum == numWords(lineNum)) { data ~= wordSeparator; } } data ~= charValue; }
  86. 86. D2 - Input • Module 2: Input. This module reads the original lines from the input media and calls the line storage module to have them stored internally.
  87. 87. D2 - Inputmodule two.input; import std.stdio; import std.uni; import two.linestorage; public: void readLines(string inputFile) { foreach(lineNum, line; File(inputFile).byLine) { foreach(charNum, c; line) { setWordChar(lineNum, wordNum, charNum, c); } } }
  88. 88. void readLines(string inputFile) { LineNum lineNum; foreach(line; File(inputFile).byLine) { WordNum wordNum; CharNum charNum; bool incWordNum; bool incLineNum; foreach(c; line) { if(c.isWhite) incWordNum = true; else { if(incWordNum) { wordNum++; charNum = 0; incWordNum = false; } setWordChar(lineNum, wordNum, charNum, c); charNum++; incLineNum = true; } } if(incLineNum) { lineNum++; incLineNum = false; } } }
  89. 89. D2 - Circular Shifter • Module 3: Circular Shifter. The principal functions provided by this module are analogs of functions provided in module 1. (…) • CHAR → CSCHAR • WORDS → CSWORDS • LINES → CSLINES • CHARS → CSCHARS • … • A function CSSETUP is provided which must be called before the other functions have their value specified.
  90. 90. D2 - Circular Shifter private: WordNum storageWordNum = (entry.firstWor storage.numWords(entry.lineNum); return storage.numCharacters(entry.lineN } // -- private: struct ShiftIndexEntry { LineNum lineNum; WordNum firstWord; } ShiftIndexEntry[] shiftIndex;
  91. 91. D2 - Circular Shifter public: enum maxLines = LineNum.max; /// (original name: p4) alias LineNum = storage.LineNum; alias WordNum = storage.WordNum; alias CharNum = storage.CharNum; /// (original name: CSSTUP) void setup() { shiftIndex = iota(storage.numLines) .map!(a => storage.numWords(a).iota .map!(b => ShiftIndexEntry(a, b))) .joiner .array; } /// (original name: CSWORD) char wordChar(LineNum lineNum, WordNum wordNum, CharNu {
  92. 92. /// (original name: CSSTUP) void setup() { shiftIndex = iota(storage.numLines) .map!(a => storage.numWords(a).iota .map!(b => ShiftIndexEntry(a, b))) .joiner .array; } /// (original name: CSWORD) char wordChar(LineNum lineNum, WordNum wordNum, CharNu { auto entry = shiftIndex[lineNum]; WordNum storageWordNum = (entry.firstWord + wordNu storage.numWords(entry.lineNum); return storage.wordChar(entry.lineNum, storageWord } /// (original name: CSWRDS)
  93. 93. /// (original name: CSSTUP) void setup() { shiftIndex = iota(storage.numLines) .map!(a => storage.numWords(a).iota .map!(b => ShiftIndexEntry(a, b))) .joiner .array; } /// (original name: CSWORD) char wordChar(LineNum lineNum, WordNum wordNum, CharNu { auto entry = shiftIndex[lineNum]; WordNum storageWordNum = (entry.firstWord + wordNu storage.numWords(entry.lineNum); return storage.wordChar(entry.lineNum, storageWord } /// (original name: CSWRDS) 0, 1, …
  94. 94. /// (original name: CSSTUP) void setup() { shiftIndex = iota(storage.numLines) .map!(a => storage.numWords(a).iota .map!(b => ShiftIndexEntry(a, b))) .joiner .array; } /// (original name: CSWORD) char wordChar(LineNum lineNum, WordNum wordNum, CharNu { auto entry = shiftIndex[lineNum]; WordNum storageWordNum = (entry.firstWord + wordNu storage.numWords(entry.lineNum); return storage.wordChar(entry.lineNum, storageWord } /// (original name: CSWRDS) 0, 1, … 3 words, 4 words, …
  95. 95. /// (original name: CSSTUP) void setup() { shiftIndex = iota(storage.numLines) .map!(a => storage.numWords(a).iota .map!(b => ShiftIndexEntry(a, b))) .joiner .array; } /// (original name: CSWORD) char wordChar(LineNum lineNum, WordNum wordNum, CharNu { auto entry = shiftIndex[lineNum]; WordNum storageWordNum = (entry.firstWord + wordNu storage.numWords(entry.lineNum); return storage.wordChar(entry.lineNum, storageWord } /// (original name: CSWRDS) 0, 1, … 3 words, 4 words, … [0, 1, 2], [0, 1, 2, 3], …
  96. 96. /// (original name: CSSTUP) void setup() { shiftIndex = iota(storage.numLines) .map!(a => storage.numWords(a).iota .map!(b => ShiftIndexEntry(a, b))) .joiner .array; } /// (original name: CSWORD) char wordChar(LineNum lineNum, WordNum wordNum, CharNu { auto entry = shiftIndex[lineNum]; WordNum storageWordNum = (entry.firstWord + wordNu storage.numWords(entry.lineNum); return storage.wordChar(entry.lineNum, storageWord } /// (original name: CSWRDS) 0, 1, … 3 words, 4 words, … [0, 1, 2], [0, 1, 2, 3], … [ShiftIndexEntry(0, 0), ShiftIndexEntry(0, 1), …] […, ShiftIndexEntry(1, 2), ShiftIndexEntry(1, 3)]
 …
  97. 97. /// (original name: CSSTUP) void setup() { shiftIndex = iota(storage.numLines) .map!(a => storage.numWords(a).iota .map!(b => ShiftIndexEntry(a, b))) .joiner .array; } /// (original name: CSWORD) char wordChar(LineNum lineNum, WordNum wordNum, CharNu { auto entry = shiftIndex[lineNum]; WordNum storageWordNum = (entry.firstWord + wordNu storage.numWords(entry.lineNum); return storage.wordChar(entry.lineNum, storageWord } /// (original name: CSWRDS) ShiftIndexEntry(0, 0) ShiftIndexEntry(0, 1) ShiftIndexEntry(0, 3) ShiftIndexEntry(1, 0) ShiftIndexEntry(1, 1) ShiftIndexEntry(1, 2) ShiftIndexEntry(1, 3) …
  98. 98. D2 - Circular Shifter /// (original name: CSWRDS) LineNum numWords(LineNum lineNum) { auto entry = shiftIndex[lineNum]; return storage.numWords(entry.lineNum); } /// (original name: CSLNES) LineNum numLines() { return shiftIndex.length.to!LineNum; } /// (original name: CSCHRS) CharNum numCharacters(LineNum lineNum, WordNum wordN { auto entry = shiftIndex[lineNum]; WordNum storageWordNum = (entry.firstWord + word storage.numWords(entry.lineNum); } /// (original name: CSWORD) char wordChar(LineNum lineNum, WordNum wordNum, CharN { auto entry = shiftIndex[lineNum]; WordNum storageWordNum = (entry.firstWord + wordN storage.numWords(entry.lineNum); return storage.wordChar(entry.lineNum, storageWor } /// (original name: CSWRDS) LineNum numWords(LineNum lineNum) { auto entry = shiftIndex[lineNum]; return storage.numWords(entry.lineNum); }
  99. 99. D2 - Circular Shifter void setup() { shiftIndex = iota(storage.numLines) .map!(a => storage.numWords(a).iota .map!(b => ShiftIndexEntry(a, b))) .joiner .array; } /// (original name: CSWORD) char wordChar(LineNum lineNum, WordNum wordNum, CharNum charNum) { auto entry = shiftIndex[lineNum]; WordNum storageWordNum = (entry.firstWord + wordNum) % storage.numWords(entry.lineNum); return storage.wordChar(entry.lineNum, storageWordNum, charNum); } /// (original name: CSWRDS) LineNum numWords(LineNum lineNum) { auto entry = shiftIndex[lineNum];
  100. 100. D2 - Circular Shifter /// (original name: CSCHRS) CharNum numCharacters(LineNum lineNum, WordNum wordNum) { auto entry = shiftIndex[lineNum]; WordNum storageWordNum = (entry.firstWord + wordNum) % storage.numWords(entry.lineNum); return storage.numCharacters(entry.lineNum, storageWordNum); } // -- private:
  101. 101. D2 - Alphabetizing • Module 4: Alphabetizing. (…) ITH(i) will give the index of the circular shift which comes ith in the alphabetical ordering.
  102. 102. auto alphabeticIndex() { auto indexOffsets = new LineNum[numLines]; makeIndex!((a, b) => a.icmp(b) < 0)(numLines.iota.map!(a => l indexOffsets); return indexOffsets; } auto line(LineNum lineNum) { return numWords(lineNum) .iota .map!(wordNum => word(lineNum, wordNum)) .joiner(" ".byCodeUnit); } auto word(LineNum lineNum, WordNum wordNum) { return numCharacters(lineNum, wordNum) .iota .map!(charNum => wordChar(lineNum, wordNum, charNum)); } D2 - Alphabetizing private:
  103. 103. D2 - Alphabetizing private: ReturnType!alphabeticIndex index; auto alphabeticIndex() { auto indexOffsets = new LineNum[numLines]; makeIndex!((a, b) => icmp(a, b) < 0) (numLines.iota.map!(a => line(a).array), indexOffsets); return indexOffsets; }
  104. 104. D2 - Alphabetizing import two.circularshifter; public: /// (original name: ALPH) void setup() { index = alphabeticIndex; } /// (original name: ITH) LineNum ithLine(LineNum lineNum) { assert(lineNum < index.length); return index[lineNum]; } // The original functions ALPHAC, EQW, ALPHW, EQL a LineNum alphac(LineNum lineNum) {
  105. 105. D2 - Output • Module 5: Output. This module will give the desired printing of the set of lines or circular shifts.
  106. 106. D2 - Output .line) .each!writeln; } // -- private: auto line(LineNum lineNum) { return numWords(lineNum) .iota .map!(wordNum => word(lineNum, wordNum)) .joiner(" "); } auto word(LineNum lineNum, WordNum wordNum) { return numCharacters(lineNum, wordNum) .iota .map!(charNum => wordChar(lineNum, wordNum, charNum)) .byDchar; }
  107. 107. D2 - Output import std.stdio; import std.utf; import two.circularshifter; import two.alphabetizer; public: void printLines() { numLines .iota .map!(lineNum => lineNum .ithLine .line) .each!writeln; } // --
  108. 108. D2 - Result A Portrait of The Artist As a Young Man a Young Man A Portrait of The Artist As and The Sea The Old Man Artist As a Young Man A Portrait of The As a Young Man A Portrait of The Artist Ascent of Man The Descent of Man Man A Portrait of The Artist As a Young Man and The Sea The Old Man Descent of Man The Ascent of of Man Descent of Man The Ascent of The Artist As a Young Man A Portrait Old Man and The Sea The Portrait of The Artist As a Young Man A Sea The Old Man and The The Artist As a Young Man A Portrait of The Ascent of Man The Old Man and The Sea The Sea The Old Man and Young Man A Portrait of The Artist As a
  109. 109. Comparing Decompositions • The runtime representation of both decompositions might be the same • D2 might have a performance impact • Requires good optimizing compiler
  110. 110. Comparing Decompositions • Amenability to change • Comprehensibility • Testability • Parallel Development
  111. 111. Changeability • Input format • Store all data in memory • Pack the characters • Create an index of the shifts vs
 store the actual data • When to alphabetize
  112. 112. Changeability Module D1 D2 Input ✔ ✔ Line Storage — Circular Shifter Alphabetizer Output • Input format • Store all data in memory • Pack the characters • Create an index of the shifts vs
 store the actual data • When to alphabetize
  113. 113. Changeability Module D1 D2 Input ✔ Line Storage — ✔ Circular Shifter ✔ Alphabetizer ✔ Output ✔ • Input format • Store all data in memory • Pack the characters • Create an index of the shifts vs
 store the actual data • When to alphabetize
  114. 114. Changeability Module D1 D2 Input ✔ Line Storage — ✔ Circular Shifter ✔ Alphabetizer ✔ Output ✔ • Input format • Store all data in memory • Pack the characters • Create an index of the shifts vs
 store the actual data • When to alphabetize
  115. 115. Changeability Module D1 D2 Input Line Storage — Circular Shifter ✔ ✔ Alphabetizer ✔ Output ✔ • Input format • Store all data in memory • Pack the characters • Create an index of the shifts vs
 store the actual data • When to alphabetize
  116. 116. Changeability Module D1 D2 Input Line Storage — Circular Shifter Alphabetizer ✔ ✔ Output ✔ • Input format • Store all data in memory • Pack the characters • Create an index of the shifts vs
 store the actual data • When to alphabetize
  117. 117. Comprehensibility • Example: understanding the output module
  118. 118. Comprehensibility • Example: understanding the output module • Decomposition 1 module one.output; import std.algorithm; import std.range; import std.stdio; import one.input; import one.circularshifter; void printLines() { shiftIndex.map!(entry => entry.line).each!writeln; } private: auto line(ShiftIndexEntry entry) { auto a = entry.firstChar; auto b = entry.lineNum+1 >= lineIndex.length ? data.length : lineIndex[entry.lineNum+1];
  119. 119. Comprehensibility • Example: understanding the output module import one.circularshifter; void printLines() { shiftIndex.map!(entry => entry.line).each!writeln; } private: auto line(ShiftIndexEntry entry) { auto a = entry.firstChar; auto b = entry.lineNum+1 >= lineIndex.length ? data.length : lineIndex[entry.lineNum+1]; auto c = lineIndex[entry.lineNum]; auto d = (entry.firstChar-1).max(0).max(c); auto x = data[a .. b]; auto y = data[c .. d]; return joiner(only(x, y).filter!(a => !a.empty), " "); }
  120. 120. Comprehensibility • Example: understanding the output module • Decomposition 2 import std.algorithm; import std.range; import std.stdio; import std.utf; import two.circularshifter; import two.alphabetizer; public: void printLines() { numLines .iota .map!(lineNum => lineNum .ithLine .line) .each!writeln; }
  121. 121. Comprehensibility • Example: understanding the output module • Decomposition 2 { numLines .iota .map!(lineNum => lineNum .ithLine .line) .each!writeln; } // -- private: auto line(LineNum lineNum) { return numWords(lineNum) .iota .map!(wordNum => word(lineNum, wordNum)) .joiner(" "); } auto word(LineNum lineNum, WordNum wordNum) {
  122. 122. Comprehensibility • Example: understanding the output module • Decomposition 2 // -- private: auto line(LineNum lineNum) { return numWords(lineNum) .iota .map!(wordNum => word(lineNum, wordNum)) .joiner(" "); } auto word(LineNum lineNum, WordNum wordNum) { return numCharacters(lineNum, wordNum) .iota .map!(charNum => wordChar(lineNum, wordNum, charNum)) .byDchar; }
  123. 123. Testability • Parnas disputes the idea that information hiding is an empirical result • It’s a mathematical theorem: 1.You have two modules: A, B 2.You can prove A correct knowing only the interface of B 3.You change B without changing the interface 4.Then A doesn’t have to change
  124. 124. Testability Module D1 D2 Input ✔ Line Storage — ✔ Circular Shifter ✔ Alphabetizer ✔ Output ✔
  125. 125. Parallel Development • Decomposition 1
 • Decomposition 2 M1
 System design
 
 M2
 …
 M1
 System design 
 M2
 …

  126. 126. Evaluation • Information hiding: Yay or Nay? • Fred Brook’s The Mythical Man Month
  127. 127. Evaluation — MMM "Show me your flowcharts and conceal your tables, and I shall continue to be mystified. Show me your tables, and I won’t usually need your flowcharts; they’ll be obvious.”
  128. 128. Evaluation — MMM "Parnas (…) has proposed a still more radical solution. His thesis is that the programmer is most effective if shielded from, rather than exposed to the details of construction of system parts other than his own. This presupposes that all interfaces are completely and precisely defined. While that is definitely sound design, dependence upon its perfect accomplishment is a recipe for disaster.”
  129. 129. Evaluation — MMM "Parnas Was Right, and I Was Wrong about Information Hiding
 
 (…) I am now convinced that information hiding, today often embodied in object-oriented programming, is the only way of raising the level of software design.
 (…) [The traditional] technique ensures that programmers can know the detailed semantics of the interfaces they work to by knowing what is on the other side. Hiding those semantics leads to system bugs. On the other hand, Parnas's technique is robust under change and is more appropriate in a design-for-change philosophy. (…)
  130. 130. Evaluation • Information hiding: Yay or Nay? • Fred Brooks 👍 • Name • Is decomposition 2 sufficiently good?
  131. 131. Master, teach me the true ways of information hiding
  132. 132. Accidental Complexity • We already removed some accidental complexity: • Byte-oriented vs word-oriented • Unicode vs weird character comparison functions • Exceptions & assertions vs archaic error routines • Proper naming & namespacing • numWords vs WORDS • CHAR, CSCHAR vs module.wordChar
  133. 133. Accidental Complexity • Yet, D2 still has a lot of issues: • Global state • Lack of constructors / initializers • Each function must check if we are in the uninitialized state or in the steady state. • Sequence Interfaces • Memory allocation and data flow • setWordChar(lineNum, wordNum, charNum, c);
  134. 134. Sequence Interface • Input: a sequence of lines • Line: a sequence of words • Word: a sequence of characters
  135. 135. Sequence Interface f o o ␣ b a r b a z • Input: a sequence of lines • Line: a sequence of words • Word: a sequence of characters
  136. 136. Sequence Interface • Input: a sequence of lines • Line: a sequence of words • Word: a sequence of characters f o o b a z b a r line line word word word
  137. 137. Sequence Interface • Functions • CHAR • SETCHAR • WORDS • LINES • DELWRD • DELLINE • CHARS f o o ␣ b a r b a z
  138. 138. Sequence Interface • Functions • CHAR • SETCHAR • WORDS • LINES • DELWRD • DELLINE • CHARS f o o ␣ b a r b a z E.g. CHAR(r,w,c)
  139. 139. Sequence Interface • Functions • DELLINE • LINES • DELWRD • WORDS • CHARS • CHAR • SETCHAR
  140. 140. Sequence Interface • Functions • DELLINE • LINES • DELWRD • WORDS • CHARS • CHAR • SETCHAR f o o b a z b a r line line word word word
  141. 141. Sequence Interface • Functions • DELLINE • LINES • DELWRD • WORDS • CHARS • CHAR • SETCHAR f o o b a z b a r pointer pointer pointer pointer pointer
  142. 142. Sequence Interface • Functions • DELLINE • LINES • DELWRD • WORDS • CHARS • CHAR • SETCHAR
  143. 143. Sequence Interface • Functions • DELLINE • LINES • DELWRD • WORDS • CHARS • CHAR • SETCHAR • Input: a sequence of lines • Line: a sequence of words • Word: a sequence of characters 
 sequence<T> ?
  144. 144. Push vs Pull Algorithm 1 Target Algorithm 2 Target Algorithm 3 Target Algorithm 1 Algorithm 2 Algorithm 3 Target
  145. 145. Push vs Pull control Output Alphabetizer Circular shifter Input Line Storage Decomposition 2
  146. 146. Decomposition 2 Push vs Pull control Output Alphabetizer Circular shifter Input Line Storage algorithm target
  147. 147. Idiomatic Decomposition • Based on hierarchical abstract interfaces • Consistent sequence interface • Pull based • Efficient • D’s ranges and algorithms
  148. 148. Idiomatic Decomposition control Output Alphabetizer Circular shifter Input
  149. 149. Idiomatic Decomposition control Output Alphabetizer Circular shifter Input
  150. 150. Idiomatic Decomposition control Output Alphabetizer Circular shifter Input
  151. 151. Idiomatic Decomposition control Output Alphabetizer Circular shifter Input Range Interface sequence<T>
  152. 152. ID - Input .alphabetized // alphabetizer .each!writeln; // output } // -- private: /// Performs "foo bar n baz" -> [["foo", "bar"], ["baz"]] auto asWordLists(Range)(Range range) { return range .lineSplitter .map!(line => line .splitter!(chr => chr.isWhite) .filter!(word => !word.empty)); } /// Performs [["foo", "bar"], ["baz"]] -> [["foo", "bar"], [" auto withCircularShifts(Range)(Range range) {
  153. 153. ID - Circular Shift { return range .lineSplitter .map!(line => line .splitter!(chr => chr.isWhite) .filter!(word => !word.empty)); } /// Performs [["foo", "bar"], ["baz"]] -> /// [["foo", "bar"], ["bar", "foo"], ["baz"]] auto withCircularShifts(Range)(Range range) { return range .map!(line => line.rotations) .joiner; } /// Performs ["foo", "bar"] -> [["foo", "bar"], [" auto rotations(Range)(Range range)
  154. 154. ID - Circular Shift /// Performs [["foo", "bar"], ["baz"]] -> /// [["foo", "bar"], ["bar", "foo"], ["baz"]] auto withCircularShifts(Range)(Range range) { return range .map!(line => line.rotations) .joiner; } /// Performs ["foo", "bar"] -> [["foo", "bar"], ["bar", "foo"]] auto rotations(Range)(Range range) { auto len = range.walkLength; return range .repeat(len) .enumerate .map!(item => item.value.cycle.drop(item.index).take(len)); }
  155. 155. /// Performs ["foo", "bar"] -> [["foo", "bar"], ["bar", "foo"]] auto rotations(Range)(Range range) { auto len = range.walkLength; return range .repeat(len) .enumerate .map!(item => item.value.cycle.drop(item.index).take(len)); }
  156. 156. /// Performs ["foo", "bar"] -> [["foo", "bar"], ["bar", "foo"]] auto rotations(Range)(Range range) { auto len = range.walkLength; return range .repeat(len) .enumerate .map!(item => item.value.cycle.drop(item.index).take(len)); } f o o b a r
  157. 157. /// Performs ["foo", "bar"] -> [["foo", "bar"], ["bar", "foo"]] auto rotations(Range)(Range range) { auto len = range.walkLength; return range .repeat(len) .enumerate .map!(item => item.value.cycle.drop(item.index).take(len)); } f o o b a r f o o b a r f o o b a r
  158. 158. /// Performs ["foo", "bar"] -> [["foo", "bar"], ["bar", "foo"]] auto rotations(Range)(Range range) { auto len = range.walkLength; return range .repeat(len) .enumerate .map!(item => item.value.cycle.drop(item.index).take(len)); } f o o b a r f o o b a r f o o b a r 0 1
  159. 159. /// Performs ["foo", "bar"] -> [["foo", "bar"], ["bar", "foo"]] auto rotations(Range)(Range range) { auto len = range.walkLength; return range .repeat(len) .enumerate .map!(item => item.value.cycle.drop(item.index).take(len)); } f o o b a r f o o b a r f o o b a r 0 1 …f o o b a r f o o b a r f o o b a r …f o o b a r f o o b a r f o o b a r
  160. 160. /// Performs ["foo", "bar"] -> [["foo", "bar"], ["bar", "foo"]] auto rotations(Range)(Range range) { auto len = range.walkLength; return range .repeat(len) .enumerate .map!(item => item.value.cycle.drop(item.index).take(len)); } f o o b a r f o o b a r f o o b a r 0 1 …f o o b a r f o o b a r f o o b a r …f o o b a r f o o b a r f o o b a r
  161. 161. /// Performs ["foo", "bar"] -> [["foo", "bar"], ["bar", "foo"]] auto rotations(Range)(Range range) { auto len = range.walkLength; return range .repeat(len) .enumerate .map!(item => item.value.cycle.drop(item.index).take(len)); } f o o b a r f o o b a r f o o b a r 0 1 …f o o b a r f o o b a r f o o b a r …b a r f o o b a r f o o b a r f o o b a r b a r f o o
  162. 162. ID - Alphabetizing /// Performs [["foo", "bar"], ["baz"]] -> ["baz", "foo bar"] auto alphabetized(Range)(Range range) { return range .map!(line => line.joiner(" ")) .array .sort!((a, b) => icmp(a, b) < 0); }
  163. 163. ID - Output /// Performs [["foo", "bar"], ["baz"]] -> ["f auto alphabetized(Range)(Range range) { return range .map!(line => line.joiner(" ")) .array .sort!((a, b) => a.icmp(b) < 0); } void print(Range)(Range range) { range.each!writeln; }
  164. 164. ID - Control import std.range; import std.stdio; import std.string; import std.uni; public: void run(string inputFile) { // Original module: readText(inputFile) // input .asWordLists // input .withCircularShifts // circular shifter .alphabetized // alphabetizer .each!writeln; // output } // -- private:
  165. 165. void run(string inputFile) { // Original module: readText(inputFile) // input .asWordLists // input .withCircularShifts // circular shifter .alphabetized // alphabetizer .each!writeln; // output } /// Performs "foo bar n baz" -> [["foo", "bar"], ["baz"]] auto asWordLists(Range)(Range range) { return range .lineSplitter .map!(line => line .splitter!isWhite .filter!(word => !word.empty)); } /// Performs [["foo", "bar"], ["baz"]] -> [["foo", "bar"], ["bar", "foo"], ["baz"]] auto withCircularShifts(Range)(Range range) { return range .map!(line => line.rotations) .joiner; } /// Performs ["foo", "bar"] -> [["foo", "bar"], ["bar", "foo"]] auto rotations(Range)(Range range) { auto len = range.walkLength; return range .repeat(len) .enumerate .map!(item => item.value.cycle.drop(item.index).take(len)); } /// Performs [["foo", "bar"], ["baz"]] -> ["baz", "foo bar"] auto alphabetized(Range)(Range range) { return range .map!(line => line.joiner(" ")) .array .sort!((a, b) => icmp(a, b) < 0); }
  166. 166. ID - Result A Portrait of The Artist As a Young Man a Young Man A Portrait of The Artist As and The Sea The Old Man Artist As a Young Man A Portrait of The As a Young Man A Portrait of The Artist Ascent of Man The Descent of Man Man A Portrait of The Artist As a Young Man and The Sea The Old Man Descent of Man The Ascent of of Man Descent of Man The Ascent of The Artist As a Young Man A Portrait Old Man and The Sea The Portrait of The Artist As a Young Man A Sea The Old Man and The The Artist As a Young Man A Portrait of The Ascent of Man The Old Man and The Sea The Sea The Old Man and Young Man A Portrait of The Artist As a
  167. 167. Conclusion • The idiomatic D decomposition naturally reflects the problem statement • The decomposition that was begging to come out • What Parnas would have done?
  168. 168. Conclusion • So, what does Parnas72 mean for D? • D has strong modelling power. What was otherwise complex become, simple, straightforward, even obvious.

×