On Reflection in OO Programming Languages v1.6

Yann-Gaël Guéhéneuc
This work is licensed under a Creative
Commons Attribution-NonCommercial-
ShareAlike 3.0 Unported License
Département de génie informatique et de génie logiciel
Gina Cody School of Engineering and Computer Science
Department of Computer Science and Software Engineering
On Reflection in OO
Programming Languages
yann-gael.gueheneuc@concordia.ca
Version 1.6
2024/01/15

2/287
Any questions/comments are welcome at
yann-gael.gueheneuc@concordia.ca
The source code is available at
http://www.ptidej.net/tutorial/javareflection

3/287
Outline
 Rationale
 Definition
 Context
– Interconnections
 Scenarios
 Theory
 Conclusion
 Appendix
– Java: Class Loading
– Python: Inheritance

4/287
Outline
 Rationale
 Definition
 Context
 Scenarios
 Theory
 Conclusion
 Appendix

5/287
Use of instanceof
public abstract class Animal {
}
public class Cat extends Animal {
}
public class Dog extends Animal {
}
public class Expression {
public static String speak(final Animal anAnimal) {
final String sound;
if (anAnimal instanceof Dog) {
sound = "woof";
}
else if (anAnimal instanceof Cat) {
sound = "miaow";
}
else {
sound = "";
}
return sound;
}
public static void main(final String[] args) {
final Dog fido = new Dog();
System.out.println(Expression.speak(fido));
}
}

6/287
Use of instanceof
}
}
}
final String sound;
sound = "woof";
}
sound = "miaow";
}
else {
sound = "";
}
return sound;
}
}
}
Must test the type of
the object at run-time

7/287
Use of instanceof
}
}
}
final String sound;
sound = "woof";
}
sound = "miaow";
}
else {
sound = "";
}
return sound;
}
}
}
Breaks information
hiding of Cat and Dog

8/287
Use of instanceof
}
}
}
final String sound;
sound = "woof";
}
sound = "miaow";
}
else {
sound = "";
}
return sound;
}
}
}
Breaks information
hiding of Cat and Dog
Requires
special case

9/287
Use of instanceof
 Reason
– Must bypass the compiler and virtual machine’s
choice of the method to call
– Remember that overloading is resolved by the
static type of the argument, not its run-time
type (cf. Week 2)

10/287
Use of instanceof
 Problem
“Don't repeat yourself” (DRY): “Every piece of
knowledge must have a single, unambiguous,
authoritative representation within a system.”
—Andy Hunt and Dave Thomas
(in The Pragmatic Programmer)

11/287
Use of instanceof
public abstract String speak();
}
@Override
public String speak() {
return "miaow";
}
}
@Override
return "woof";
}
}
System.out.println(fido.speak());
}
}

12/287
Use of instanceof
}
@Override
return "miaow";
}
}
@Override
return "woof";
}
}
}
}

13/287
Use of instanceof
public class Cat {
return "miaow";
}
}
public class Dog {
return "woof";
}
}
public static void main(final String[] args) throws
IllegalAccessException,
IllegalArgumentException,
InvocationTargetException {
final Method[] method = Dog.class.getDeclaredMethods();
System.out.println(method[0].invoke(fido, new Object[0]));
}
}

14/287
Use of instanceof
public class Cat {
return "miaow";
}
}
public class Dog {
return "woof";
}
}
public static void main(final String[] args) throws
IllegalAccessException,
IllegalArgumentException,
InvocationTargetException {
}
}
Discover method of
object of unrelated
classes

15/287
Plugins
 Plugins are external, user-defined pieces of
code loaded, used, and unloaded to benefit
from their features only when needed
 Problem
– Closed-world assumption (at compile-time and
runtime) leads to the need for a mechanism to
perform dynamic loading

16/287
Serialisation
 Serialisation is the process of translating
data structures or object state into a format
that can be stored (file, network…)
 Problem
– Information hiding prevents access to the
internals of the data structures and to the
complete object states

17/287
Debugging/Profiling
 Debugging/profiling are the processes of
monitoring the execution of a program
– To understand its behaviour
– To measure its space and time complexity
 Problem
– Accessing the internals of objects and
controlling the program to collect relevant data

18/287
Patching
 Patching is the process of changing the
behaviour of a class or an object
– To modify its behaviour
– To add new behaviour
 Problem
– Modifying the behaviour at run-time without the
need to recompile and restart the program

19/287
Outline
 Rationale
 Definition
 Context
 Scenarios
 Theory
 Conclusion
 Appendix

20/287
Definition
 Reflection is the ability of a computer
program to examine and modify the
structure and behaviour of an object at
runtime

21/287
Use of instanceof
}
@Override
return "miaow";
}
}
@Override
return "woof";
}
}
}
}

22/287
Use of instanceof
}
@Override
public String miaow() {
return "miaow";
}
}
@Override
public String bark() {
return "woof";
}
}
}
}

23/287
Use of instanceof
}
@Override
return "miaow";
}
}
@Override
return "woof";
}
}
}
}
Unrelated classes

24/287
Use of instanceof
}
@Override
return "miaow";
}
}
@Override
return "woof";
}
}
}
}
Unrelated classes
Choice at run-time

25/287
Use of instanceof
}
@Override
return "miaow";
}
}
@Override
return "woof";
}
}
}
}
Unrelated classes
Choice at run-time

26/287
Outline
 Rationale
 Definition
 Context
 Scenarios
 Theory
 Conclusion
 Appendix

27/287
Context
 Reflection
– Ability of a computer program to examine and
modify the structure and behaviour of an object
at runtime

28/287
Context
 Reflection
– Theoretical interest
• What is an interpreter?
Brian Cantwell Smith “Procedural Reflection in
Programming Languages” (Ph.D. thesis, 1982)
– Practical interest
• Libraries
• Frameworks

29/287
Context
 Reflection
• Libraries
• Frameworks

30/287
Context
 Reflection
• Libraries
• Frameworks

31/287
Context
 Libraries
– Collections of data structures and algorithms
(possibly encapsulated in classes) used to
develop and run programs
 Frameworks
– Reusable sets of libraries, tools, and
conventions used to develop and run programs

32/287
Interconnections
 Clients–Libraries/Frameworks
– Linking
– Forking
– Inter-process communication
– Subclassing
– Dynamic loading/invoking

33/287
Interconnections
 Linking
(Contrast with virtual machines)
– Typically C/C++
– Several object files (.o)
– One executable (.exe)

34/287
Interconnections
 Forking
– Typical in most
languages
– Process duplication
• Is a real duplication
• Creates a new OS
process
final StringBuffer commandLine = new StringBuffer();
commandLine.append("..DOTbindotty ");
commandLine.append(aFilePath);
final Process process =
Runtime.getRuntime().exec(commandLine.toString());
final OutputMonitor errorStreamMonitor =
new OutputMonitor(...,process.getErrorStream(),...);
errorStreamMonitor.start();
final OutputMonitor inputStreamMonitor =
new OutputMonitor(...,process.getInputStream(),...);
inputStreamMonitor.start();
try {
process.waitFor();
}
catch (final InterruptedException ie) {
ie.printStackTrace(
Output.getInstance().errorOutput());
}
if (process.exitValue() != 0) {
...
}

35/287
Interconnections
 IPC
– Typical in most languages
– Use remote procedure calls
– Use well-defined protocols
• COM
• CORBA
• XML-RPC
– Web services

36/287
Interconnections
package net.ptidej.rpc;
import java.net.URL;
import org.apache.xmlrpc.client.XmlRpcClient;
import org.apache.xmlrpc.client.XmlRpcClientConfigImpl;
public class Client {
public static void main(final String[] args)
throws Exception {
final XmlRpcClientConfigImpl config =
new XmlRpcClientConfigImpl();
config.setServerURL(
new URL("http://127.0.0.1:8080/xmlrpc"));
final XmlRpcClient client = new XmlRpcClient();
client.setConfig(config);
final Object[] params = new Object[] {
new Integer(33), new Integer(9) };
final Integer result = (Integer)
client.execute("Calculator.add", params);
System.out.println(result);
}
}
import org.apache.xmlrpc.server.PropertyHandlerMapping;
import org.apache.xmlrpc.server.XmlRpcServer;
import org.apache.xmlrpc.webserver.WebServer;
public class Server {
public static void main(final String[] args)
throws Exception {
final WebServer webServer = new WebServer(8080);
final XmlRpcServer xmlRpcServer =
webServer.getXmlRpcServer();
final PropertyHandlerMapping phm =
new PropertyHandlerMapping();
phm.addHandler("Calculator", Calculator.class);
xmlRpcServer.setHandlerMapping(phm);
webServer.start();
}
}
public class Calculator {
public int add(final int i1, final int i2) {
return i1 + i2;
}
public int sub(final int i1, final int i2) {
return i1 - i2;
}
}

37/287
Interconnections
 Subclassing
– Typical in most object-
oriented languages
– Structural, static relation
public class OutputMonitor extends Thread {
...
public OutputMonitor(...) {
this.setName(threadName);
this.setPriority(Thread.MAX_PRIORITY);
...
}
public void run() {
try {
int value = 0;
byte[] bytes;
char lastWrittenChar;
while ((value = this.inputStream.read()) > 0) {
synchronized (System.err) {
synchronized (System.out) {
if (value != 13 && value != 10) {
lastWrittenChar = (char) value;
...
}}
}
}
catch (final IOException ioe) {
...

38/287
Interconnections
 Subclassing
– Hooks and templates
• Hot spots = hooks
• Frozen spots = templates
– Hooks are typically abstract methods
– Templates typically use hooks

39/287
Interconnections
 Subclassing
– Hooks and templates
• JUnit

40/287
Interconnections
 Template
 Hooks
public abstract class TestCase
extends Assert implements Test {
public void runBare() throws Throwable {
setUp();
try {
runTest();
}
finally {
tearDown();
}
}
protected void setUp() throws Exception {
}
protected void tearDown() throws Exception {
}
...
}

41/287
Interconnections
 Subclassing
– Heavily used in object-oriented programs
– Heavily used in design patterns
(Only Singleton does not explicitly use subclassing)
• Abstract Factory
• Composite
• Decorator
• Observer
• Visitor

42/287
Interconnections
 Dynamic loading
– In different programming languages (but not all),
it is the possibility to load, use, and unload a
piece of code at runtime
– In Java, it is the possibility to load and unload
a class and to choose and invoke its methods
(and to access its fields…) at runtime

43/287
Interconnections
 Poor man’s profiler
– Calls the main() method of a class whose name is
given at runtime, by the user
– Displays the time spent in the called method
public final class PoorManProfiler {
try {
final Class toBeRun = Class.forName(args[0]);
final Method mainMethod =
toBeRun.getMethod("main", new Class[] { String[].class });
final long startTime = System.currentTimeMillis();
mainMethod.invoke(null, new Object[] { new String[0] });
final long endTime = System.currentTimeMillis();
System.out.println();
System.out.println(endTime - startTime);
}
catch (final Exception e) {
e.printStackTrace();
}
}
}

44/287
Return to Reflection
 Reflection enables dynamic loading
– Reflection  dynamic loading
– Dynamic loading  Reflection
• Think of dynamically-loaded shared libraries in
C-based operating systems, such as AmigaOS,
Linux, UN*X, Windows…
⁄

45/287
Outline
 Rationale
 Definition
 Context
 Scenarios
 Theory
 Conclusion
 Appendix

46/287
Scenarios
 Original problem
– Different interpretations
– Different contexts
 Four scenarios
 Three programming languages

47/287
Scenarios
 Scenario 1: invoke an arbitrary method on
an object (see the problems with
instanceof and plugins)
– Given a class C
– Given an object o, instance of C
– Identify all the methods available on o
– Invoke a method using its name foo

48/287
Scenarios
 Scenario 2: access the complete (including
private) state of an object (see the problem
with serialisation)
– Save on disk the complete state of o
– Restore from disk the object o at a later time

49/287
Scenarios
 Scenario 3: count the number of instances of
a class created at runtime (see the problem
with debugging/profiling)
– Given a class C
– Record the number of its instances ever created
– Report this number when the program ends

50/287
Scenarios
 Scenario 4: patch the method of a class to
change its behaviour (see the problem with
patching)
– Without changing o or C
– Change the behaviour of o.foo()

51/287
Scenarios Java
Smalltalk
C++
Python

53/287
Scenarios
 Scenario 1:
– Given a class C
public class C {
private int i;
public C(final int anInt) {
this.i = anInt;
}
public void foo(final String s) {
System.out.print(s);
System.out.println(this.i);
}
}

54/287
Scenarios
 Scenario 1:
– Given a class C
Identify all the methods available on o
foo
getClass
hashCode
equals
toString
notify
notifyAll
wait
wait
wait
Invoke a method using its name foo
This is foo: 42

55/287
Scenarios
 Scenario 1:
– Given a class C
final C o = new C(42);
System.out.println("Identify all the methods available on o");
final Class<?> classOfO = o.getClass();
final Method[] methodsOfC = classOfO.getMethods();
for (int i = 0; i < methodsOfC.length; i++) {
final Method method = methodsOfC[i];
System.out.print('t');
System.out.println(method.getName());
}
System.out.println("Invoke a method using its name foo");
final Method fooMethod = classOfO.getMethod("foo", new Class[] { String.class });
fooMethod.invoke(o, new Object[] { "tThis is foo: " });

56/287
Scenarios
 Scenario 1:
– Given a class C
foo
getClass
hashCode
equals
toString
notify
notifyAll
wait
wait
wait
This is foo: 42

57/287
Scenarios
 Scenario 2:
public class C {
private int i;
public C() {
this(-1);
}
this.i = anInt;
}
}
}

58/287
Scenarios
 Scenario 2:
Save on disk the complete state of o
i = 42
Restore from disk the object o at a later time
This is foo on o2: 43

59/287
Scenarios
 Scenario 2:
final C o1 = new C(42);
System.out.println("Save on disk the complete state of o");
final Class<?> classOfO = o1.getClass();
final Field[] fieldsOfC = classOfO.getDeclaredFields();
for (int i = 0; i < fieldsOfC.length; i++) {
final Field field = fieldsOfC[i];
field.setAccessible(true);
System.out.print('t' + field.getName());
System.out.println(" = " + field.getInt(o1));
}
System.out.println("Restore from disk the object o at a later time");
final C o2 = (C) classOfO.newInstance();
final Field fieldiOfC = classOfO.getDeclaredField("i");
fieldiOfC.setAccessible(true);
fieldiOfC.setInt(o2, 43);
o2.foo("tThis is foo on o2: ");

60/287
Scenarios
 Scenario 2:
i = 42

61/287
Scenarios
 Scenario 3:
– Given a class C
– Record the numbers of its instance ever created
public class C {
private static int numberOfInstances;
public static int getNumberOfInstances() {
return C.numberOfInstances;
}
private int i;
public C() {
this(-1);
}
C.numberOfInstances++;
this.i = anInt;
}
}
}

62/287
Scenarios
 Scenario 3:
net.ptidej.reflection.scenario3.C@150bd4d
net.ptidej. reflection.scenario3.C@1bc4459
net.ptidej.reflection.scenario3.C@12b6651
3
– Given a class C

63/287
Scenarios
 Scenario 3:
public class Main {
System.out.println(o1 + "n" + o2 + 'n' + o3);
System.out.println(C.getNumberOfInstances());
}
}
– Given a class C

64/287
Scenarios
 Scenario 3:
net.ptidej.reflection.scenario3.C@1bc4459
3
– Given a class C

65/287
Scenarios
 Scenario 3:
net.ptidej.reflection.scenario3.C@1bc4459
3
To fulfill this scenario,
we must modify C
– Given a class C

66/287
Scenarios
 Class Class is a metaclass
– It describes other classes
 Class Method is a class
– It describes methods
 Class Field is a class
– It describes fields

67/287
Scenarios
– It allows to reify classes
– It allows to reify methods, these language
constructs become first-class entities
– It allows to reify fields , these language

68/287
Scenarios
– It allows to reify fields, these language

69/287
Scenarios
– It allows to reify fields, these language
Reification is the process of
making available an implicit
construct of a language

70/287
Scenarios
 We never modified class Class
– We only used it to obtain the methods and fields
declared by particular classes, e.g., C
 We used static fields and methods to count
number of instances

71/287
Scenarios
 Scenario 4:
public interface I {
public void foo(final String s);
}
public class C implements I {
private int i;
this.i = anInt;
}
}
}

72/287
Scenarios
 Scenario 4:
This is o: 42
Forwarding method: "foo"
Original arguments: "This is o: "
New arguments: "THIS IS O: "
THIS IS O: 42

73/287
Scenarios
 Scenario 4:
final I o = new C(42);
o.foo("This is o: ");
final InvocationHandler handler = new InvokationHandler(o);
final I proxy =
(I) Proxy.newProxyInstance(
I.class.getClassLoader(),
new Class[] { I.class },
handler);
Assert.assertTrue(proxy instanceof I);
Assert.assertFalse(proxy instanceof C);
Assert.assertTrue(proxy.equals(o));
proxy.foo("This is o: ");

74/287
Scenarios

Scenario
4:
public class InvokationHandler implements InvocationHandler {
private final I i;
public InvokationHandler(final I aConcreteImplementationOfI) {
this.i = aConcreteImplementationOfI;
}
public Object invoke(
final Object aProxy,
final Method aMethod,
final Object[] someArgs) throws Throwable {
if (aMethod.getName().equals("foo")) {
final String arg = (String) someArgs[0];
final String newArg = arg.toUpperCase();
System.out.print("Forwarding method: "");
System.out.print(aMethod.getName());
System.out.print(""ntOriginal arguments: "");
System.out.print(arg);
System.out.print(""ntNew arguments: "");
System.out.print(newArg);
System.out.println('"');
this.i.foo(newArg);
return null;
}
else {
return aMethod.invoke(this.i, someArgs);
}
}
}
–
Given
an
object
o,
instance
of
C
–
Without
changing
o
or
C
–
Change
the
behaviour
of
o.foo()

75/287
Scenarios
 Scenario 4:
final I proxy =
handler);

76/287
Scenarios
 Scenario 4:
final I proxy =
handler);
Properties of the
proxy ensured by
the Java VM

78/287
Scenarios
 Smalltalk
– Everything is an object

79/287
Scenarios
 Smalltalk
– Everything is an object
– Everything is an object, really
static

80/287
Scenarios
 Smalltalk
– SmalltalkImage is the class that “represent[s]
the current image and runtime environment,
including system organization, the virtual
machine, object memory, plugins, and source
files. [Its] instance variable #globals is a
reference to the system dictionary of global
variables and class names. [Its] singleton
instance is called Smalltalk.”

81/287
Scenarios
 Smalltalk
– Object is “the root class for almost all of the
other classes in the class hierarchy. […] Class
Object provides default behavior common to
all normal objects, such as access, copying,
comparison, error handling, message sending,
and reflection. Also utility messages that all
objects should respond to are defined here.”

82/287
Scenarios
 Smalltalk
– Class “[adds] instance-specific behavior to
various class-describing objects in the system.
This typically includes messages for initializing
class variables and instance creation messages
particular to a class. There is only one instance
of a particular Metaclass, namely the class
which is being described.”

83/287
Scenarios
 Scenario 1:
– Given a class C
Object subclass: #C
instanceVariableNames: 'i'
classVariableNames: ''
poolDictionaries: ''
category: 'CSE3009'.
C class compile: 'newWithi: anInt
^(self new) i: anInt ; yourself.'.
C compile: 'foo: aText
Transcript show: aText.
Transcript show: i.
Transcript cr.'.
C compile: 'i: anInt
i := anInt.'.

84/287
Scenarios
 Scenario 1:
– Given a class C
an IdentitySet(#handles: #longPrintString #actionMap
#hasContentsInExplorer [...] #foo: [...] #i: [...]
#creationStamp)
This is foo: 42

85/287
Scenarios
 Scenario 1:
– Given a class C
|o|
o := C newWithi: 42.
Transcript show: 'Identify all the methods available on o' ; cr.
Transcript show: C allSelectors ; cr.
Transcript show: 'Invoke a method using its name foo' ; cr.
(C compiledMethodAt: #foo:) valueWithReceiver: o arguments: #('This is foo: ').

86/287
Scenarios
 Scenario 1:
– Given a class C
|o|
Transcript show: 'Identify all the methods available on o' ; cr.
Transcript show: C allSelectors ; cr.
Transcript show: 'Invoke a method using its name foo' ; cr.
(C compiledMethodAt: #foo:) valueWithReceiver: o arguments: #('This is foo: ').
Simple and straightforward

87/287
Scenarios
 Scenario 2:
Object subclass: #C
Transcript show: i.
Transcript cr.'.
i := anInt.'.

88/287
Scenarios
 Scenario 2:
42

89/287
Scenarios
 Scenario 2:
|o1 o2|
o1 := C newWithi: 42.
Transcript show: 'Save on disk the complete state of o' ; cr.
Transcript show: (o1 instVarAt: 1) ; cr.
Transcript show: 'Restore from disk the object o at a later time' ; cr.
o2 := C new.
o2 instVarNamed: 'i' put: 43.
o2 foo: 'This is foo on o2: '.

90/287
Scenarios
 Scenario 2:
|o1 o2|
Transcript show: 'Save on disk the complete state of o' ; cr.
Transcript show: (o1 instVarAt: 1) ; cr.
Transcript show: 'Restore from disk the object o at a later time' ; cr.
o2 := C new.
o2 instVarNamed: 'i' put: 43.
o2 foo: 'This is foo on o2: '.
Again, simple and straightforward

91/287
Scenarios
 Scenario 2:
– Pharo (a recent implementation of Smalltalk)
provides is a general object serialiser that can
serialise any object
– Uses reflection to implement these methods!
1@2 serializeToFileNamed: 'hello.fuel'.
FLMaterializer materializeFromFileNamed: 'hello.fuel'
Thanks to Marcus Denker for pointing out these helper methods

92/287
Scenarios
 Scenario 3:
Object subclass: #C
classVariableNames: 'NumberOfInstances'
Transcript show: i.
Transcript cr.'.
i := anInt.'.
– Given a class C

93/287
Scenarios
 Scenario 3:
Object subclass: #C
classVariableNames: 'NumberOfInstances'
Transcript show: i.
Transcript cr.'.
i := anInt.'.
– Given a class C
Only difference with
Scenarios 1 and 2

94/287
Scenarios
 Scenario 3:
Transcript show: i.
Transcript cr.'.
i := anInt.'.
– Given a class C

95/287
Scenarios
 Scenario 3:
Transcript show: i.
Transcript cr.'.
i := anInt.'.
Same as Scenarios 1 and 2
– Given a class C

96/287
Scenarios
 Scenario 3:
C class compile: 'initialize
NumberOfInstances := 0.'.
C class compile: 'new
NumberOfInstances := NumberOfInstances + 1.
^ self basicNew initialize.'.
C class compile: 'numberOfInstances
^ NumberOfInstances.'.
– Given a class C

97/287
Scenarios
 Scenario 3:
C class compile: 'initialize
NumberOfInstances := 0.'.
C class compile: 'new
C class compile: 'numberOfInstances
– Given a class C
“Write access” to the metaclass

98/287
Scenarios
 Scenario 3:
|o1 o2 o3|
C initialize.
Transcript show: o1 ; show: ' ' ; show: o2 ; show: ' ' ; show: o3 ; cr.
Transcript show: C numberOfInstances.
– Given a class C

99/287
Scenarios
 Scenario 3:
a C a C a C
3
– Given a class C

100/287
Scenarios
 Scenario 3:
a C a C a C
3
To fulfill this
scenario, we modified
C class not C
– Given a class C

101/287
Scenarios
 Scenario 3:
In Smalltalk, each class has one anonymous
meta-class, which can be modified
• Adding new variables and methods which are thus
class variables and class methods
• These methods apply on the class, these fields
belong to the class (thus unique)
– Given a class C

102/287
Scenarios
 Scenario 3:
– What about a subclass D of C?
– How would its instances be counted?
a C a C a C
3
– Given a class C
Thanks to Christophe Dony for suggesting this use of class vs. class instance variables

103/287
Scenarios
 Scenario 3:
– In C++ and Java: static = global
• The class variables is shared by all the instances of
its declaring class… and its subclasses!
final D o4 = new D();
// Eclipse advises to use C to call getNumberOfInstances().
System.out.println(o4);
System.out.println(D.getNumberOfInstances());
– Given a class C

104/287
Scenarios
 Scenario 3:
– In Smalltalk: class vs. class instance
• Class variable vs. class instance variable
– A class variable is shared by all the instance of the
class and all the instances of its subclasses
– A class instance variable is unique to each class,
inherited from the super-class, like instance variables
C class instanceVariableNames:
'IndividualClassNumberOfInstances'
– Given a class C

105/287
Scenarios
 Scenario 3:
C subclass: #D
instanceVariableNames: ''
category: 'Ptidej'.
D class compile: 'initialize
IndividualClassNumberOfInstances := 0.'.
D class compile: 'new
IndividualClassNumberOfInstances := IndividualClassNumberOfInstances + 1.
D class compile: 'numberOfInstances
D class compile: 'individualClassNumberOfInstances
^ IndividualClassNumberOfInstances.'.
– Given a class C

106/287
Scenarios
 Scenario 3:
| o4 |
D initialize.
o4 := D new.
Transcript show: o4 ; cr.
Transcript show: C numberOfInstances ; cr.
Transcript show: D numberOfInstances ; cr.
Transcript show: D individualClassNumberOfInstances ; cr.
– Given a class C
a D
4
4
1

107/287
Scenarios
 Scenario 3:
– In Java: the JVM contains the world
• Possible to connect to a JVM using the Java Debug
Interface of the Java Platform Debug Architecture
• Possible to reify the classes existing in the remote
JVM and to obtain their instances
com.sun.jdi.ReferenceType.instances(long)
• Special feature of the JVM, “outside” of the normal
constructs of the language
– Given a class C
http://stackoverflow.com/questions/1947122/
is-there-a-simple-way-of-obtaining-all-object-instances-of-a-specific-class-in-j/1947200#1947200

108/287
Scenarios
 Scenario 3:
– In Smalltalk: the image contains the world
• Possible to ask it all the instances of a particular class
or all the instances currently in memory
Transcript show: (C allInstances size).
– Given a class C
Thanks to Marcus Denker for pointing out this solution

109/287
Scenarios
 Scenario 4:
Object subclass: #C
Transcript show: i.
Transcript cr.'.
i := anInt.'.

110/287
Scenarios
 Scenario 4:
This is o: 42
Intercepting message send: foo:
Original arguments: This is o:
New arguments: THIS IS O:
THIS IS O: 42

111/287
Scenarios
 Scenario 4:
Several choices
• ObjectWrappers: http://magaloma.seasidehosting.st/
Kernel#ObjectTracer
• MethodWrappers: http://pharo.gforge.inria.fr/
PBE1/PBE1ch15.html
• Reflectivity: http://scg.unibe.ch/Research/Reflectivity
• …

112/287
Scenarios
 Scenario 4:
Several choices
Kernel#ObjectTracer
PBE1/PBE1ch15.html
• …

113/287
Scenarios
 Scenario 4:
| o aWrapper |
o foo: 'This is o: '.
aWrapper := WrapperForC on: o.
"o become: aWrapper."
aWrapper foo: 'This is o: '.
aWrapper xxxUnTrace.

114/287
 Scenario 4:
ObjectTracer subclass: #WrapperForC
instanceVariableNames: ''
WrapperForC compile: 'doesNotUnderstand: aMessage
"Intercept the selector foo:"
| sel arg newArg |
sel := aMessage selector.
sel = ''foo:'' ifTrue: [
arg := (aMessage arguments) at: 1.
newArg := arg asUppercase.
Transcript show: ''Intercepting message send: '' ; show: sel ; cr.
Transcript tab ; show: ''Original arguments: '' ; show: arg ; cr.
Transcript tab ; show: ''New arguments: '' ; show: newArg ; cr.
aMessage argument: newArg.
aMessage sendTo: (self xxxViewedObject)
]
ifFalse: [
"Transcript show: aMessage."
]'.

115/287
Scenarios
 Scenario 4:
This is o: 42
THIS IS O: 42
| o aWrapper |
aWrapper := WrapperForC on: o.
"o become: aWrapper."
aWrapper foo: 'This is o: '.
aWrapper xxxUnTrace.

116/287
Scenarios
 Scenario 4:
– We created the class WrappedForC that does
not understand many messages (as subclass
of ProtoObject)
– We wrapped an instance of WrappedForC
around o and use doesNotUnderstand: to
proxy messages

117/287
Scenarios
 Scenario 4:
– We created the class WrappedForC that does
not understand many messages (as subclass
of ProtoObject)
– We wrapped an instance of WrappedForC
around o and use doesNotUnderstand: to
proxy messages

118/287
Scenarios
 Scenario 4:
Several choices
Kernel#ObjectTracer
PBE1/PBE1ch15.html
• …

119/287
Scenarios
 Reflectivity
– Marcus Denker, RMOD Team, INRIA
Lille, and others
– Extensions to the standard reflection features of
Smalltalk (both structural and behavioural)
• Structural reflection is extended by sub-method
reflection: method bodies are first-class
• Behavioural reflection is provided by Geppetto, an
implementation of Partial Behavioural Reflection

120/287
Scenarios
 Scenario 4:
Object subclass: #C
Transcript show: i.
Transcript cr.'.
i := anInt.'.

121/287
Scenarios
 Scenario 4:
– Sub-method reflection
nodes := (C>>#foo:) sends select: [:each |
((each selector ~= #show:)
or: (each arguments) size ~= 1)
ifTrue: [ false ]
ifFalse: [ (each arguments at: 1) name = 'aText'. ].
].

122/287
Scenarios
 Scenario 4:
– Meta-object
gpLink := GPLink metaObject: [:arg1 :node |
| newArg |
newArg := arg1 asUppercase.
Transcript show: 'Intercepting message send: '.
Transcript show: node ; cr.
Transcript tab ; show: 'Original arguments: '.
Transcript show: arg1 ; cr.
Transcript tab ; show: 'New arguments: '.
Transcript show: newArg ; cr.
Transcript show: newArg.
].
gpLink control: #instead.

123/287
Scenarios
 Scenario 4:
– Linking
nodes do: [:node |
node link: gpLink].
gpLink uninstall.
This is o: 42
THIS IS O: 42

124/287
Scenarios Java
Smalltalk
C++
Python

126/287
Scenarios
 Scenarios 1, 2, 3, and 4
More difficult to implement “as is”
• http://stackoverflow.com/questions/359237/why-does-
c-not-have-reflection
• http://www.open-std.org/jtc1/sc22/wg21/docs/papers/
2005/n1751.html

127/287
Scenarios
Third-party solutions
• Mirror: http://kifri.fri.uniza.sk/~chochlik/mirror-
lib/html/index.html#introduction
• VisualStudio.NET: http://msdn.microsoft.com/en-
us/library/y0114hz2%28v=vs.110%29.aspx
• …

128/287
Scenarios
Third-party solutions
• Mirror: http://kifri.fri.uniza.sk/~chochlik/mirror-
lib/html/index.html#introduction
• VisualStudio.NET: http://msdn.microsoft.com/en-
us/library/y0114hz2%28v=vs.110%29.aspx
• …

129/287
Scenarios
 Mirror
– Matúš Chochlík, University of Žilina,
Žilina, Slovakia
“[C]ompile-time and run-time meta-data
describing C++ constructs like namespaces,
types typedef-ined types, enums, classes
with their base classes, member variables,
constructors, member functions, etc.”

130/287
Scenarios
 Scenario 1:
– Given a class C
namespace cse3009 {
class C {
private:
int i;
public:
C(const int _i) : i(_i) {
}
int get_i(void) const {
return this->i;
}
void set_i(const int _i) {
this->i = _i;
}
void foo(const std::string _s) const {
std::cout << _s << this->i << std::endl;
}
};
}

131/287
Scenarios
 Scenario 1:
– Given a class C
(The type of o is cse3009::C)
cse3009::C::i
cse3009::C::foo
This is foo: 42

132/287
Scenarios
 Scenario 1:
– Given a class C
int main(void) {
cse3009::C * o = new cse3009::C(42);
std::cout << "Identify all the methods available on o" << std::endl;
identify_members(o);
std::cout << "Invoke a method using its name foo" << std::endl;
invoke_member_function(o);
return 0;
}

133/287
Scenarios
 Scenario 1:
– Given a class C
template<typename O>
void identify_members(O * _o) {
using namespace mirror;
typedef MIRRORED_CLASS(O) meta_Class;
std::cout << "t(The type of o is " << meta_Class::full_name()
<< ")" << std::endl;
std::cout << "t";
mp::for_each_ii< all_member_variables<meta_Class> >(info_printer());
std::cout << std::endl << "t";
mp::for_each_ii< member_functions<meta_Class> >(info_printer());
std::cout << std::endl;
}

134/287
Scenarios
 Scenario 1:
– Given a class C
void invoke_member_function(O * _o) {
typedef MIRRORED_CLASS(cse3009::C) meta_Class;
typedef mp::at_c<overloads<
mp::at_c<member_functions<meta_Class>, 0>::type>, 0>::type
meta_foo;
my_invoker_maker::invoker<meta_foo>::type invoker(
std::string("tThis is foo: "));
invoker.call_on(*_o);
}

135/287
Scenarios
 Mirror
– Provides essentially the same concepts and
features as java.lang.reflect of Java
– Uses templates and (manual) registration to
collect metadata that does not exist unless
manually created through macro preprocessing

136/287
Scenarios
 Scenario 1:
– Given a class C
MIRROR_REG_BEGIN
MIRROR_QREG_GLOBAL_SCOPE_NAMESPACE(std)
MIRROR_REG_CLASS_BEGIN(struct, std, string)
MIRROR_REG_CLASS_END
MIRROR_QREG_GLOBAL_SCOPE_NAMESPACE(cse3009)
MIRROR_REG_CLASS_BEGIN(class, cse3009, C)
MIRROR_REG_CLASS_MEM_VARS_BEGIN
MIRROR_REG_CLASS_MEM_VAR_GET_SET(
private, _, int, i, _.get_i(), _.set_i(_i))
MIRROR_REG_CLASS_MEM_VARS_END
MIRROR_REG_MEM_FUNCTIONS_BEGIN
MIRROR_REG_MEM_OVLD_FUNC_BEGIN(foo)
MIRROR_REG_MEM_FUNCTION_BEGIN(public, _, void, foo, 1)
MIRROR_REG_MEM_FUNCTION_PARAM(std::string, _s)
MIRROR_REG_MEM_FUNCTION_END(1, _)
MIRROR_REG_MEM_OVLD_FUNC_END(foo)
MIRROR_REG_MEM_FUNCTIONS_END
MIRROR_REG_CLASS_END
MIRROR_REG_END

137/287
Scenarios
 Scenario 1:
– Given a class C
template<class Unused>
struct my_manufacturer<std::string, Unused> {
const std::string s;
template<class ConstructionInfo>
inline my_manufacturer(const std::string _s,
const ConstructionInfo construction_info) : s(_s) {
}
void finish(std::string) const {
}
inline std::string operator()(void) const {
return s;
}
};
typedef mirror::invoker_maker<my_manufacturer, mirror::default_fact_suppliers,
my_enumerator, void> my_invoker_maker;

138/287
Scenarios
 Mirror
– Uses complex code that can be easily and
advantageously hidden in libraries
– Will possibly become part of Boost and the
standard reflection library for C++
• http://kifri.fri.uniza.sk/~chochlik/jtc1_sc22_wg21/
std_cpp_refl-latest.pdf

139/287
Scenarios
 Scenario 2:
namespace cse3009 {
class C {
private:
int i;
public:
}
int get_i(void) const {
return this->i;
}
void set_i(const int _i) {
this->i = _i;
}
}
};
}

140/287
Scenarios
 Scenario 2:
(The type of o is cse3009::C)
cse3009::C::i = 42
43

141/287
Scenarios
 Scenario 2:
int main(void) {
cse3009::C * o = new cse3009::C(42);
std::cout << "Save on disk the complete state of o" << std::endl;
save_on_disk(o);
std::cout << "Restore from disk the object o at a later time" << std::endl;
restore_from_disk<cse3009::C>();
return 0;
}

142/287
Scenarios
 Scenario 2:
void save_on_disk(O * _o) {
typedef MIRRORED_CLASS(O)meta_Class;
std::cout << "t(The type of o is " <<
meta_Class::full_name() << ")" << std::endl << "t";
mp::for_each_ii<all_member_variables<meta_Class>>(
info_printer_variables<O>(*_o));
}

143/287
Scenarios
 Scenario 2:
template<class O>
struct info_printer_variables {
O o;
inline info_printer_variables(O _o) : o(_o) {
}
template<class IterInfo>
void operator()(IterInfo) {
std::cout << IterInfo::type::full_name()
<< " = " << IterInfo::type::get(o);
if (!IterInfo::is_last::value)
std::cout << ", ";
}
};

144/287
Scenarios
 Scenario 2:
template<class O>
struct info_printer_variables {
O o;
inline info_printer_variables(O _o) : o(_o) {
}
void operator()(IterInfo) {
std::cout << IterInfo::type::full_name()
<< " = " << IterInfo::type::get(o);
if (!IterInfo::is_last::value)
std::cout << ", ";
}
};
Get the value of reflected variable

145/287
Scenarios
 Scenario 2:
void restore_from_disk(void) {
typedef typename my_factory_maker::factory<O>::type my_factory_type;
my_factory_type my_factory(43);
O o(my_factory());
std::cout << "t" << o.get_i() << std::endl;
}

146/287
Scenarios
 Scenario 2:
typedef mirror::factory_maker<
my_manufacturer,
mirror::default_fact_suppliers,
my_enumerator,
void> my_factory_maker;

147/287
Scenarios
 Scenario 2:
typedef mirror::factory_maker<
my_manufacturer,
my_enumerator,
void> my_factory_maker;
Factory generating instances
of (possibly) different types

148/287
Scenarios
 Scenario 2:
struct my_manufacturer<void, Unused> {
template<typename Context>
my_manufacturer(int _value, Context) { }
void finish(int) { }
inline int add_constructor(
int _value, ConstructionInfo) const {
return _value;
}
inline int index(void) {
return 0;
}
};

149/287
Scenarios
 Scenario 2:
struct my_manufacturer<void, Unused> {
template<typename Context>
my_manufacturer(int _value, Context) { }
inline int add_constructor(
int _value, ConstructionInfo) const {
return _value;
}
inline int index(void) {
return 0;
}
};
To pass on to
the constructor

150/287
template<class Product, typename IsEnum>
struct my_base_manufacturer {
Product x;
struct constr_param_name_printer {
inline void operator()(IterInfo) const {
if (!IterInfo::is_first::value)
std::cout << ", ";
std::cout << IterInfo::type::base_name();
}
};
struct constr_context_printer {
std::cout << "::";
}
};
my_base_manufacturer(int tabs,
ConstructionInfo construction_info,
const Product& _x) : x(tabs) { }
void finish(int) {
}
inline Product operator()(void) {
return x;
}
};
template<class Product, class Unused>
struct my_enumerator :
my_base_manufacturer<Product, std::true_type> {
inline my_enumerator(int tabs,
ConstructionInfo construction_info) :
my_base_manufacturer<Product, std::true_type>(
tabs, construction_info, Product()) { }
return NULL;
}
};
struct my_manufacturer {
typedef typename mirror::factory_maker<
my_manufacturer,
my_enumerator,
Unused> maker;
typename maker::template factory<Product>::type f;
inline my_manufacturer(
int _value, ConstructionInfo construction_info) :
f(construction_info) { }
return f();
}
};

151/287
Product x;
std::cout << ", ";
}
};
std::cout << "::";
}
};
void finish(int) {
}
return x;
}
};
return NULL;
}
};
my_manufacturer,
my_enumerator,
Unused> maker;
return f();
}
};
Less complex than it looks!

152/287
Product x;
std::cout << ", ";
}
};
std::cout << "::";
}
};
void finish(int) {
}
return x;
}
};
return NULL;
}
};
my_manufacturer,
my_enumerator,
Unused> maker;
return f();
}
};
Less complex than it looks!
Can be replaced by existing factories

153/287
Scenarios
 Scenario 2:
– Mirror provides many ready-to-use factories
• From strings (mirror/example/factories/tetrahedron_script.cpp)
• From a SOCI database (mirror/example/factories/tetrahedron_soci.cpp)
// make the input object for the factory
script_factory_input in;
// create a factory plugged with the script parser
script_factory_maker::factory<tetrahedron>::type f = in.data();
// make a tetrahedron object factory
soci_quick_factory_maker::factory<test::tetrahedron>::type
tetrahedron_factory = soci_fact_data(r);
while(r != e) {
test::tetrahedron t = tetrahedron_factory();
// and write it to std output
std::cout << stream::to_json::from(t, [&i](std::ostream& out) {
out << "tetrahedron_" << i; } ) << std::endl;

154/287
Scenarios
 Scenario 3:
namespace cse3009 {
class C {
private:
int i;
static int numberOfInstances;
public:
numberOfInstances++;
}
...
}
static int getNumberOfInstances() {
return numberOfInstances;
}
};
}
– Given a class C

155/287
Scenarios
 Scenario 3:
int cse3009::C::numberOfInstances = 0;
int main(void) {
cse3009::C * o1 = new cse3009::C(42);
cse3009::C * o2 = new cse3009::C(1);
cse3009::C * o3 = new cse3009::C(100);
std::cout << o1 << std::endl << o2 << std::endl << o3 << std::endl;
std::cout << cse3009::C::getNumberOfInstances() << std::endl;
return 0;
}
– Given a class C

156/287
Scenarios
 Scenario 3:
0x800418f8
0x80041908
0x80041918
3
– Given a class C

157/287
Scenarios
 Scenario 3:
0x800418f8
0x80041908
0x80041918
3
– Given a class C
To fulfill this scenario, we
must modify C as in Java

158/287
Scenarios
 Scenario 4:
– Several choices
• Bjarne Stroustrup's Template:
http://www.stroustrup.com/wrapper.pdf
• Herb Sutter's Functor-based Approach:
http://channel9.msdn.com/Shows/Going+Deep/
C-and-Beyond-2012-Herb-Sutter-Concurrency-and-
Parallelism
• Ion Armagedescu’s CPatcher:
http://www.codeproject.com/Articles/34237/A-C-Style-
of-Intercepting-Functions

159/287
Scenarios
 Scenario 4:
– Several choices
Parallelism

160/287
Scenarios
 Scenario 4:
Bjarne Stroustrup's Template:
(Prefix::operator->())
This is o: 42

161/287
Scenarios
 Scenario 4:
int main(void) {
std::cout << "Bjarne Stroustrup's Template:" << std::endl;
cse3009::C o2(42);
Prefix<cse3009::C> prefixed_o2(&o2);
prefixed_o2->foo("This is o: ");
return 0;
}

162/287
Scenarios
 Scenario 4:
template<class T>
class Prefix {
T* p;
public:
Prefix(T * _p) : p(_p) { }
T* operator->() {
std::cout << "(Prefix::operator->())" << std::endl;
return p;
}
};

163/287
Scenarios
 Scenario 4:
– Is similar to a proxy in Java
– Cannot easily access and modify the
arguments of the proxy’ed method call
– Makes it difficult to do something after the
original method call

164/287
Scenarios
 Scenario 4:
– Several choices
Parallelism

165/287
Scenarios
 Scenario 4:
Dependent on the compiler!
GCC v4.8.2
vs.
Microsoft Visual Studio Express 2013
Visual C++ 2013 06157-004-0441005-02428

166/287
Scenarios
 Scenario 4:
– GCC v4.8.2
In file included from ..Sourcescenario4.cpp:32:0:
..Source../CPatcher/patcher.h: In instantiation of 'void
CPatch::HookClassFunctions(T1&, T2, bool, bool) [with T1 =
void (cse3009::C::*)(std::basic_string<char>); T2 = void
(class_for_patches::*)(std::basic_string<char>)]':
..Source../CPatcher/patcher_defines.h:131:2: required
from 'CPatch::CPatch(TReturnVal (TClassSource::*&)(TArg1),
TReturnVal (TClassTarget::*)(TArg1), bool, bool) [with
TClassSource = cse3009::C; TClassTarget =
class_for_patches; TReturnVal = void; TArg1 =
std::basic_string<char>]' ..Sourcescenario4.cpp:52:61:
required from here

167/287
Scenarios
 Scenario 4:
– GCC v4.8.2
may not be legal
• http://www.codeproject.com/Articles/34237/A-C-Style-
of-Intercepting-Functions?msg=4796042
• http://stackoverflow.com/questions/1307278/casting-
between-void-and-a-pointer-to-member-function
*reinterpret_cast<long*>(&fn_funcToHook)
*reinterpret_cast<long*>(&fn_Hook)

168/287
Scenarios
 Scenario 4:
– Microsoft Visual Studio Express 2013
void(ptidej::C::*pfn_foo)(const std::string) = &ptidej::C::foo;
class proxy {
public:
void my_foo(const std::string s) {
std::cout << "Function foo() is patched!" << std::endl << std::flush;
(reinterpret_cast<ptidej::C*>(this)->*pfn_foo)(s);
}
};
int _tmain(int argc, _TCHAR* argv[]) {
CPatch patch_aclass_doSomething(pfn_foo, &proxy::my_foo);
ptidej::C o(42);
o.foo(std::string("This is o: "));
return 0;
}

169/287
Scenarios
 Scenario 4:
– Microsoft Visual Studio Express 2013
Function foo() is patched!
This is o: 42

170/287
Scenarios
 Scenario 4:
– Is similar to a proxy in Java
– Can easily access and modify the arguments
of the proxy’ed method call
– Makes it difficult to do something after the
original method call

172/287
Scenarios
 Scenario 1:
– Given a class C
class C:
def __init__(self, i:int):
self._i = i
def foo(self, s:str):
print(s + str(self._i))
o = C(42)

173/287
Scenarios
 Scenario 1:
– No single way ·
– No complete way ☲
– Methods are attributes that wrap functions
• The Python interpreter checks and calls special methods
on attributes when a program accesses these attributes
– Given a class C
https://softwareengineering.stackexchange.com/questions/206860/why-are-methods-considered-the-class-attributes-in-python

174/287
Scenarios
 Scenario 1:
– Simple way
– Given a class C
print("Identify all the methods available on o")
for name in dir(o):
if callable(getattr(o, name)):
print(name)
print("Invoke a method using its name foo")
thefoo = getattr(o, 'foo’)
thefoo("This is foo: ")

175/287
Scenarios
 Scenario 1:
– Simple way
– Given a class C
__class__
__delattr__
__dir__
…
__init__
__init_subclass__
…
__new__
…
__str__
__subclasshook__
foo
This is foo: 42

176/287
Scenarios
 Scenario 1:
– Simple way, some explanations
– Given a class C
for name in dir(o):
print(name)

177/287
Scenarios
 Scenario 1:
– Given a class C
Returns all the names
in the scope/object
for name in dir(o):
print(name)

178/287
Scenarios
 Scenario 1:
– Given a class C
Returns all the names
in the scope/object
Gets the attribute
with that name in o
for name in dir(o):
print(name)

179/287
Scenarios
 Scenario 1:
– Simple way, a problem
– Given a class C
for name in dir(o):
print(name)

180/287
Scenarios
 Scenario 1:
– Simple way, a problem
– Given a class C
Not completely dynamic
for name in dir(o):
print(name)

181/287
Scenarios
 Scenario 1:
– Some other ways
– Given a class C
class C:
self._i = i
def fooNoArg(self):
print("This is foo")
def fooWithString(self, s:str):
def barNoArg():
print("This is bar")
def barWithString(s:str):
print(s + "42")
o = C(42)

182/287
Scenarios
 Scenario 1:
– Some other ways
• Two questions
– How to obtain a callable attribute?
– How to execute a callable attribute?
– Given a class C

183/287
Scenarios
 Scenario 1:
– Some other ways
• Two questions
– Given a class C

184/287
Scenarios
 Scenario 1:
• Methods
• Functions
– Given a class C

185/287
Scenarios
 Scenario 1:
• Methods
• Functions
– Given a class C
thefoo = getattr(o, "fooNoArg")
thefoo = getattr(o, "fooWithString")

186/287
Scenarios
 Scenario 1:
• Methods
• Functions
– Defined in a class
– Defined in a module
– Given a class C

187/287
Scenarios
 Scenario 1:
• Methods
• Functions
– Given a class C
thefoo = C.fooNoArg
thefoo = C.__dict__['fooNoArg']
thefoo = list(C.__dict__.values())[2]

188/287
Scenarios
 Scenario 1:
• Methods
• Functions
– Given a class C
thebar = barNoArg
thebar = locals()['barNoArg']
tehbar = list(locals().values())[10]

189/287
Scenarios
 Scenario 1:
• Special mentions!
– Given a class C
special = attrgetter("fooWithString")
special(o)("This is foo: ")
special = methodcaller("fooWithString", "This is foo: ")
special(o)
special = compile("""
def barNoArg():
print("This is bar")
barNoArg()""", "<string>", "exec")
exec(special)

190/287
Scenarios
 Scenario 1:
– Some other ways
• Two questions
– Given a class C

191/287
Scenarios
 Scenario 1:
• With a string
• With a method (bound method)
• With a function (unbound method)
– Given a class C

192/287
Scenarios
 Scenario 1:
• With a string
– Given a class C
eval("o.fooNoArg()") # Or exec("o.fooNoArg()")
eval("o.fooWithString('This is foo: ')") # Or exec("o.fooWithString('This is foo: ')")
eval("barNoArg()") # Or exec("barNoArg()")
eval("barWithString('This is bar')") # Or exec("barWithString()")
https://github.com/pwwang/python-varname

193/287
Scenarios
 Scenario 1:
• With a string
– Given a class C
thefoo = getattr(o, "fooNoArg")
thefoo() # Or thefoo.__call__()
thefoo = getattr(o, "fooWithString")
thefoo("This is foo: ") # Or thefoo.__call__("This is foo: ")
# Not applicable to functions

194/287
Scenarios
 Scenario 1:
• With a string
– Given a class C
thefoo = C.fooNoArg
thefoo(o) # Or thefoo.__call__(o)
thefoo = C.fooWithString
thefoo(o, "This is foo: ") # Or thefoo.__call__(o, "This is foo: ")
thebar = barNoArg
thebar() # Or thebar.__call__()
thebar = barWithString
thebar("This is bar: ") # Or thebar.__call__("This is bar: ")

195/287
Scenarios
 Scenario 1:
– No complete way
• The __dict__ and __slots__ attributes are writable
• The method __getattribute__() is “[c]alled uncon-
ditionally to implement attribute accesses” and can
thus hide/provide what it wants
• Only get_referents() may be safe…
– Given a class C
import gc
print(gc.get_referents(C))
print(gc.get_referents(o))
https://stackoverflow.com/questions/42497929/lowlevel-introspection-in-python3
[{'__module__': '__main__', '__init__’: ...]
[42, <class '__main__.C'>]

196/287
Scenarios
 Scenario 2:
o._i = 42 (int)
o._i = 43

197/287
Scenarios
 Scenario 2:
class C:
self._i = i
def foo(self, s:str):
o = C(42)
print("Save on disk the complete state of o")
for name in vars(o):
print("o." + str(name) + " = " + str(getattr(o, name)) +
" (" + type(getattr(o, name)).__name__ + ")")
print("Restore from disk the object o at a later time")
o = C.__new__(C)
# print(o._i) # o doesn't have an attribute _i yet!
setattr(o, "_i", 43)
print(f"o._i = {o._i}")

198/287
Scenarios
 Scenario 2:
– Quite trivial because Python is a dynamic
programming language
– The methods __call__(), __new__(), and
__init__() control instantiation

199/287
Scenarios
 Scenario 3:
class C(....................................):
pass
class D():
pass
x = C()
print(C.getNumberOfInstances())
y = C()
z = C()
x = D()
y = D()
– Given a class C

200/287
Scenarios
 Scenario 3:
class C(....................................):
pass
class D():
pass
x = C()
y = C()
z = C()
x = D()
y = D()
– Given a class C
1
2
3

201/287
Scenarios
 Scenario 3:
class C(....................................):
pass
class D():
pass
x = C()
y = C()
z = C()
x = D()
y = D()
– Given a class C
1
2
3
The magic’s here

202/287
Scenarios
 Scenario 3:
class C(metaclass=ReferenceCountingMetaClass):
pass
class D():
pass
x = C()
y = C()
z = C()
x = D()
y = D()
– Given a class C
The magic’s here

203/287
Scenarios
 Scenario 3:
– Given a class C
class ReferenceCountingMetaClass(type):
def __init__(self, name, bases, namespace):
self._instances = 0
def __call__(self):
newInstance = super().__call__()
self._instances = self._instances + 1
return newInstance
def getNumberOfInstances(self):
return self._instances

204/287
 Scenario 3:
Scenarios
– Given a class C
self._instances = 0
def __call__(self):
newInstance = super().__call__()
self._instances = self._instances + 1
return newInstance
return self._instances
Override the __call__ metaclass instance method
Define the get…() metaclass instance method

205/287
Scenarios
 Scenario 3:
– A method declared in a class applies on the
instances of this class, i.e., objects
– A method declared in a metaclass applies on the
instances of this metaclass, i.e., classes
– Given a class C

206/287
Scenarios
 Scenario 3:
– The overriding of __call__ allows changing the
instantiation of a class because the instantiation
process is actually a method call
(Similar to C++, Java, or Smalltalk, the instantiation could
be done directly using cls.__new__(cls) but without the
automatic initialisation of the instance by __init__())
– Given a class C
x = C()

207/287
Scenarios
 Scenario 4:
– Straightforward because
• Everything is an object in Python, including methods
• Everything can be assigned anything
– Except read-only properties or other special cases

208/287
Scenarios
 Scenario 4:
class C:
def foo(self):
print("This is foo()")
def bar(self, s:str):
print(f"This is bar() with {s}")
o = C()
o.foo()
def foo2():
print("This is the new foo()")
o.foo = foo2
o.foo()
o.bar("1")
def bar2(s:str):
print(f"This is the new bar() with {s}")
o.bar = bar2
o.bar("2")
This is foo()
This is the new foo()
This is bar() with 1
This is the new bar() with 2

209/287
Scenarios
 Scenario 4:
– Special mentions!
• No final keyword in Python
But
• The concept of properties
• The __setattr__() method

210/287
Scenarios
 Scenario 4:
– The concept of properties
class C:
def __init__(self):
self._x = 42
@property
def x(self):
return self._x
o = C()
print(o.x)
o.x = 0
print(o.x)
42
Traceback (most recent call last):
File "...", line 12, in <module>
o.x = 0
^^^
AttributeError: property 'x' of 'C'
object has no setter

211/287
Scenarios
 Scenario 4:
– The __setattr__() method
class C:
def __setattr__(self, attr, value):
if hasattr(self, attr):
raise Exception("Attempting to alter read-only value")
self.__dict__[attr] = value
def foo(self):
print("This is foo()")
o = C()
o.foo()
def foo2():
print("This is the new foo()")
o.foo = foo2
o.foo()
This is foo()
Traceback (most recent call last):
File "...", line 15, in <module>
o.foo = foo2
^^^^^
File "...", line 4, in __setattr_
raise Exception("Attempting…")

212/287
Scenarios Java 



Smalltalk 



C++ 



Python 




213/287
Scenarios Java 



Smalltalk 



C++ 



Python 




214/287
Scenarios Java 



Smalltalk 



C++ 



Python 



Differences in semantics and syntax due to the difference
of runtime environments and of object models

215/287
Outline
 Rationale
 Definition
 Context
 Scenarios
 Theory
 Conclusion
 Appendix

216/287
Theory
 Class
– A programming construct that encapsulates
some state (fields) and behaviours (methods)
– A construct whose instances are objects
 Metaclass
– A programming construct that encapsulates
some state (fields) and behaviours (methods)
– A construct whose instances are classes
Aristote (Ἀριστοτέλης)
*-384 †-322

217/287
Theory
 An object is an instance of a class
– A class generates an object
 A class is an instance of a metaclass
– A metaclass generates a class
 A metaclass is an instance of…

218/287
Theory
(a) A meta-metaclass?

219/287
Theory
(b) Itself?

220/287
Theory
(b) Itself

221/287
Theory
 Typically
– The class Object is the parent of all classes,
including metaclasses
– One given metaclass is the generator of all
classes, including Object

222/287
Theory
 The object model
defines the organisation
among metaclasses,
classes, and objects
– Inheritance vs.
Instantiation
– Multiplicity, reification,
mutability
– …
 The meta-object
protocol (MOP ⧠)
defines the interactions
among metaclasses,
classes, and objects
– Create or delete a
class, method, field
– Create inheritance,
instantiation relations
– …
https://en.wikipedia.org/wiki/The_Art_of_the_Metaobject_Protocol

223/287
https://www.researchgate.net/publication/2827810_Preprocessing_C_Meta-Class_Aspects

224/287
https://www.researchgate.net/publication/246576384_Java_Reflection_in_Action

225/287
https://www.theses.fr/2000MON20072
Inheritance
Instantiation

226/287
https://blog.invisivel.net/2012/04/10/pythons-object-model-explained/

227/287
Object Models
 No, implicit, or explicit metaclass
 Same metaclass for all classes, one
metaclass per class, or different
metaclasses for different classes
 One level, two levels, or any number of
levels of metaclasses (instantiation)
 Mutability of metaclasses

228/287
MOPs ⧠
 Compare
self._instances = {}
def __call__(self, *args, **kwargs):
newInstance = super().__call__(*args, **kwargs)
if self not in self._instances:
self._instances[self] = 0
self._instances[self] = self._instances[self] + 1
return newInstance
return self._instances[self]
class C(metaclass=ReferenceCountingMetaClass):
pass
class C {
private:
int i;
static int numberOfInstances;
public:
numberOfInstances++;
}
...
static int getNumberOfInstances() {
return numberOfInstances;
}
};

229/287
Theory
 The classes
– Class
– Method
– Field
– …
reify constructs of the programming
languages, they become first-class entities,
to allow programs to manipulate these
constructs at runtime

230/287
 Instance variable ∈ C
 Instance method ∈ C
 Class variable ∈ C class
 Class method ∈ C class
 Class instance variables
– To count the numbers of
instances of subclasses of C
(D, E…) independently from
one another
Java
 Field ∈ C
 Method ∈ C
 Static field ∈ C
 Static method ∈ C
 



Smalltalk
Theory

231/287
one another
Smalltalk
Theory

232/287
one another
 Class variable ∈ C
 Class method ∈ C
 Class instance variables ∈ C class
 Class instance methods ∈ C class
 Class class variables ∈ C class
 Class class methods ∈ C class
 Etc.
Smalltalk Python
Theory

233/287
Theory
 Not quite ·
– The decorations
@classmethod and
@staticmethod are
about bindings
– Not about the
receiver / call site
– Not the object model
(i.e., metaclasses)
Python
 Class variable ∈ C
 Class method ∈ C
 Class instance variables ∈ C class
 Class instance methods ∈ C class
 Class class variables ∈ C class
 Class class methods ∈ C class
 Etc.

235/287
Practice
 Java
– public final class Class<T>
 Smalltalk
– One, unique metaclass per class
 C++
– No metaclass, really
 Python
– No overloading
– No class vs. instance methods

236/287
Practice
 In particular in Python
– Cannot have both a class and an instance
__new__() method in the same class
– The class Object defines a static (à la Python)
__new__() method that hide any __new__()
method from a metaclass
https://stackoverflow.com/questions/6760685/what-is-the-best-way-of-implementing-singleton-in-python

237/287
Practice
https://xkcd.com/974/

238/287
Theory
 Earlier, we said that reflection is of
• Libraries
• Frameworks

239/287
Theory
• Libraries
• Frameworks

240/287
Theory
• Libraries
• Frameworks
A virtual machine
is an interpreter

241/287
Theory
 In Java, the metaclass is anonymous and
cannot be modified
 In Smalltalk, “[t]here is only one instance of a
particular Metaclass, namely the class which
is being described”

242/287
Theory
 Many other variations are possible
– LISP
• http://www.cliki.net/mop
– Perl 5, 6
• http://search.cpan.org/perldoc?Class::MOP
– Python
• https://www.python.org/doc/essays/metaclasses/
– MetaclassTalk – Noury Bouraqadi, Pierre Cointe
• http://csl.ensm-douai.fr/MetaclassTalk
– NeoClassTalk – Fred Rivard, Pierre Cointe
• http://c2.com/cgi/wiki?NeoClassTalk
– OpenC++ – Shigeru Chiba, Gregor Kiczales
• http://www.csg.ci.i.u-tokyo.ac.jp/~chiba/openc++.html

243/287
Theory
 A Meta Object Protocol exposes some or all
internal structure of the interpreter to the
programmer
 The MOP is (often) a set of classes and
methods that allow a program to inspect the
state of the supporting system and alter its
behaviour

244/287
Theory
 MOP can be accessible at
– Compile-time
• OpenJava
– http://www.csg.ci.i.u-tokyo.ac.jp/openjava/
– Load-time
• Javassist
– http://www.csg.ci.i.u-tokyo.ac.jp/~chiba/javassist/
– Run-time
• AspectJ
– http://eclipse.org/aspectj/

On Reflection in OO Programming Languages v1.6

On Reflection in OO Programming Languages v1.6

Recommended

Recommended

More Related Content

Similar to On Reflection in OO Programming Languages v1.6

Similar to On Reflection in OO Programming Languages v1.6 (20)

More from Yann-Gaël Guéhéneuc

More from Yann-Gaël Guéhéneuc (20)

Recently uploaded

Recently uploaded (20)

On Reflection in OO Programming Languages v1.6