On Inherent
Complexity of
Computation
Attila Szegedi
@asz
THE FOLLOWING IS INTENDED TO OUTLINE
OUR GENERAL PRODUCT DIRECTION. FOR
INFORMATION PURPOSES ONLY,AND MAY NOT
BE INCORPORA...
I won’t tell you anything
humankind doesn’t already
know for 40 years.
I won’t tell you anything
humankind doesn’t already
know for 40 years.
I won’t tell you anything
humankind doesn’t already
know for 40 years.
This is not a Lisp talk.
I won’t tell you anything
humankind doesn’t already
know for 40 years.
This is not a Lisp talk. No, seriously.
How complex is your
system?
Is it this?
Hardware
Operating system
Application
Configuration
Persistent data
Runtime state
Is it maybe
more like this?
Hardware
Hypervisor
Operating system
Virtual machine
Application server
Application
Plugins
Co...
Hardware
Hypervisor
Operating system
Virtual machine
Application server
Application
Plugins
Configuration
Persistent data
R...
The electricity to heat
part is easy.
100011110001011010100100000101111100000100100010101001010100101000010111111000110111011000110101010
1001000101010010101001...
GET / HTTP/1.1
HTTP/1.1 301 Moved Permanently
Content-Length: 0
Date: Thu, 02 May 2013 09:13:23 GMT
Server: Apache/2.2.17 ...
GET / HTTP/1.1
HTTP/1.1 301 Moved Permanently
Content-Length: 0
Date: Thu, 02 May 2013 09:13:23 GMT
Server: Apache/2.2.17 ...
Hardware
Hypervisor
Operating system
Virtual machine
Application server
Application
Plugins
Configuration
Persistent data
R...
How many layers are
there? Are they
inherent? Are they real?
Hardware
Hypervisor
Operating system
Virtual machine
Application server
Application
Plugins
Configuration
Persistent data
R...
Hardware
Operating system
Virtual machine
Application
A virtual machine takes bytecode
and transforms it into something
re...
Hardware
Virtual machine
Application
Then there are some attempts to
even eliminate the operating system.
Hardware
And some to go all the way down to
BitCoin craze
MHash/J
Generic CPU 0.1-1.14
GPU 2.4
FPGA 25
ASIC 160
https://en.bitcoin.it/wiki/Mining_hardware_comparison
BitCoin craze
MHash/J
Generic CPU 0.1-1.14
GPU 2.4
FPGA 25
ASIC 160
https://en.bitcoin.it/wiki/Mining_hardware_comparison
These are actual products
Hardware
Hypervisor
Operating system
Virtual machine
Application server
Application
Plugins
Configuration
Persistent data
R...
Application
Configuration
In languages where code is first-
class data, you don’t have to
distinguish between the two.
Application =
Configuration
• In Lua, recommended data serialization
format is Lua source code.
• You can just eval() JSON ...
No, seriously, what is
computation?
Computation is a
transition from a more
likely to a less likely
state.
(Decrease in physical entropy; increase in
informat...
Wait what?
• In information theory, entropy is the
measure of uncertainty of state.
• In physics, it’s the amount of missi...
This is very likely This is highly unlikely
This is very likely
•An empty database
•An uninitialized disk
•RAM when power is
turned on
•Quiet network
This is very likely This is highly unlikely
This is highly unlikely
•A populated database
•A populated disk
•RAM after substantial
activity
•Active network
Humans are part of the
computation
• Having an ongoing record of people’s
thoughts and events persisted in a social
networ...
Humans are part of the
computation
Humans are part of the
computation
• Thinking of
boundaries of
identities in
real world
can be quite
fascinating.
Humans are part of the
computation
• Thinking of
boundaries of
identities in
real world
can be quite
fascinating.
• Where ...
Humans are part of the
computation
• Thinking of
boundaries of
identities in
real world
can be quite
fascinating.
• Where ...
Humans are part of the
computation
• Thinking of
boundaries of
identities in
real world
can be quite
fascinating.
• Where ...
Humans are part of the
computation
Humans are part of the
computation
• Where does a computer end?
Humans are part of the
computation
• Where does a computer end?
• Where does a person end?
Humans are part of the
computation
• Where does a computer end?
• Where does a person end?
• Can we influence another
perso...
Humans are part of the
computation
Humans are part of the
computation
• Where does a computer end?
Humans are part of the
computation
• Where does a computer end?
• Where does a person end?
Humans are part of the
computation
• Where does a computer end?
• Where does a person end?
• Can we influence another
perso...
Humans are part of the
computation
Arrow of time
• Transitioning from more likely to less likely
gives us an arrow of time.
• We naturally presume that more ...
Is unzip not a
computation then?
• It creates more likely (decompressed) from
less likely (compressed) data!
• It is still...
Is unzip not a
computation then?
• You started with not having both the
compressed and decompressed data, and
you end up w...
Arrow of time revisited
• Future is a quantum superposition of all
possible successive quantum states.
• Past, too, is a q...
This seemed like a
good time for this slide
Thermodynamics in the
way
• Any physical process that dissipates heat
will be irreversible.
• It’s possible to go:
• fast,...
Thermodynamics in the
way
• e x t = const
• e := energy dissipation in form of heat
• t := time to operate an electric cir...
Why does the arrow of
time matter anyway?
Here’s why
Here’s why
A
B
A NAND B
Here’s why
A
B
A NAND B
Direction
Reversible gate
A
B
A NAND B
Direction
reversing bit
Reversible gate
A
B
A NAND B
Direction
reversing bit
Meandering. Not caring
which way it flows.
Running analogy
Tempo
Endurance Distance
Running analogy
Speed
Power
consumption
Battery life
How fast can we go?
How fast can we go?
• I couldn’t figure out hard data for this…
How fast can we go?
• I couldn’t figure out hard data for this…
• Theoretical upper limit: divide highest power
density wit...
How fast can we go?
• I couldn’t figure out hard data for this…
• Theoretical upper limit: divide highest power
density wit...
How fast can we go?
• I couldn’t figure out hard data for this…
• Theoretical upper limit: divide highest power
density wit...
How fast can we go?
• I couldn’t figure out hard data for this…
• Theoretical upper limit: divide highest power
density wit...
Where are we?
• We could go much hotter and faster.
• Insert science fiction of atomic particle
machines that can run few h...
100011110001011010100100000101111100000100100010101001010100101000010111111000110111011000110101010
1001000101010010101001...
Digital computation
• Binary inputs go in, binary output goes out
• Not all of the input precedes the output
(think stream...
Digital computation
• A particular computation (or “program”) is
mapping of all possible inputs to all possible
outputs.
“” -> “10001111000101101010010000010111110000010010001010100101010010100”
“0” -> “0100010101100101110110110101101”
“1” -> ...
Kolmogorov
complexity
• Defined for strings.
• The length of a program that produces the given
string.
• The programming la...
Inherent complexity of
a computation
Inherent complexity of
a computation
Inherent complexity of
a computation
• Inherent complexity of the computation:
the length of the most compact form that
ca...
Inherent complexity of
a computation
• Inherent complexity of the computation:
the length of the most compact form that
ca...
• Oh, wait, Kolmogorov complexity is a non-
computable function.
• Oh, wait, Kolmogorov complexity is a non-
computable function.
• Oh, wait, Kolmogorov complexity is a non-
computable function.
• Consequence: no matter
how good your optimizing
compile...
• Oh, wait, Kolmogorov complexity is a non-
computable function.
• Consequence: no matter
how good your optimizing
compile...
“” ->
100100010101001010100101000010111111000110111011000110000011001111000011000110000010011011101111111
0110110010111001...
• Sometimes, part of the decoder is external.
• Sometimes, part of the decoder is external.
• It can be in your head.
• Sometimes, part of the decoder is external.
• It can be in your head.
• Again, human is part of the
computation.
Complex look is sometimes a simple pattern
Complex look is sometimes a simple pattern
Complex look is sometimes a simple pattern
SzegediButterfly1 {
init:
  z = #pixel
loop:
  float x = real(z)
  float y = im...
Complex look is sometimes a simple pattern
Complex look is sometimes a simple pattern
Complex look is sometimes a simple pattern
SzegediButterfly2 {
init:
  z = #pixel
loop:
  float x = real(z)
  float y = im...
Even if you could find
the most compact
form…
• … you probably wouldn’t want to use it.
• It would be even less humanly
mai...
You can always reduce
complexity
• You always have excess complexity in your
system.
• It’s worth thinking of how to reduc...
Artificial boundaries
hobble you
• Favor environments that don’t force you
into artificial layer boundaries. Code is
data, d...
Take aways
• Be sensible with the effort and the result.
• Don’t overshoot into compact-but-
unreadable territory.
• Hitti...
Image Credits
Snowy branches: http://www.flickr.com/photos/aeioux/2398264997/
XKCD comic: http://xkcd.com/297/
Complex gizm...
On Inherent Complexity of Computation, by Attila Szegedi
On Inherent Complexity of Computation, by Attila Szegedi
On Inherent Complexity of Computation, by Attila Szegedi
On Inherent Complexity of Computation, by Attila Szegedi
On Inherent Complexity of Computation, by Attila Szegedi
On Inherent Complexity of Computation, by Attila Szegedi
On Inherent Complexity of Computation, by Attila Szegedi
Upcoming SlideShare
Loading in …5
×

On Inherent Complexity of Computation, by Attila Szegedi

1,085 views

Published on

The system you just recently deployed is likely an application processing some data, likely relying on some configuration, maybe using some plugins, certainly relying on some libraries, using services of an operating system running on some physical hardware. The previous sentence names 7 categories into which we compartmentalise various parts of a computation process that’s in the end going on in a physical world. Where do you draw the line of functionality between categories? From what vantage points do these distinctions become blurry? Finally, how does it all interact with the actual physical world in which the computation takes place? (What is the necessary physical minimum required to perform a computation, anyway?) Let’s make a journey from your AOP-assembled, plugin-injected, YAML-configured, JIT compiled, Hotspot-executed, Linux-on-x86 hosted Java application server talking JSON-over-HTTP-over-TCP-over-IP-over-Ethernet all the way down to electrons. And then back. Recorded at GeekOut 2013.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,085
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
12
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

On Inherent Complexity of Computation, by Attila Szegedi

  1. 1. On Inherent Complexity of Computation Attila Szegedi @asz
  2. 2. THE FOLLOWING IS INTENDED TO OUTLINE OUR GENERAL PRODUCT DIRECTION. FOR INFORMATION PURPOSES ONLY,AND MAY NOT BE INCORPORATED INTO ANY CONTRACT. IT IS NOT A COMMITMENT TO DELIVER ANY MATERIAL, CODE, OR FUNCTIONALITY,AND SHOULD NOT BE RELIED UPON IN MAKING PURCHASING DECISION.THE DEVELOPMENT, RELEASE,AND TIMING OF ANY FEATURES OR FUNCTIONALITY DESCRIBED FOR ORACLE'S PRODUCTS REMAINS AT THE SOLE DISCRETION OF ORACLE.
  3. 3. I won’t tell you anything humankind doesn’t already know for 40 years.
  4. 4. I won’t tell you anything humankind doesn’t already know for 40 years.
  5. 5. I won’t tell you anything humankind doesn’t already know for 40 years. This is not a Lisp talk.
  6. 6. I won’t tell you anything humankind doesn’t already know for 40 years. This is not a Lisp talk. No, seriously.
  7. 7. How complex is your system?
  8. 8. Is it this? Hardware Operating system Application Configuration Persistent data Runtime state
  9. 9. Is it maybe more like this? Hardware Hypervisor Operating system Virtual machine Application server Application Plugins Configuration Persistent data Runtime state
  10. 10. Hardware Hypervisor Operating system Virtual machine Application server Application Plugins Configuration Persistent data Runtime state Electricity Electric impulses on a wire Heat
  11. 11. The electricity to heat part is easy.
  12. 12. 100011110001011010100100000101111100000100100010101001010100101000010111111000110111011000110101010 100100010101001010100101000010111111000110111011000110000011001111000011000110000010011011101111111 011011001011100100100000101000011001011110010110110111000011101110110010111011101110100110110011110 011010100001111011111101110111010011001011101110111010010110110011001100101110111011001111110100110 100011101010000011000010101000100110000111101001100101111010100000101010011010001110101101100100000 110000110010100000100110111000011111001100000110010110000110001110011100000110000111001111010110001 110011111010110010110011100000100011110011011010100101010100111100101111001011101101100101111001011 101010000010000011110000110000111000111101000110010110111111001010111011001010111011000111011110000 010100010101011100010111010111011101110100111010110100110101011000101101101000011011111110111110010 111100101100101110010010110110000101111001111010100000101000010010001010000101111110101101110110011 101110110101101101110001111010111000101110101110111011101001110101110111101110110001110001101010110 001011011010000110100111011101100111110001011000011100011110101111101010000011010001110100111010011 100001110101011111011111100111110010111001011101011110111111101011110100101110110010111001011011111 111000110110111011001110010111000011000111011101110000110100011100001010100110011011111100011110000 111101001101001110111111011101110101000001101000111010011101001110000111010101111101111110011111001 011100101110101111011111110101111010010111011001011100101101111101010000111100001110001111010001100 101101101100001111011111101110111010011100101101111110110011101010000011011011100001111100010110111 000011100111110010111110111001111000011000010101000101111100011100001101001111001011001011110011111 010100000101010011010001110101101100100000110000110010100000100110111000011111001100000110010110000 110001110011100000110000111001111010110001111000111010110010110011100000100011110011011010100101010 101101100001111001011110011110101000001000001110001111000111100101111000011101001011011000101110111 011000111101111110010011010011101110110011110101000011110111111011101101110110010111000111110100110 100111011111101110111010100000110001111011001101111111001111001011010100001111011111101110111010011 001011101110111010010110110101001111001111000011001011110101000001110100110010111110001110100101111 110100011101001101101110110011101110000011000111101000110000111100101110011110010111101001111011010 1011010100100011010110111100010101010
  13. 13. GET / HTTP/1.1 HTTP/1.1 301 Moved Permanently Content-Length: 0 Date: Thu, 02 May 2013 09:13:23 GMT Server: Apache/2.2.17 (Ubuntu) X-Powered-By: PHP/5.3.5-1ubuntu7.11 X-Pingback: http://geekout.ee/xmlrpc.php Location: http://geekout.ee/ Cache-Control: max-age=300 Expires: Thu, 02 May 2013 09:18:23 GMT Vary: Accept-Encoding Connection: close Content-Type: text/html; charset=UTF-8
  14. 14. GET / HTTP/1.1 HTTP/1.1 301 Moved Permanently Content-Length: 0 Date: Thu, 02 May 2013 09:13:23 GMT Server: Apache/2.2.17 (Ubuntu) X-Powered-By: PHP/5.3.5-1ubuntu7.11 X-Pingback: http://geekout.ee/xmlrpc.php Location: http://geekout.ee/ Cache-Control: max-age=300 Expires: Thu, 02 May 2013 09:18:23 GMT Vary: Accept-Encoding Connection: close Content-Type: text/html; charset=UTF-8 Hardware Hypervisor Operating system Virtual machine Application server Application Plugins Configuration Persistent data Runtime state
  15. 15. Hardware Hypervisor Operating system Virtual machine Application server Application Plugins Configuration Persistent data Runtime state Complexity Abstraction
  16. 16. How many layers are there? Are they inherent? Are they real?
  17. 17. Hardware Hypervisor Operating system Virtual machine Application server Application Plugins Configuration Persistent data Runtime state
  18. 18. Hardware Operating system Virtual machine Application A virtual machine takes bytecode and transforms it into something resembling a native application on the OS/hardware.
  19. 19. Hardware Virtual machine Application Then there are some attempts to even eliminate the operating system.
  20. 20. Hardware And some to go all the way down to
  21. 21. BitCoin craze MHash/J Generic CPU 0.1-1.14 GPU 2.4 FPGA 25 ASIC 160 https://en.bitcoin.it/wiki/Mining_hardware_comparison
  22. 22. BitCoin craze MHash/J Generic CPU 0.1-1.14 GPU 2.4 FPGA 25 ASIC 160 https://en.bitcoin.it/wiki/Mining_hardware_comparison
  23. 23. These are actual products
  24. 24. Hardware Hypervisor Operating system Virtual machine Application server Application Plugins Configuration Persistent data Runtime state
  25. 25. Application Configuration In languages where code is first- class data, you don’t have to distinguish between the two.
  26. 26. Application = Configuration • In Lua, recommended data serialization format is Lua source code. • You can just eval() JSON in JavaScript. • Lisp.‘nuff said. • Standard disclaimer about having your program execute unverified external code.
  27. 27. No, seriously, what is computation?
  28. 28. Computation is a transition from a more likely to a less likely state. (Decrease in physical entropy; increase in information entropy.)
  29. 29. Wait what? • In information theory, entropy is the measure of uncertainty of state. • In physics, it’s the amount of missing information to precisely describe the state of the system. • The two usages of the term are therefore exactly opposite. • Way to troll each other, fellow scientists!
  30. 30. This is very likely This is highly unlikely
  31. 31. This is very likely •An empty database •An uninitialized disk •RAM when power is turned on •Quiet network
  32. 32. This is very likely This is highly unlikely
  33. 33. This is highly unlikely •A populated database •A populated disk •RAM after substantial activity •Active network
  34. 34. Humans are part of the computation • Having an ongoing record of people’s thoughts and events persisted in a social network database is less likely than not having them. • Just think about the combined effort of creating Twitter/Facebook/etc databases • both people building and operating it, and • people pouring data into it.
  35. 35. Humans are part of the computation
  36. 36. Humans are part of the computation • Thinking of boundaries of identities in real world can be quite fascinating.
  37. 37. Humans are part of the computation • Thinking of boundaries of identities in real world can be quite fascinating. • Where does a computer end?
  38. 38. Humans are part of the computation • Thinking of boundaries of identities in real world can be quite fascinating. • Where does a computer end? • Where does a person end?
  39. 39. Humans are part of the computation • Thinking of boundaries of identities in real world can be quite fascinating. • Where does a computer end? • Where does a person end? • Can we influence another person in such a way that part of us actually lives in them?
  40. 40. Humans are part of the computation
  41. 41. Humans are part of the computation • Where does a computer end?
  42. 42. Humans are part of the computation • Where does a computer end? • Where does a person end?
  43. 43. Humans are part of the computation • Where does a computer end? • Where does a person end? • Can we influence another person in such a way that part of us actually lives in them?
  44. 44. Humans are part of the computation
  45. 45. Humans are part of the computation • Where does a computer end?
  46. 46. Humans are part of the computation • Where does a computer end? • Where does a person end?
  47. 47. Humans are part of the computation • Where does a computer end? • Where does a person end? • Can we influence another person in such a way that part of us actually lives in them?
  48. 48. Humans are part of the computation
  49. 49. Arrow of time • Transitioning from more likely to less likely gives us an arrow of time. • We naturally presume that more likely states precede less likely ones. • Opposite cases happen, and they are associated with idea of destruction.
  50. 50. Is unzip not a computation then? • It creates more likely (decompressed) from less likely (compressed) data! • It is still a computation, though.
  51. 51. Is unzip not a computation then? • You started with not having both the compressed and decompressed data, and you end up with both. • That’s less likely than just having the compressed data.
  52. 52. Arrow of time revisited • Future is a quantum superposition of all possible successive quantum states. • Past, too, is a quantum superposition of all possible preceding quantum states. • See Wheeler’s Delayed Choice Experiment.
  53. 53. This seemed like a good time for this slide
  54. 54. Thermodynamics in the way • Any physical process that dissipates heat will be irreversible. • It’s possible to go: • fast, hot and irreversible, or • slow, cool, and reversible. • (And any shade of gray in between.)
  55. 55. Thermodynamics in the way • e x t = const • e := energy dissipation in form of heat • t := time to operate an electric circuit • Going slow conserves energy
  56. 56. Why does the arrow of time matter anyway?
  57. 57. Here’s why
  58. 58. Here’s why A B A NAND B
  59. 59. Here’s why A B A NAND B Direction
  60. 60. Reversible gate A B A NAND B Direction reversing bit
  61. 61. Reversible gate A B A NAND B Direction reversing bit
  62. 62. Meandering. Not caring which way it flows.
  63. 63. Running analogy Tempo Endurance Distance
  64. 64. Running analogy Speed Power consumption Battery life
  65. 65. How fast can we go?
  66. 66. How fast can we go? • I couldn’t figure out hard data for this…
  67. 67. How fast can we go? • I couldn’t figure out hard data for this… • Theoretical upper limit: divide highest power density with lowest energy for a physical state transition; you get “operations/m3/s”.
  68. 68. How fast can we go? • I couldn’t figure out hard data for this… • Theoretical upper limit: divide highest power density with lowest energy for a physical state transition; you get “operations/m3/s”. • Problem with:
  69. 69. How fast can we go? • I couldn’t figure out hard data for this… • Theoretical upper limit: divide highest power density with lowest energy for a physical state transition; you get “operations/m3/s”. • Problem with: • relativistic effects
  70. 70. How fast can we go? • I couldn’t figure out hard data for this… • Theoretical upper limit: divide highest power density with lowest energy for a physical state transition; you get “operations/m3/s”. • Problem with: • relativistic effects • controlling those amounts of energy output
  71. 71. Where are we? • We could go much hotter and faster. • Insert science fiction of atomic particle machines that can run few hours of subjective human brain simulation in few femtoseconds before heat-disintegrating. • We could also go much slower and cooler • Ain’t nobody got time for that!
  72. 72. 100011110001011010100100000101111100000100100010101001010100101000010111111000110111011000110101010 100100010101001010100101000010111111000110111011000110000011001111000011000110000010011011101111111 011011001011100100100000101000011001011110010110110111000011101110110010111011101110100110110011110 011010100001111011111101110111010011001011101110111010010110110011001100101110111011001111110100110 100011101010000011000010101000100110000111101001100101111010100000101010011010001110101101100100000 110000110010100000100110111000011111001100000110010110000110001110011100000110000111001111010110001 110011111010110010110011100000100011110011011010100101010100111100101111001011101101100101111001011 101010000010000011110000110000111000111101000110010110111111001010111011001010111011000111011110000 010100010101011100010111010111011101110100111010110100110101011000101101101000011011111110111110010 111100101100101110010010110110000101111001111010100000101000010010001010000101111110101101110110011 101110110101101101110001111010111000101110101110111011101001110101110111101110110001110001101010110 001011011010000110100111011101100111110001011000011100011110101111101010000011010001110100111010011 100001110101011111011111100111110010111001011101011110111111101011110100101110110010111001011011111 111000110110111011001110010111000011000111011101110000110100011100001010100110011011111100011110000 111101001101001110111111011101110101000001101000111010011101001110000111010101111101111110011111001 011100101110101111011111110101111010010111011001011100101101111101010000111100001110001111010001100 101101101100001111011111101110111010011100101101111110110011101010000011011011100001111100010110111 000011100111110010111110111001111000011000010101000101111100011100001101001111001011001011110011111 010100000101010011010001110101101100100000110000110010100000100110111000011111001100000110010110000 110001110011100000110000111001111010110001111000111010110010110011100000100011110011011010100101010 101101100001111001011110011110101000001000001110001111000111100101111000011101001011011000101110111 011000111101111110010011010011101110110011110101000011110111111011101101110110010111000111110100110 100111011111101110111010100000110001111011001101111111001111001011010100001111011111101110111010011 001011101110111010010110110101001111001111000011001011110101000001110100110010111110001110100101111 110100011101001101101110110011101110000011000111101000110000111100101110011110010111101001111011010 1011010100100011010110111100010101010 This is digital computation
  73. 73. Digital computation • Binary inputs go in, binary output goes out • Not all of the input precedes the output (think streams). • Later input can be shaped by earlier output (client resubmits a cookie received from server)
  74. 74. Digital computation • A particular computation (or “program”) is mapping of all possible inputs to all possible outputs.
  75. 75. “” -> “10001111000101101010010000010111110000010010001010100101010010100” “0” -> “0100010101100101110110110101101” “1” -> “010001010110100000010001010111110010101000011100101” “01” -> “111100010110101001000001011111000001001000101010010101001010” ... • We don’t write programs this way. Right? • We recognize patterns and map classes of inputs to classes of outputs. • It’s basically a compressed representation. • If the number of patterns is not finite, you’re describing a random process.
  76. 76. Kolmogorov complexity • Defined for strings. • The length of a program that produces the given string. • The programming language doesn’t matter • ‘cause you can just prefix your program with an interpreter written in another language, for a constant difference. • Can also apply to infinite sequences, such as our “enumerate all programs”
  77. 77. Inherent complexity of a computation
  78. 78. Inherent complexity of a computation
  79. 79. Inherent complexity of a computation • Inherent complexity of the computation: the length of the most compact form that can encode its full input-output mapping.
  80. 80. Inherent complexity of a computation • Inherent complexity of the computation: the length of the most compact form that can encode its full input-output mapping. • Incidentally, a string is “random” if it’s shorter than any program that can generate it.
  81. 81. • Oh, wait, Kolmogorov complexity is a non- computable function.
  82. 82. • Oh, wait, Kolmogorov complexity is a non- computable function.
  83. 83. • Oh, wait, Kolmogorov complexity is a non- computable function. • Consequence: no matter how good your optimizing compiler is, you can never prove that there’s not an even better one.
  84. 84. • Oh, wait, Kolmogorov complexity is a non- computable function. • Consequence: no matter how good your optimizing compiler is, you can never prove that there’s not an even better one. • a.k.a “Full Employment Theorem” for compiler writers.
  85. 85. “” -> 100100010101001010100101000010111111000110111011000110000011001111000011000110000010011011101111111 011011001011100100100000101000011001011110010110110111000011101110110010111011101110100110110011110 011010100001111011111101110111010011001011101110111010010110110011001100101110111011001111110100110 100011101010000011000010101000100110000111101001100101111010100000101010011010001110101101100100000 110000110010100000100110111000011111001100000110010110000110001110011100000110000111001111010110001 110011111010110010110011100000100011110011011010100101010100111100101111001011101101100101111001011 101010000010000011110000110000111000111101000110010110111111001010111011001010111011000111011110000 010100010101011100010111010111011101110100111010110100110101011000101101101000011011111110111110010 111100101100101110010010110110000101111001111010100000101000010010001010000101111110101101110110011 101110110101101101110001111010111000101110101110111011101001110101110111101110110001110001101010110 001011011010000110100111011101100111110001011000011100011110101111101010000011010001110100111010011 100001110101011111011111100111110010111001011101011110111111101011110100101110110010111001011011111 111000110110111011001110010111000011000111011101110000110100011100001010100110011011111100011110000 111101001101001110111111011101110101000001101000111010011101001110000111010101111101111110011111001 011100101110101111011111110101111010010111011001011100101101111101010000111100001110001111010001100 101101101100001111011111101110111010011100101101111110110011101010000011011011100001111100010110111 000011100111110010111110111001111000011000010101000101111100011100001101001111001011001011110011111 010100000101010011010001110101101100100000110000110010100000100110111000011111001100000110010110000 110001110011100000110000111001111010110001111000111010110010110011100000100011110011011010100101010 101101100001111001011110011110101000001000001110001111000111100101111000011101001011011000101110111 011000111101111110010011010011101110110011110101000011110111111011101101110110010111000111110100110 100111011111101110111010100000110001111011001101111111001111001011010100001111011111101110111010011 001011101110111010010110110101001111001111000011001011110101000001110100110010111110001110100101111 110100011101001101101110110011101110000011000111101000110000111100101110011110010111101001111011010 1011010100100011010110111100010101010 You can’t cheat complexity(input) + complexity(decoder)
  86. 86. • Sometimes, part of the decoder is external.
  87. 87. • Sometimes, part of the decoder is external. • It can be in your head.
  88. 88. • Sometimes, part of the decoder is external. • It can be in your head. • Again, human is part of the computation.
  89. 89. Complex look is sometimes a simple pattern
  90. 90. Complex look is sometimes a simple pattern
  91. 91. Complex look is sometimes a simple pattern SzegediButterfly1 { init:   z = #pixel loop:   float x = real(z)   float y = imag(z)   z = sqr(y) - sqrt(abs(x)) +             (sqr(x) - sqrt(abs(y))) * 1i + #pixel bailout:   |z| <= @Bailout default:   param Bailout     caption = "Bailout Value"     default = 127.0   endparam }
  92. 92. Complex look is sometimes a simple pattern
  93. 93. Complex look is sometimes a simple pattern
  94. 94. Complex look is sometimes a simple pattern SzegediButterfly2 { init:   z = #pixel loop:   float x = real(z)   float y = imag(z)   z = sqr(x) - sqrt(abs(y)) +            (sqr(y) - sqrt(abs(x))) * 1i + #pixel bailout:   |z| <= @Bailout default:   param Bailout     caption = "Bailout Value"     default = 127.0   endparam }
  95. 95. Even if you could find the most compact form… • … you probably wouldn’t want to use it. • It would be even less humanly maintainable than flipping bits in a zipped source file. • Ever seen 256-byte 8-bit CPU demos?
  96. 96. You can always reduce complexity • You always have excess complexity in your system. • It’s worth thinking of how to reduce it. • Pattern recognition (refactoring). • Favor environments that don’t force you into artificial layer boundaries. Code is data, data is code. • Patterns can be more overarching without artificial boundaries.
  97. 97. Artificial boundaries hobble you • Favor environments that don’t force you into artificial layer boundaries. Code is data, data is code. • Patterns can be more overarching without artificial boundaries. • Java program loads an XML file containing a flow graph and some embedded JavaScript? • You have a problem.
  98. 98. Take aways • Be sensible with the effort and the result. • Don’t overshoot into compact-but- unreadable territory. • Hitting on the minimally complex representation is as likely as inventing the perpetuum mobile. (Thermodynamics metaphor again.)
  99. 99. Image Credits Snowy branches: http://www.flickr.com/photos/aeioux/2398264997/ XKCD comic: http://xkcd.com/297/ Complex gizmo: http://www.flickr.com/photos/michaelheiss/2871996129/ Electric heater: http://www.flickr.com/photos/jocelynb/426268348/ Layer cake: http://www.flickr.com/photos/julessilver/3259734572 Abacus: http://www.flickr.com/photos/skidder/37675092 Airplanes: http://www.flickr.com/photos/good_day/198611998/ Meandering: http://www.flickr.com/photos/31856336@N03/7831522814/ Headphones: http://www.flickr.com/photos/doremigirl/8362322435 Kolmogorov Lecture: http://en.wikipedia.org/wiki/File:Kolm_complexity_lect.jpg All used images are Creative Commons licensed, and used according to their terms of license.

×