Neural Pulsars: Kevin Haywood


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Neural Pulsars: Kevin Haywood

  1. 1. Kevin Haywood Advancing the Design of the Neural Pulsers
  2. 2. Foundations Neural nets are a mainstay of artificial intelligence research, the models that will be used here are derived from the earliest work in the field, the McCulloch-Pitts Cells of 1943.1 Neural nets in general excel at certain types of computation, such as pattern recognition, which are very difficult to perform using other methods. They also possess the unique ability to learn through evolution, so that they become extremely efficient in handling a given task. At their most basic, neural nets are formed by neurons, and the connections between them. Neurons receive input signals from other neurons, and output a signal to other neurons. Each neuron possesses a quality known as its threshold of activation, which is the level of input required to make the neuron output a signal in response. The many possible combinations of inputs, thresholds, and outputs amongst neurons in a network result in a system that is capable of great complexity. In recent years, electronic musical equipment has begun to appear that operate on metaphors of neural networks, and this emergence into my field of expertise has finally produced an environment from which I can make meaningful inquiries. The systems relevant to this proposal are not capable of learning, but they do base their operation on the unique neural net properties of threshold activation, interconnectedness, and signal feedback. The strategies of implementation and degrees of artistic licence vary between designers, and the differences are quite radical between the two examples that are currently implemented in analog hardware: the Resonator Fig. 1 Michaelis Resonator Neuronium, by Jürgen Michaelis,2 and the Wiard Neural Pulsers, by Grant Neuronium, 2005 Richter.3 The Resonator Neuronium (Figure 1) is a complete analog synthesizer that uses neural circuitry to generate all of its sound-shaping functions. Its complex architecture weaves together the traditional disparate building blocks of audio synthesis into a larger symbiotic system, where every element has a degree of influence over every other. In contrast, the Wiard Neural Pulsers (Figure 2) is intended as a single, specialized module in a far larger modular synthesis system. Instead of dealing with smoothly-varying signals, the Neural Pulsers operate with pulses, like in Figure 3 – abrupt transitions in voltage lasting about 10 milliseconds. Fig. 2 Wiard Neural This type of signal is often called a “trigger,” because of its common use in Pulsers, 2003 analog synthesizers as a timing function, triggering the start of events within the system. Therefore, the primary use of the Neural Pulsers is as a multi- part rhythm generator. It also interfaces easily with the many sensor systems that can interpret triggers, such as those commonly used for interactive 10V performance and installation. While the designers of both systems have documented their efforts online, The Neural Pulsers is the much simpler, and hence more approachable circuit. It’s also directly signal-compatible with the modular equipment that I have. Therefore, I chose to prototype and analyze the Neural Pulsers as a beginning for my explorations. What I will propose then is a rethinking of 0V 10mS Richter’s research into the development of experimental hardware neurons for music synthesizers, yeilding a device whose functionality is an order of Fig. 3 A pulse waveform magnitude greater than the previous implementation. 2
  3. 3. The Wiard Neural Pulsers Richter’s Neural Pulsers circuit consists of two distinct EIGHTH QUARTER HALF WHOLE parts: a master timing clock, and the neurons themselves. The clock defines discrete time – the synchronous moments at which the neurons’ states are updated – and simultaneously provides the logic to drive the neurons. Fig. 4 The divided clock outputs Figure 4 shows how the clock is divided, with 4 outputs of the Neural Pulsers. whose divisions correspond to musical whole, half, quarter, and eighth notes. The eighth note output is actually the clock itself, and therefore the smallest unit of time in the circuit. In most cases, at least one of these outputs is patched to the neurons, and usually two or more are used in combination. The use of banana jacks for patching makes it 2 OUT easy to distribute the signal from a single output to multiple inputs, so that any one neuron is capable of influencing all EXC 0 of the others. The use of discrete time in the circuit imposes a unique characteristic, and that is the ever-present logic 1 delay. A neuron that receives input at time t cannot act upon that input until time t + 1. This fundamental property can be exploited, and must always be kept in mind when analyzing INH a patch’s state. Figure 5 details the core of the circuit, which is the 4 identical Fig. 5 The panel layout of a single Wiard McCulloch-Pitts neurons. These exist in the simplest form neuron, shown at actual size. possible, with two equally-weighted excitatory inputs, and a single overriding inhibitory input. Inputs are excitatory if the signals coming into them contribute positively toward reaching the threshold – the level at which the neuron will “fire,” and send a signal from its output. Inhibiting inputs have the opposite effect, reducing the overall input level. To summarize the threshold logic: The value of all inputs are considered simultaneously, and weighting describes the relative influence of one input in • A neuron with threshold 2 (AND) needs two comparison to another. Equally-weighted inputs of the same excitatory signals and no inhibitory signal type are interchangable. The inhibitory input in this design present at time t in order to fire at t + 1. is not equally-weighted to the excitatory inputs, but is said to be overriding because irregardless of what other signals • A neuron with threshold 1 (OR) needs only are present, a signal at this inhibitory input will disable the a single signal, at either of the excitatory neuron’s output at the next clock cycle. inputs, with no inhibitory signal present at time t in order to fire at t + 1. The threshold of a neuron in the Neural Pulsers is set by a 3-position switch to the level of 0, 1, or 2. This arrangement • A neuron with threshold 0 (NOT) will always defines the Boolean logic gates NOT, OR, and AND, fire at time t + 1, unless there was an respectively. A logical TRUE state results if the combination inhibitory signal present at time t. of pulses at the inputs satisfies the selected function, causing the neuron to fire, and output a pulse at the next • This also determines that a threshold 0 clock. This pulse appears at a status indicator LED, and neuron wlll not fire at the first clock cycle at two identical outputs – one banana jack, for patching of a patch’s operation, because the neuron back into the system, and one 3.5mm jack, for driving any always takes a full clock cycle to check its external device that can respond to +10V triggers. inputs before it can process them. 3
  4. 4. The neuron itself is situated between the clock outputs and its I/O graph, with its three input jacks on the left, and the sole output jack on the right. The threshold of the neuron is printed directly upon it, and this is the first thing to note, since it will factor in all remaining calculations. For Figure 7’s neuron to fire at the next clock cycle (neurons always require 1 clock cycle to Understanding the Diagrams to Follow process their inputs), there must be no inhibitory pulse present, and the sum of its excitatory inputs must be A patch is formed by interconnecting any of the available greater than or equal to 1. clocks, inputs, and outputs. The I/O diagrams chart all relevant details of a patch over the course of time. The At step 1, the neuron has 1 inhibitory pulse, and 1 timelines span 17 clock cycles (eighth notes) in order excitatory pulse at its inputs. Because of the inhibitory to show a patch’s initial state, as well as to indicate pulse, the neuron will not fire at the next clock cycle, what happens when the logic settles into a loop. I/O and indeed, there is no output pulse indicated at step points are color-coded, unfilled circles, corresponding 2 of the graph. Step 2 has no inputs at all, and since to the real-world banana jacks that they represent. this neuron needs at least 1 to fire, there will also be Clock and neuron outputs are red, excitatory inputs are no output from the neuron at step 3. At step 3, a single blue, and inhibitory inputs are grey. Connections made excitatory pulse arrives, and unlike step 1, there is no by patch cables are indicated in black. inhibitory pulse present. Therefore, at step 4 the neuron finally fires an output pulse in response! This simple A path extends from every jack onto the I/O graph, with pattern then repeats indefinitely. one exception: the clock ouputs. The clock’s output patterns never change, and so they are assumed In this example, no other neurons were used, but the logic present to reduce clutter. If they were mapped onto the is the same no matter how many are interconnected. grid, they would appear as in Figure 6. A solid circle The essential rule to remember is that, while pulses are indicates each time a signal appears at a jack. The transferred instantaneously from output to input, there clock output jacks will normally be shown unmapped, is always a single clock cycle delay required to process positioned to the left of the neurons, as in Figure 7. inputs into an output pulse. 1 1 1 5 9 13 17 1 5 9 13 17 ½ ½ 1 ¼ ¼ ⅛ ⅛ Fig. 6 Visualization of the clock outputs Fig. 7 A single Wiard neuron driven by two of the clock outputs 4
  5. 5. Working with the Wiard Neural Pulsers 1 5 9 13 17 Despite multiple neurons and patching possibilities, the Wiard module turns out to be a very simplistic A 0 device. As configured, it’s capable of little more than 1 the most basic output sequences. The central role of the clock in rhythm generation makes it nearly ½ B 1 impossible to create irregular patterns. The most ¼ complex output is realized through the techniques of signal duplication, delay, and feedback. C 1 ⅛ Signal duplication has many uses, such as routing D 1 the output of neuron A into both the excitatory input of neuron B and the inhibitory input of neuron C. The firing of A will then make B and C operate very differently at precicely the same moment. Fig. 8 Canonic pulse train of length n, followed by n stages of inactivity, where n is the number of neurons used. Delay is induced by routing a signal into an excitatory input of a neuron with threshold 1, and 1 5 9 13 17 then taking the identical, but one-cycle-behind output. It can be used to unbalance the otherwise static outputs from the clock, when mixed together A 1 through another neuron. 1 ½ B 1 Feedback occurs when a neuron’s output is routed into one of its inputs. As pointed out by Minsky ¼ in his analysis of McCulloch-Pitts, this creates a C 1 1-bit memory, where a neuron’s firing state at time ⅛ t depends upon its firing state at time t - 1.4 D 1 Another of the Neural Pulsers’ strengths is the external output available from each neuron, which makes them capable of driving 4 discrete voices, Fig. 9 Creating an irregular pattern at D by using B as a for example. If delay is implemented serially in delay. each of the 4 neurons, the logical equivalent of a musical canon is produced, as in Figure 8. 5
  6. 6. Advancing the Design of the Neural Pulsers I initially thought that the most significant limitation of the Wiard Neural Pulsers was the small number of neurons used. A simple delay function requires one neuron per stage, and functions of any complexity use up the 4 neural elements quickly. Conversely, it is generous to describe any function available from 4 neurons in this implementation as complex. It seemed that the logical answer was simply more neurons, and I estimate that having 10 or so would yield greater possibilities. Still, I don’t imagine that the brute addition of 6 or more neurons would make this system into more than the sum of its parts – what I imagine instead is more of the same. With 36 faceplate elements already, the Fig. 10 Reconfigured Input physical and financial burdens of this method of enhancement rule it stage, featuring a pair out. of subtracting inhibiting inputs. What I’ve done instead is to reevaluate the architecture of the system. Having read Minsky’s excerpt on McCulloch-Pitts cells, I already knew that other implementations were possible. I thought through several uninspiring alternatives before arriving at my solution, which completely reconfigures the input, threshold, and output stages, leaving behind +2 most of the unneccesary restrictions of Richter’s design. +1 0 -1 The first enhancement is to reconfigure the neuron input stage. The Wiard neurons suffer from having their only inhibitory input be one with total precedence. From this point on, I’ll refer to such overriding behavior as a disabling input, because I see enormous potential in introducing a pair of inhibitory inputs which merely subtract from the input value, Fig. 11 Reconfigured Threshold rather than incapacitate a neuron entirely. Sharing equal weighting with and Output stages, with the existing adding excitatory inputs, the new subtracting inhibitory synchronous discrete inputs would expand the range of incremental change available at the outputs for each input summer. Five inputs configured as in Figure 10 can present the possible threshold. levels {+2, +1, 0, -1, -2} and the logical NOT to the threshold comparator. 6
  7. 7. This is two more levels than the threshold switch is capable of distinguishing, so the switch now becomes a bottleneck, limiting the expanded sensitivity of the new input stage. A switch with more poles could take its place, but the dilemna importantly reveals that there is no benefit in maintaining the existing threshold/output architecture, where a single comparison produces a single output. Figure 11 shows that if instead, each of the 4 possible thresholds {+2, +1, 0, -1} are +2 given synchronous, independent logic outputs, the traditional threshold mechanism can be discarded, leaving an open and far more flexible +1 logic device. The neural structure that results from the above changes 0 is so much more powerful than the original Wiard design, that a single -1 neuron of the proposed format can rival the functionality of the 4 that were previously used. The great amount of I/O points in the proposed neuron require a subsequent expansion of the visual feedback system, to clarify the complex interconnections available. The output LEDs in the Neural Pulsers are perfect at this task, and my implementation will assign one Fig. 12 Proposed neuron core, to every I/O point, in addition to a central clock LED. The status of every with LED status indicators. part of the module can then always be known, making the task of logic Shown at actual size. routing an intiutive one. The final enhancements upgrade the clock, adding 3.5mm jacks for each clock division output, as well as one for an external clock input. These simple additions complete the integration of the neuron with external equipment, allowing the neuron to be synchronized to the timing of another device as either master or slave. I can also be then driven by non-regular pulse patterns generated from elsewhere. The clock rate should also be expanded into the audio range, so that neurons’ pulse outputs can be used as harmonic generators. 1 5 9 13 17 1 Fig. 13 Irregularity. ½ 2 1 0 ¼ -1 ⅛ 1 5 9 13 17 1 Fig. 14 Generating an incremental ½ 2 increase in threshold from 1 0 minimum to maximum, ¼ -1 then resetting and ⅛ repeating. 7
  8. 8. Footnotes 1) McCulloch, W. S. and Pitts, W. “A Logical Calculus of the Ideas Immanent in Nervous Activity” ©1943 2) 3) 4) Minsky, Marvin “Computation: Finite and Infinite Machines” ©1967 Prentice-Hall Excerpted at 8