3. Chips
• What can you do consuming 1 watt?
– 90 Gflops / watt
• 144 core Green Arrays chip
– 70 Gflops / watt
• Adapteva multi-core chip
• Is Strontium Valley the next Silicon Valley?
• 100 trillion graphene field effect transistors
possible on a single chip
7. Memristors are coming from HP
• Flash alternative in 18 months
– Order of magnitude cheaper
• DRAM alternative in 36 months
– 2 orders of magnitude improvement in switching
energy per bit
– Technology allows putting the memory on top of the
processor
• Followed shortly by SRAM alternative
• Samsung has an even larger group working on
this technology
10. Importance of MIMD
• Multiple Instruction – Multiple Data vs SIMD
in most GPUs
• MIMD is very important for many classes of
problem
– Including particularly AI and AGI problems
– Problems where different functions are to be
applied to each data item per cycle
• Traditional separate machines/processes
solutions are not as efficient
12. Mind Computer Interfaces
• Artificial cerebellum
• Extracting movie from brain contents
• BioBolt brain interface to computer
• Implantable computers to restore brain functions
• Protein based transistor for ease of interface
• Memristors to mimic Hebbian learning
• Quasi-liquid memristors
• Future vision of brain co-processors
• Connection to Individual Neurons
The first story is the very sad news of the death of Steve Jobs. We have lost a giant. Pretty much no one accomplished as much to bring computers and software to the people. Brilliant, iconoclastic, domineering, with great gifts at picturing what would be the next big thing and fabulous aesthetic sense. RIP. How many more great minds will we lose before we cry ENOUGH and get very serious about ending aging and curing all diseases including cancers? ""You've got to find what you love. And that is as true for your work as it is for your lovers. Your work is going to fill a large part of your life, and the only way to be truly satisfied is to do what you believe is great work. And the only way to do great work is to love what you do.... Don't settle." - Commencement address, Stanford University, June 12, 2005""Your time is limited, so don't waste it living someone else's life. Don't be trapped by dogma--which is living with the results of other people's thinking. Don't let the noise of others' opinions drown out your own inner voice. And most important, have the courage to follow your heart and intuition. They somehow already know what you truly want to become. Everything else is secondary." - Commencement address, Stanford University, June 12, 2005Some views of his life and impact are at the following links.http://allaboutstevejobs.com/index.htmlhttp://news.cnet.com/8301-13579_3-20117298-37/steve-jobs-book-to-share-his-memorable-quotes/#ixzz1aFqUH0hthttp://www.mercurynews.com/business/ci_19071337http://www.pcworld.com/article/241552/how_steve_jobs_changed_mobility.html
144 core chip that run 90 billion operations per seond per watthttp://www.greenarraychips.com/Multi-computer chips from GreenArrays offer a combination of great computing power, small size, low energy consumption, and high value.GreenArrays, the makers of the GA144 chip, has an OEM partnership with Schmart Board that will allow hobbyists and experimenters to use a chip with 144 cores running about 90 billion instructions per second at a total power of under one watt (typical) while running at full clip and which only uses about 14 microwatts when everything is idle.Adapteva has a multicore chip that can run at 70 gigaFLOPs per watt.
http://www.adapteva.com/index.php?option=com_content&view=article&id=72&Itemid=79• Out-of-the box floating point C programs enables significantly faster time to market and lower development costs• 10-100X advantage in energy efficiency compared to traditional multicore floating point processors• Up to 5 TFLOP sustained effective performance on a single chip, enables a new set of high performance applications• Low latency zero-overhead inter-core communication simplifies parallel programming• Scalable architecture allows code reuse across a wide range of markets and applications
http://nextbigfuture.com/2011/09/rice-hong-kong-polytechnic-physicists.htmlhttp://www.media.rice.edu/media/NewsBot.asp?MODE=VIEW&ID=16217&SnID=1953227986Designed method of synthesizing standing graphenenanoribbons (GNRs)formation on a substrate: (i) Synthesizing transition metal pattern on a substrate using the lithography technology. (ii) Using the transition metal substrate as catalyst to grow a layer of graphene on the patterned metal surface (iii) Cutting the top part of the patterned metal and grown graphene to achieve desired height by ion beam etching. (iv) Etching away transition metal chemically to achieve the predesigned standing GNR pattern with controlled height for device application.To stand a ribbon of graphene upright, it needs diamond on the soles of its shoes. A new paper by collaborators at Rice University and Hong Kong Polytechnic University demonstrates the possibility that tiny strips of graphene -- one-atom-thick sheets of carbon -- can stand tall on a substrate with a little support. This leads to the possibility that arrays of graphene walls could become ultrahigh density components of electronic or spintronic devices.The work was published this month in the online edition of the Journal of the American Chemical Society.Calculations by Rice theoretical physicist Boris Yakobson, Assistant Professor Feng Ding of Hong Kong Polytechnic and their collaborators showed substrates not only of diamond but also nickel could chemically bind the edge of a strip of a graphene nanoribbon. Because the contact is so slight, the graphene walls retain nearly all of their inherent electrical or magnetic properties. And because they're so thin, Yakobson and Ding calculated a theoretical potential of putting 100 trillion graphene wall field-effect transistors (FETs) on a square-centimeter chip.That potential alone may make it possible to blow past the limits implied by Moore's Law -- something Yakobson once discussed with Intel founder Gordon Moore himself.To stand a ribbon of graphene upright, it needs diamond on the soles of its shoes. A new paper by collaborators at Rice University and Hong Kong Polytechnic University demonstrates the possibility that tiny strips of graphene -- one-atom-thick sheets of carbon -- can stand tall on a substrate with a little support. This leads to the possibility that arrays of graphene walls could become ultrahigh density components of electronic or spintronic devices.
Researchers have invented a tiny Etch-A-Sketch® that draws infinitesimally small “wires” on a surface, then erases them. The device works by switching an oxide crystal between insulating and conducting states. The interface between these two materials can be switched between an insulating and metallic state using a sharp conducting probe. Electronic circuits can be “written” and “erased” at scales approaching the distance between atoms (two nanometers). The device, less than four nanometers wide, enables photonic interaction with objects as small as single molecules or quantum dots. Beyond being just plain cool, this device could be the basis of an entirely new kind of transistor. "The number of transistors that can be placed inexpensively on an integrated circuit doubles approximately every two years. At some point, though, this trend has to stop. Materials start acting “weird” when they are made too small. The useful properties of silicon, for example, are believed to break down at distances smaller than 10 nanometers."“The question is, once you’ve pushed silicon to its limit, is there going to be another system to do computation?” asks Levy. That’s really what we’ve been granted funding to explore. We’re trying to break down the major barriers that are potential show-stoppers that would otherwise make it difficult to turn these new types of devices into real, useful things.”Devices that confine and process single electrons represent an important scaling limit of electronics. Such devices have been realized in a variety of materials and exhibit remarkable electronic, optical and spintronic properties. Here, we use an atomic force microscope tip to reversibly ‘sketch’ single-electron transistors by controlling a metal–insulator transition at the interface of two oxides. In these devices, single electrons tunnel resonantly between source and drain electrodes through a conducting oxide island with a diameter of ~1.5 nm. We demonstrate control over the number of electrons on the island using bottom- and side-gate electrodes, and observe hysteresis in electron occupation that is attributed to ferroelectricity within the oxide heterostructure. These single-electron devices may find use as ultradense non-volatile memories, nanoscale hybrid piezoelectric and charge sensors, as well as building blocks in quantum information processing and simulation platforms.
"We’re planning to put a replacement chip on the market to go up against flash within a year and a half," said Williams, "and we also intend to have an SSD replacement available in a year and a half.""In 2014 possibly, or certainly by 2015, we will have a competitor for DRAM and then we’ll replace SRAM."Williams said that the memristor metrics being achieved, in terms of energy to change a bit, read, write time, retention and endurance, were so compelling that the HP-Hynix team now considered flash replacement a done deal. Williams compared HP's resistive RAM technology against flash and claimed to meet or exceed the performance of flash memory in all categories. Read times are less than 10 nanoseconds and write/erase times are about 0.1-ns. HP is still accumulating endurance cycle data at 10^12 cycles and the retention times are measured in years, he said.This creates the prospect of adding dense non-volatile memory as an extra layer on top of logic circuitry. "We could offer 2-Gbytes of memory per core on the processor chip. Putting non-volatile memory on top of the logic chip will buy us twenty years of Moore's Law, said Williams.Further out Williams said the memristor could be used for computation under a scheme called "implication logic" in a fraction of the area taken up in CMOS by Boolean logic. In addition a memristor device is a good analog of the synapse in brain function.One of the best things about the memristor memory is that it is a simple structure made using materials that are already common in the world's wafer fabs making CMOS-compatible devices relatively straight forward, he said.In conclusion Williams stressed that HP would not be getting into the semiconductor components business but would seek to commercialize and then license the technology to all comers.
http://nextbigfuture.com/2011/09/fetram-organic-ferroelectric-material.htmlFETRAM. An Organic Ferroelectric Material Based Novel Random Access Memory CellPurdue University - Researchers are developing a new type of computer memory that could be faster than the existing commercial memory and use far less power than flash memory devices. The technology combines silicon nanowires with a "ferroelectric" polymer, a material that switches polarity when electric fields are applied, making possible a new type of ferroelectric transistor.This diagram shows the layout for a new type of computer memory that could be faster than the existing commercial memory and use far less power than flash memory devices. The technology, called FeTRAM, combines silicon nanowires with a "ferroelectric" polymer, a material that switches polarity when electric fields are applied, making possible a new type of ferroelectric transistor. (Birck Nanotechnology Center, Purdue University)he new technology is called FeTRAM, for ferroelectric transistor random access memory.memory in the commercial market."However, our present device consumes more power because it is still not properly scaled," Das said. "For future generations of FeTRAM technologies one of the main objectives will be to reduce the power dissipation. They might also be much faster than another form of computer memory called SRAM."The FeTRAM technology fulfills the three basic functions of computer memory: to write information, read the information and hold it for a long period of time."You want to hold memory as long as possible, 10 to 20 years, and you should be able to read and write as many times as possible," Das said. "It should also be low power to keep your laptop from getting too hot. And it needs to scale, meaning you can pack many devices into a very small area. The use of silicon nanowires along with this ferroelectric polymer has been motivated by these requirements."The new technology also is compatible with industry manufacturing processes for complementary metal oxide semiconductors, or CMOS, used to produce computer chips. It has the potential to replace conventional memory systems.The FeTRAMs are similar to state-of-the-art ferroelectric random access memories, FeRAMs, which are in commercial use but represent a relatively small part of the overall semiconductor market. Both use ferroelectric material to store information in a nonvolatile fashion, but unlike FeRAMS, the new technology allows for nondestructive readout, meaning information can be read without losing it.This nondestructive readout is possible by storing information using a ferroelectric transistor instead of a capacitor, which is used in conventional FeRAMs.
http://nextbigfuture.com/2011/10/hitachi-has-3d-projector-that-can.htmlHitachi has demonstrated a 3D projector that can project images onto real-world objects. For the demo, a 3D image of a bird hatching was displayed on an artificial egg that was cradled in an artificial nest. But that’s not all. Viewers looking at the demo see the bird as a hologram, in that as the viewing angle changes, so too does the view of the image, just as it would were you to be watching a real bird. When a viewer looking at the image moves up or down or left or right, what they see changes to suit the viewing angle. And if that’s not enough, the projected image can be viewed by multiple people from multiple angles, and everyone sees it as they would were there an actual real-world object sitting there. Also, if the object is moved around a little, the system compensates for it automatically.It is a full parallax 3D display. This technology has been shown before but the new system is improved."Hitachi displays stereoscopic images in real space. They can be overlay stereoscopic views of real objects and can be seen by multiple people without glasses. [Stereoscopic Display Technology has developed]. This technology uses the video information from twenty-four projectors, stereoscopic display technology in real space using a combination of lenses and mirrors produce copies for half of the images. Stereoscopic images can be overlaid with a real object, and without using glasses, so you can see several people at the same time, digital signage is placed where a lot of people back and forth (digital signage ) You can view stereoscopic images that can be applied to more powerful and lower cost of various design validation, and training in skills such as manufacturing and health care can be expected to take advantage of a wide range of areas."
Today'sGPUs are typically SIMD based, they apply a single instruction to multiple data elements per cycle. Many types of graph based problems and processing stacks require multiple (different instructions) be applied per data item (node) per cycle.This is particularly important in many AI and AGI problem spaces. A pdf on the subject is at http://aggregate.org/EXHIBITS/SC09/mogsimlcpc09final.pdf. Ben Goertzel has a good article on the importance to AGI at http://hplusmagazine.com/2011/09/22/mimd-gpus-for-scalable-graph-analysis-the-next-computer-hardware-revolution/Ben thinks that complex graph analysis will drive MIMD forward.
Facebook network graph, indicating a small subnetwork of Facebook’s social network. From http://www.flickr.com/photos/greenem/11696663/"A “graph,” in this context, means a collection of nodes with links going between them (not a coordinate graph). This is something that consumers don’t know much about. However, they rely on it taking place server-side for delivery of the web services they use every day.A graph is the tool used inside computer software to represent a large number of entities that all relate to each other in complex ways. If the relationships are strictly hierarchical in nature, then one can use specialized graphs called “trees,” but most of the time real-world relatedness is more complex than this and one has general, tangled-up graphs (or even more complex beasts called “hypergraphs,” which can be reduced to graphs by the appropriate mathematics)... what I’m calling “graph analysis” very broadly is the asking of queries based on graph structure (“find a subgraph that looks like this;” “find all nodes that are reachable from this particular node via paths of length K with certain properties”), the transformation of graphs into other graphs with more convenient structure, the learning of graphs from other graphs (which is a very general model of inference) and the enablement of dynamical systems on graphs.""Google uses graph analysis to answer search queries and place ads. Facebook uses it to deliver information based on your social network. Government agencies use it to study the vast masses of data they gather about matters of interest around the globe.. Financial institutions will need to use it more and more in future to make decisions in accordance with the increasingly interconnected world. Biomedical scientists will need to use it more and more to discover and analyze therapeutics in the context of complex biological networks.Graph visualization is a powerful tool, but it’s limited to the detailed visualization of small graphs and the rough visualization of large graphs. Graph analysis and search software can do better and can analyze large graphs in detail, but this requires heavy computational firepower as well as sophisticated algorithms. Mass-produced, economical MIMD hardware could help a lot here.""Graph analysis matches naturally to distributed-memory MIMD parallelism because many times one wants each node of a graph to be able to do something different, based on the specific data that it contains. In a full-on MIMD approach, each graph node could get its own processor and memory to use as it wishes. An alternative, nearly as good, is for each processor/memory unit to handle a cluster of interconnected graph nodes. "
http://nextbigfuture.com/2011/09/roundup-of-cybernetics-and-mind.htmlArtificial cerebellumMovies from the brain?http://nextbigfuture.com/2011/09/scientists-use-brain-imaging-to-reveal.htmlProtein based transitorProton-based transistor could let machines communicate with living things and in the future could enable better cybernetics and implants6ShareHuman bodies and all other living things send signals and perform work using ions or protons. Applications in the next decade or so, Rolandi said, would likely be for direct sensing of cells in a laboratory. The current prototype has a silicon base and could not be used in a human body. Longer term, however, a biocompatible version could be implanted directly in living things to monitor, or even control, certain biological processes directly.MimicHebbian Learning in synapse with two memristorshttp://nextbigfuture.com/2010/08/two-memristors-are-needed-to-mimic.htmlOne of the defining features of the connections between neurons is that they become stronger when neurons fire together; hence the phrase "neurons that fire together, wire together", a phenomenon otherwise known as Hebbian learning. Various experiments have shown that this effect is most pronounced early in the learning process, when the increase in connection strength is greatest. Later learning merely reinforces the linksUsing a single memristor to connect two neurons, the memristance decreases when a voltage is applied which increases the current which in turn causes the memristance to drop further, in a kind of positive feedback effect. Using two memristors in series solves the problem according to work by FarnoodMerrikh-Bayat and SaeedBagheriShouraki. Choosing their memristance carefully allows them to reproduce Hebbian-type synapse strengthening more or less exactly. Qausi-liquid memristorshttp://nextbigfuture.com/2010/12/quasi-liquid-memristors-might-make.htmlResearchers at North Carolina State University have demonstrated new "soft" electronic components, built from liquid metals and hydrogels. The scientists hope that such components—quasi-liquid diodes and memristors—will work better than traditional electronics to interface with wet squishy things, such as the human brain.Brain co-processorshttp://nextbigfuture.com/2010/09/machine-coprocessors-for-brain.htmlhttp://www.jfs.tku.edu.tw/14-1/A02.pdf http://repository.upenn.edu/cgi/viewcontent.cgi?article=1037&context=neuroethics_pubsBrain-Computer Interfaces: Past, Present and Futurehttp://nextbigfuture.com/2010/08/brian-litt-and-brain-computer.html
http://nextbigfuture.com/2011/09/scientists-use-brain-imaging-to-reveal.htmlUsing functional Magnetic Resonance Imaging (fMRI) and computational models, UC Berkeley researchers have succeeded in decoding and reconstructing people’s dynamic visual experiences – in this case, watching Hollywood movie trailers.As yet, the technology can only reconstruct movie clips people have already viewed. However, the breakthrough paves the way for reproducing the movies inside our heads that no one else sees, such as dreams and memories, according to researchers.“This is a major leap toward reconstructing internal imagery,” said Professor Jack Gallant, a UC Berkeley neuroscientist and coauthor of the study published online today (Sept. 22) in the journal Current Biology. “We are opening a window into the movies in our minds.”Eventually, practical applications of the technology could include a better understanding of what goes on in the minds of people who cannot communicate verbally, such as stroke victims, coma patients and people with neurodegenerative diseases.It may also lay the groundwork for brain-machine interface so that people with cerebral palsy or paralysis, for example, can guide computers with their minds.However, researchers point out that the technology is decades from allowing users to read others’ thoughts and intentions, as portrayed in such sci-fi classics as “Brainstorm,” in which scientists recorded a person’s sensations so that others could experience themPreviously, Gallant and fellow researchers recorded brain activity in the visual cortex while a subject viewed black-and-white photographs. They then built a computational model that enabled them to predict with overwhelming accuracy which picture the subject was looking at.In their latest experiment, researchers say they have solved a much more difficult problem by actually decoding brain signals generated by moving pictures.
Photo 1: All the components needed to monitor electrical signals from the brain and skeletal muscle - electrodes, sensors, power supply and communications - are mounted on an ultrathin, skin-like membrane. Photo Credit: University of IllinoisPhoto 2: An ultrathin, electronic patch with the mechanics of skin, applied to the wrist for EMG and other measurements (Image: John Rogers)http://nextbigfuture.com/2011/08/wearable-electronics-demonstrate.htmlThe University of California, San Diego has demonstrated that a thin flexible, skin-like device, mounted with tiny electronic components, is capable of acquiring electrical signals from the brain and skeletal muscles and potentially transmitting the information wirelessly to an external computer. The development, published Aug. 12 in the journal Science, means that in the future, patients struggling with reduced motor or brain function, or research subjects, could be monitored in their natural environment outside the lab. For example, a person who struggles with epilepsy could wear the device to monitor for signs of oncoming seizures.
Researchers at the University of Michigan take another step towards the Holy Grail of neurological technology--a device that would restore natural movement to the paralyzed.http://nextbigfuture.com/2011/06/brain-implant-could-revive-paralyzed.htmlA brain implant developed at the University of Michigan uses the body's skin like a conductor to wirelessly transmit the brain's neural signals to control a computer, and may eventually be used to reactivate paralyzed limbs.The implant is called the BioBolt, and unlike other neural interface technologies that establish a connection from the brain to an external device such as a computer, it's minimally invasive and low power.
http://nextbigfuture.com/2011/02/implantable-neurochips-to-restore-brain.htmlContinuous operation of a cortical recurrent BCI leads to long-lasting changes in physiological connections Top: intracranial microstimulation at 3 different motor cortex sites with the monkey at rest evoked 3 different muscle responses (centre) and different isometric torques about the wrist (right). Arrows at right indicate means of 200 ms torque trajectories. Middle: conditioning involved 2 days of triggering microstimuli at site Nstim for every spike recorded at Nrec during free behaviour and sleep. Bottom: after conditioning the output effects evoked from site Nrec had changed to include those from Nstim, an effect that lasted beyond a week. A plausible mechanism is Hebbian strengthening of synaptic connections from Nrec to Nstimhttp://nextbigfuture.com/2011/02/implantable-neurochips-to-restore-brain.htmlDr. Eberhard E. Fetz, UW professor of physiology and biophysics and a core staff researcher at the Washington National Primate Research Center have successfully deployed tiny, battery-powered implantable brain-computer interfaces called neurochips in animals.The neurochip can record nerve cell activity in one part of the brain, process this activity and then stimulate cells in another brain region. The battery-powered device operates continuously during free behavior. When primates carry out their usual daily activities – socializing, climbing, eating, and exploring – their brains can learn to exploit these new resources under normal behavioral conditions.One potential clinical application is to bridge lost biological connections. For example, the researchers have shown that monkeys can learn to bypass an anesthetic block in the nerves of the arm and to activate temporarily paralyzed muscles with activity of cortical neurons. In some ways the device acts as a volition processor, tapping into signals representing the will to move and using them to stimulate the paralyzed muscles to reach targets.“Using an implantable computer interface to implement novel interactions between brain sites opens many fundamentally new research directions,” Fetz said, “depending on the site of recording and stimulation, and how these signals are processed and transformed.”He explained that a second application is to promote neural plasticity, which could strengthen connections and allow some of the brain’s functions to be rescued when impaired. This happens naturally when people recover the ability to move or speak again after a stroke or brain injury. The bidirectional brain computer interface could facilitate this recovery and exploit the brain’s innate talent for re-organizing itself as it heals.“We expect that the recurrent type of brain computer interface we are trying to develop,” he added, “will eventually have numerous clinical applications for bridging damaged biological pathways and strengthening weak neural connections.” For example, signals from the motor-control regions of the brain can be used to stimulate parts of the spinal cord to evoke coordinated movements. This would create connections that could replace lost pathways between the brain and spinal cord, a loss that occurs with strokes and spinal cord injuries.
