Following my post on the subject yesterday, a friend and retired Microsoft software architect sent me some additional and incisive commentary on Steve Jurvetson's "Technology Design or Evolution," which appeared July 11 at MIT's Technology Review. The passages from Jurvetson's article appear as indented quotations, followed by the software designer's commentary:
Many of the most interesting problems in computer science, nano-technology, and synthetic biology require the construction of complex systems. But how would we build a really complex system -- such as a general artificial intelligence (AI) that exceeded human intelligence?Some technologists advocate design; others prefer evolutionary search algorithms. Still others would conflate the two, hoping to incorporate the best of both while avoiding their limitations. But while both processes are powerful, they are very different and not easily combined. Rather, they present divergent paths.
>> Any form of evolution is a search across a fitness landscape for a "solution." Computer scientists use evolutionary search when it's easier to describe the attributes of a good answer, designing a solution is beyond their ability (or resources), and they have lots of computational power. We can only design solutions for problems we can understand. If we cannot understand the problem, we cannot design a solution. If, however, we know something of the constraints and control elements (those things that affect the situation - think of them as knobs and dials we can adjust), then we can search for a solution by trying out various combinations of settings.
Designed systems offer predictability, efficiency, and control. Their subsystems are easily understood, which allows their reuse in different contexts. But designed systems also tend to break easily, and they have conquered only simple problems so far. Compare, for example, Microsoft code and biological code: Word is larger than the human genome.
>> This was not written by a software engineer of any repute. First, we regularly design systems whose behavior we cannot map or fully understand. The software on the space shuttle (and any other application where the cost of error is high) is redundant: Multiple versions solve the same problem and compare their answers. Why? Because we can never know if a software program is right. This is known as the Halting Problem: You cannot tell if a program will "halt" without actually running the program. Think of it another way: If we really did understand our software, then the only explanation for "bugs" is that they were intentionally added. A large class of software bugs results from our simply not knowing what a program really does. This is why some designed systems break: We don't understand enough of the problem to design the answer. So, in effect, he argues against himself.
He's also mistaken regarding reuse, the "Holy Grail" of software. While we reuse components, to an extent, we do not really reuse elements like an engineer reuses a #6 screw. Again, complexity is to blame: Everything requires adaptation to its actual purpose.
Lastly, his comparison of Word against the Human genome is meaningless for several reasons. First, what is he counting to make the comparison? Bits? Instructions? Total amount of ink needed to write it all down? A simple thought experiment will tell anyone that the human genome is much more informationally dense than Word (well, Word is kinda dense - but that's another issue): The genome "creates" a human. Word, when it's working, helps create documents. He's using the hidden assumption that the genome is not designed - a position that needs defending.
By contrast, evolved systems demonstrate that simple, iterative algorithms, distributed over time and space, can accumulate design and create complexity that is robust, resilient, and well adapted to its environment. In fact, biological evolution provides the only "existence proof" that an algorithm can produce complexity transcending that of its antecedents. Biological evolution is so inspiring that engineers have mimicked its operations in areas such as genetic programming, artificial life, and the iterative training of neural networks.
>> This is nuts. If biological evolution offers "the only 'existence proof'", then, perhaps, it's not really an existence proof! That is, if every other complex thing around is designed, then maybe, just maybe, this highly functional and really, really complex thing is designed as well. Unless, of course, that option is not allowed on the table, in which case it's not designed and, by definition, offers an 'existence proof' for the assumption just made. (This is when you really wish these folks would take courses in basic logic and critical thinking.)
But evolved systems have their disadvantages. For one, they suffer from "subsystem inscrutability." That is, when we direct the evolution of a system, we may know how the evolutionary process works, but we will not necessarily understand how the resulting system works internally. For example, when the computer scientist Danny Hillis evolved a simple sort algorithm, the process produced inscrutable and mysterious code that did a good job at sorting numbers. But had he taken the time to reverse-engineer his system, the effort would not have provided much generalized insight into evolved artifacts.
>> Yup. Evolutionary searches find "a" solution. It is unlikely to be the optimal solution. It's very, very likely to contain much unwanted material (e.g., extra instructions). The richer the constraint set, and the knobs/dials used when searching, the better the quality... and the longer the search (if it ever ends). If you throw enough paint at a wall, you may get a portrait, but don't expect it to be pretty.
Why is this? Stephen Wolfram's theory of computational equivalence suggests that simple, formulaic shortcuts for understanding evolution may never be discovered. We can only run the iterative algorithm forward to see the results, and the various computational steps cannot be skipped.
>> I'm sorry - Alan Turning saw this problem well before Stephen Wolfram. It's the Halting Problem. Systems, once they exceed a level of complexity, cannot be proven to halt (stop) without actually executing them. This generalizes to say that you can't really know how a complex system behaves without executing the system. Ah, the glory of rediscovered ideas!
Thus, if we evolve a complex system, it is a black box defined by its interfaces. We cannot easily apply our design intuition to the improvement of its inner workings. We can't even partition its subsystems without a serious effort at reverse-engineering. And until we can understand the interfaces between partitions, we can't hope to transfer a subsystem from one evolved complex system to another.
>> He is correct. He also fails to mention that, because we cannot understand the solution, extremely serious bugs may lurk within it. Again, the quality is determined by the richness of the constraint set and those knobs/dials we can adjust during the search. Unless we carefully specify all the relevant constraints and can devise a very rich set of knobs/dials, the solution will likely fail, and fail badly, in some settings - maybe even that for which it was intended. Ask him: Would he fly on a plane whose software was built by an evolutionary algorithm? Know any pilots that would?
A grand engineering challenge therefore remains: can we integrate the evolutionary and design paths to exploit the best of both? Can we transcend human intelligence with an evolutionary algorithm yet maintain an element of control?
>> There is a general problem here: We cannot comprehend complexity beyond a certain level. We conquer complexity through a divide-and-conquer strategy, but, in software, each time we "divide" we add another interface layer that adds its own complexity. Further, interfaces, by definition, do not expose all internal details of the black-box, so, unless the interfaces are designed (oops!) very rigorously and then very carefully implemented, they introduce new locations for errors/bugs. Windows Vista is actually an excellent example here: The complexity of Vista is stretching to the limit Microsoft's ability to complete the project. Make no mistake on this. So far, the only answer we have is divide and conquer by carefully implementing carefully designed interfaces. This will carry us a long ways, but we may reach some interesting breaking points; who knows?
The answer is not yet clear. If we artificially evolve a smart AI, it will be an alien intelligence defined by its sensory interfaces, and understanding its inner workings may require as much effort as we are now expending to explain the human brain.Humans are not the end point of evolution. We are inserting ourselves into the evolutionary process. The next step in the evolutionary hierarchy of abstractions will accelerate the evolution of evolvability itself.
>> His assumption here is quite grand, especially since the only working example of evolution is biology (see his 'existence proof' above). Any time we've evolved something, the results are not impressive. So Danny Hillis evolved a numeric sorting algorithm? Big deal. The books on my shelves list many different, much more efficient numeric sorting algorithms. It's quite a leap of faith to go from evolving a solution to a simple problem (one solved by all undergraduate computer science students) to a truly intelligent artificial agent. Good thing Jurvetson's not religious. ;)
