Dynamics of Complex Systems
Studies in Nonlinearity
Series Editor: Robert L. Devaney
Ralph Abraham, Dynamics: The Geometry of Behavior
Ralph H. Abraham and Christopher D. Shaw, Dynamics: The Geometry of
Behavior
Robert L. Devaney, Chaos, Fractals, and Dynamics: Computer Experiments
in Mathematics
Robert L. Devaney, A First Course in Chaotic Dynamical Systems: Theory
and Experiment
Robert L. Devaney, An Introduction to Chaotic Dynamical Systems, Second
Edition
Robert L. Devaney, James F. Georges, Delbert L. Johnson, Chaotic
Dynamical Systems Software
Gerald A. Edgar (ed.), Classics on Fractals
James Georges, Del Johnson, and Robert L. Devaney, Dynamical Systems
Software
Michael McGuire, An Eye for Fractals
Steven H. Strogatz, Nonlinear Dynamics and Chaos: With Applications to
Physics, Biology, Chemistry, and Engineering
Nicholas B. Tufillaro, Tyler Abbott, and Jeremiah Reilly, An Experimental
Approach to Nonlinear Dynamics and Chaos
Yaneer Bar-Yam
Dynamics of
Complex Systems
The Advanced Book Program
Addison-Wesley
Reading,Massachusetts
s
tt
Figure 2.4.1 ©1992 Benjamin Cummings, from E. N. Marieb/Human Anatomy and
Physiology. Used with permission.
Figure 7.1.1 (bottom) by Brad Smith, Elwood Linney, and the Center for In Vivo Microscopy
at Duke University (A National Center for Research Resources, NIH). Used with permission.
Many of the designations used by manufacturers and sellers to distinguish their products are
claimed as trademarks.Where those designations appear in this book and Addison-Wesley
was aware of a trademark claim, the designations have been printed in initial capital letters.
Library of Congress Cataloging-in-Publication Data
Bar-Yam,Yaneer.
Dynamics of complex systems / Yaneer Bar-Yam.
p. cm.
Includes index.
ISBN 0-201-55748-7
1. Biomathematics. 2. System theory. I. Title.
QH323.5.B358 1997
570'.15' 1—DC21 96-52033
CIP
Copyright © 1997 by Yaneer Bar-Yam
All rights reserved. No part of this publication may be reproduced, stored in a retrieval system,
or transmitted, in any form or by any means, electronic, mechanical, photocopying,
recording, or otherwise, without the prior written permission of the publisher. Printed in the
United States of America.
Addison-Wesley is an imprint of Addison Wesley Longman, Inc.
Cover design by Suzanne Heiser and Yaneer Bar-Yam
Text design by Jean Hammond
Set in 10/12.5 Minion by Carlisle Communications, LTD
1 2 3 4 5 6 7 8 9—MA—0100999897
First printing, August 1997
Find us on the World Wide Web at
http://www.aw.com/gb/
This book is dedicated with love to my family
Zvi,Miriam, Aureet and Sageet
Naomi
and our children
Shlomiya, Yavni,Maayan and Taeer
Aureet’s memory is a blessing.
vii
Contents
Preface xi
Acknowledgments xv
0 Overview: The Dynamics of Complex Systems—Examples,
Questions, Methods and Concepts 1
0.1 The Field of Complex Systems 1
0.2 Examples 2
0.3 Questions 6
0.4 Methods 8
0.5 Concepts: Emergence and Complexity 9
0.6 For the Instructor 14
1 Introduction and Preliminaries 16
1.1 Iterative Maps (and Chaos) 19
1.2 Stochastic Iterative Maps 38
1.3 Thermodynamics and Statistical Mechanics 58
1.4 Activated Processes (and Glasses) 95
1.5 Cellular Automata 112
1.6 Statistical Fields 145
1.7 Computer Simulations (Monte Carlo, Simulated Annealing) 186
1.8 Information 214
1.9 Computation 235
1.10 Fractals, Scaling and Renormalization 258
2 Neural Networks I: Subdivision and Hierarchy 295
2.1 Neural Networks: Brain and Mind 296
2.2 Attractor Networks 300
2.3 Feedforward Networks 322
2.4 Subdivided Neural Networks 328
2.5 Analysis and Simulations of Subdivided Networks 345
2.6 From Subdivision to Hierarchy 364
2.7 Subdivision as a General Phenomenon 366
3 Neural Networks II: Models of Mind 371
3.1 Sleep and Subdivision Training 372
3.2 Brain Function and Models of Mind 393
4 Protein Folding I: Size Scaling of Time 420
4.1 The Protein-Folding Problem 421
4.2 Introduction to the Models 427
4.3 Parallel Processing in a Two-Spin Model 432
4.4 Homogeneous Systems 435
4.5 Inhomogeneous Systems 458
4.6 Conclusions 471
5 Protein Folding II: Kinetic Pathways 472
5.1 Phase Space Channels as Kinetic Pathways 473
5.2 Polymer Dynamics: Scaling Theory 477
5.3 Polymer Dynamics: Simulations 488
5.4 Polymer Collapse 503
6 Life I: Evolution—Origin of Complex Organisms 528
6.1 Living Organisms and Environments 529
6.2 Evolution Theory and Phenomenology 531
6.3 Genome, Phenome and Fitness 542
6.4 Exploration, Optimization and Population Interactions 550
6.5 Reproduction and Selection by Resources and Predators 576
6.6 Collective Evolution: Genes, Organisms and Populations 604
6.7 Conclusions 619
viii C o n t e n t s
7 Life II: Developmental Biology—Complex by Design 621
7.1 Developmental Biology: Programming a Brick 621
7.2 Differentiation: Patterns in Animal Colors 626
7.3 Developmental Tool Kit 678
7.4 Theory, Mathematical Modeling and Biology 688
7.5 Principles of Self-Organization as Organization by Design 691
7.6 Pattern Formation and Evolution 695
8 Human Civilization I: Defining Complexity 699
8.1 Motivation 699
8.2 Complexity of Mathematical Models 705
8.3 Complexity of Physical Systems 716
8.4 Complexity Estimation 759
9 Human Civilization II: A Complex(ity) Transition 782
9.1 Introduction: Complex Systems and Social Policy 783
9.2 Inside a Complex System 788
9.3 Is Human Civilization a Complex System? 791
9.4 Toward a Networked Global Economy 796
9.5 Consequences of a Transition in Complexity 815
9.6 Civilization Itself 822
Additional Readings 827
Index 839
C o n t e n t s ix
Preface
“Com p l ex ” is a word of the ti m e s , as in the of ten - qu o ted “growing com p l ex i ty of
l i fe .” S c i en ce has begun to try to understand com p l ex i ty in natu re , a co u n terpoint to
the trad i ti onal scien tific obj ective of u n derstanding the fundamental simplicity of
l aws of n a tu re . It is bel i eved ,h owever, that even in the stu dy of com p l ex i ty there exist
simple and therefore com preh en s i ble laws . The field of s tu dy of com p l ex sys tem s
holds that the dynamics of com p l ex sys tems are fo u n ded on universal principles that
m ay be used to de s c ri be dispara te probl ems ra n ging from parti cle physics to the econ
omics of s oc i eti e s . A coro ll a ry is that tra n s ferring ideas and re sults from inve s ti gators
in hitherto dispara te areas wi ll cro s s - ferti l i ze and lead to important new re su l t s .
In this text we introduce several of the problems of science that embody the concept
of complex dynamical systems. Each is an active area of research that is at the
forefront of science.Our presentation does not try to provide a comprehensive review
of the research literature available in each area. Instead we use each problem as an opportunity
for discussing fundamental issues that are shared among all areas and therefore
can be said to unify the study of complex systems.
We do not expect it to be possible to provide a succinct definition of a complex
system. Instead, we give examples of such systems and provide the elements of a definition.
It is helpful to begin by describing some of the attributes that characterize
complex systems. Complex systems contain a large number of mutually interacting
parts. Even a few interacting objects can behave in complex ways. However, the complex
systems that we are interested in have more than just a few parts.And yet there is
generally a limit to the number of parts that we are interested in. If there are too many
parts, even if these parts are strongly interacting, the properties of the system become
the domain of conventional thermodynamics—a uniform material.
Thus far we have defined complex systems as being within the mesoscopic domain
—containing more than a few, and less than too many parts.However, the mesoscopic
regime describes any physical system on a particular length scale,and this is too
broad a definition for our purposes. Another characteristic of most complex dynamical
systems is that they are in some sense purposive.This means that the dynamics of
the system has a definable objective or function. There often is some sense in which
the systems are engineered.We address this topic directly when we discuss and contrast
self-organization and organization by design.
A central goal of this text is to develop models and modeling techniques that are
useful when applied to all complex systems. For this we will adopt both analytic tools
and computer simulation. Among the analytic techniques are statistical mechanics
and stochastic dynamics.Among the computer simulation techniques are cellular automata
and Monte Carlo. Since analytic treatments do not yield complete theories of
complex systems, computer simulations play a key role in our understanding of how
these systems work.
The human brain is an important example of a complex system formed out of its
component neurons. Computers might similarly be understood as complex interacting
systems of transistors.Our brains are well suited for understanding complex sysxi
xii P re fa c e
tems, but not for simulating them.Why are computers better suited to simulations of
complex systems? One could point to the need for precision that is the traditional domain
of the computer. However, a better reason would be the difficulty the brain has
in keeping track of many and arbitrary interacting objects or events—we can typically
remember seven independent pieces of information at once. The reasons for this are
an important part of the design of the brain that make it powerful for other purposes.
The architecture of the brain will be discussed beginning in Chapter 2.
The study of the dynamics of complex systems creates a host o f new interdisciplinary
fields. It not only breaks down barriers between physics, chemistry and biology,
but also between these disciplines and the so-called soft sciences of psychology,
sociology, economics,and anthropology.As this breakdown occurs it becomes necessary
to introduce or adopt a new vocabulary. Included in this new vocabulary are
words that have been considered taboo in one area while being extensively used in another.
These must be adopted and adapted to make them part of the interdisciplinary
discourse. One example is the word “mind.” While the field of biology studies the
brain,the field of psychology considers the mind.However, as the study of neural networks
progresses,it is anticipated that the function of the neural network will become
identified with the concept of mind.
An o t h er area in wh i ch scien ce has trad i ti on a lly been mute is in the con cept of m e a ning
or purpo s e . The field of s c i en ce trad i ti on a lly has no con cept of va lues or va lu a ti on .
Its obj ective is to de s c ri be natu ral ph en om ena wi t h o ut assigning po s i tive or nega tive
con n o t a ti on to the de s c ri pti on .However, the de s c ri pti on of com p l ex sys tems requ i res a
n o ti on of p u rpo s e ,s i n ce the sys tems are gen era lly purpo s ive .Within the con text of p u rpose
there may be a con cept of va lue and va lu a ti on . If , as we wi ll attem pt to do, we add
ress soc i ety or civi l i z a ti on as a com p l ex sys tem and iden tify its purpo s e ,t h en va lue and
va lu a ti on may also become a con cept that attains scien tific sign i f i c a n ce . Th ere are even
f u rt h er po s s i bi l i ties of i den ti f ying va lu e ,s i n ce the very con cept of com p l ex i ty all ows us
to iden tify va lue with com p l ex i ty thro u gh its difficulty of rep l acem en t . As is usual wi t h
a ny scien tific adva n ce ,t h ere are both dangers and opportu n i ties with su ch devel opm en t s .
Finally, it is curious that the origin and fate of the universe has become an accepted
subject of scientific discourse—cosmology and the big bang theory—while the
fate of humankind is generally the subject of religion and science fiction. There are
exceptions to this rule, particularly surrounding the field of ecology—limits to population
growth, global warming—however, this is only a limited selection of topics
that could be addressed. Overcoming this limitation may be only a matter of having
the appropriate tools. Developing the tools to address questions about the dynamics
of human civilization is appropriate within the study of complex systems. It should
also be recognized that as science expands to address these issues, science itself will
change as it redefines and changes other fields.
Different fields are often distinguished more by the type of questions they ask
than the systems they study. A significant effort has been made in this text to articulate
questions, though not always to provide complete answers, since questions that
define the field of complex systems will inspire more progress than answers at this
early stage in the development of the field.
Like other fields, the field of complex systems has many aspects, and any text
must make choices about which material to include.We have suggested that complex
systems have more than a few parts and less than too many of them.There are two approaches
to this intermediate regime. The first is to consider systems with more than
a few parts, but still a denumerable number—denumerable,that is, by a single person
in a reasonable amount of time. The second is to consider many parts, but just fewer
than too many. In the first approach the main task is to describe the behavior of a particular
system and its mechanism of operation—the function of a neural network of
a few to a few hundred neurons, a few-celled organism, a small protein,a few people,
etc. This is done by describing completely the role of each of the parts. In the second
approach, the precise number of parts is not essential,and the main task is a statistical
study of a collection of systems that differ from each other but share the same
structure—an ensemble of systems. This approach treats general properties of proteins,
neural networks, societies, etc. In this text, we adopt the second approach.
However, an interesting twist to our discussion is that we will show that any complex
system requires a description as a particular few-part system.A complementary volume
to the present one would consider examples of systems with only a few parts and
analyze their function with a view toward extracting general principles. These principles
would complement the seemingly more general analysis of the statistical
approach.
The order of presentation of the topics in this text is a matter of taste. Many of
the chapters are self-contained discussions of a particular system or question.The first
chapter contains material that provides a foundation for the rest. Part of the role of
this chapter is the introduction of “simple” models upon which the remainder of the
text is based. Another role is the review of concepts and techniques that will be used
in later chapters so that the text is more self-contained. Because of the interdisciplinary
nature of the subject matter, the first chapter is considered to have particular importance.
Some of the material should be familiar to most graduate students, while
other material is found only in the professional literature. For example, basic probability
theory is reviewed, as well as the concepts and properties of cellular automata.
The purpose is to enable this text to be read by students and researchers with a variety
of backgrounds.However, it should be apparent that digesting the variety of concepts
after only a brief presentation is a difficult task. Additional sources of material
are listed at the end of this text.
Throughout the book, we have sought to limit advanced formal discussions to a
minimum.When possible, we select models that can be described with a simpler formalism
than must be used to treat the most general case possible. Where additional
layers of formalism are particularly appropriate, reference is made to other literature.
Simulations are described at a level of detail that,in most cases,should enable the student
to perform and expand upon the simulations described.The graphical display of
such simulations should be used as an integral part of exposure to the dynamics of
these systems. Such displays are generally effective in d eveloping an intuition about
what are the important or relevant properties of these systems.
P re fa c e xiii
Acknowledgments
This book is a composite of many ideas and reflects the efforts of many individuals
that would be impossible to acknowledge.My personal efforts to compose this body
of knowledge into a coherent framework for future study are also indebted to many
who contributed to my own development. It is the earliest teachers, who we can no
longer identify by memory, who should be acknowledged at the completion of a major
effort. They and the teachers I remember from elementary school through graduate
school, especially my thesis advisor John Joannopoulos, have my deepest g ratitude.
Consistent with their dedication, may this be a reward for their efforts.
The study of complex systems is a new endeavor, and I am grateful to a few colleagues
and teachers who have inspired me to pursue this path. Charles Bennett
through a few joint car trips opened my mind to the possibilities of this field and the
paths less trodden that lead to it.Tom Malone, through his course on networked corporations,
not only contributed significant concepts to the last chapter of this book,
but also motivated the creation of my course on the dynamics of complex systems.
There are colleagues and students who have inspired or contributed to my understanding
of various aspects of material covered in this text. Some of this contribution
arises from reading and commenting on various aspects of this text, or through
discussions of the material that eventually made its way here. In some cases the discussions
were originally on unrelated matters, but because they were eventually connected
to these subjects,they are here acknowledged. Roughly divided by area in correspondence
with the order they appear in the text these include: Glasses—David
Adler; Cellular Automata—Gerard Vichniac, Tom Toffoli, Norman Margolus, Mike
Biafore, Eytan Domany,Danny Kandel; Computation—Jeff Siskind;Multigrid—Achi
Brandt, Shlomi Taasan, Sorin Costiner; Neural Networks—John Hopfield, Sageet
Bar-Yam, Tom Kincaid, Paul Appelbaum, Charles Yang, Reza Sadr-Lahijany, Jason
Redi, Lee-Peng Lee, Hua Yang, Jerome Kagan, Ernest Hartmann; Protein Folding—
Elisha Haas, Charles DeLisi, Temple Smith, Robert Davenport, David Mukamel,
Mehran Kardar; Polymer Dynamics—Yitzhak Rabin, Mark Smith, Boris Ostrovsky,
Gavin Crooks, Eliana DeBernardez-Clark; Evolution—Alan Perelson, Derren Pierre,
Daniel Goldman, Stuart Kauffman, Les Kaufman; Developmental Biology—Irving
Epstein, Lee Segel, Ainat Rogel, Evelyn Fox Keller; Complexity—Charles Bennett,
MichaelWerman,Michel Baranger; Human Economies and Societies—Tom Malone,
Harry Bloom, Benjamin Samuels, Kosta Tsipis, Jonathan King.
A special acknowledgment is necessary to the students of my course from Boston
University and MIT. Among them are students whose projects became incorporated
in parts of this text and are mentioned above. The interest that my colleagues have
shown by attending and participating in the course has brightened it for me and their
contributions are meaningful: Lewis Lipsitz, Michel Baranger, Paul Barbone, George
Wyner,Alice Davidson,Ed Siegel,MichaelWerman,Larry Rudolfand Mehran Kardar.
Among the readers of this text I am particularly indebted to the detailed comments
of Bruce Boghosian, and the supportive comments of the series editor Bob
Devaney. I am also indebted to the support of Charles Cantor and Jerome Kagan.
xv
I would like to acknowledge the constructive efforts of the editors at Addison-
Wesley starting from the initial contact with Jack Repcheck and continuing with
Jeff Robbins. I thank Lynne Reed for coordinating production, and at Carlisle
Communications: Susan Steines, Bev Kraus, Faye Schilling, and Kathy Davis.
The software used for the text, graphs, figures and simulations of this book, includes:
Microsoft Excel and Word, Deneba Canvas, Wolfram’s Mathematica, and
Symantec C.The hardware includes:Macintosh Quadra,and IBM RISC workstations.
The contributions of my family, to whom this book is dedicated, cannot be described
in a few words.
”
Yaneer Bar-Yam
Newton, Massachusetts, June 1997
xvi Ac k n ow l e d g m e n t s
