We introduce a practical measure for the complexity of sequences of
symbols (``strings'') that is rooted in automata theory but avoids the
problems of Kolmogorov-Chaitin complexity. This physical complexity
can be estimated for ensembles of sequences, for which it
reverts to the difference between the maximal entropy of the ensemble
and the actual entropy given the specific environment within which the
sequence is to be interpreted. Thus, the physical complexity measures
the amount of information about the environment that is coded in the
sequence, and is conditional on such an environment. In practice, an
estimate of the complexity of a string can be obtained by counting the
number of loci per string that are fixed in the ensemble, while the
volatile positions represent, again with respect to the environment,
randomness. We apply this measure to tRNA sequence data.
Artificial neural networks (ANNs) based on McCullough-Pitts neurons
and the standard connectionist paradigm can be analyzed analytically,
but do not appear to help in understanding biological information
processing in the brain. Instead of constructing ANNs, we attempt to
model decentralized growth and development of neural networks inspired
by the molecular biology and physiology of real nervous systems. In
this model, each individual artificial neuron is an autonomous unit
whose behavior is only determined by the genetic information it
harbors and local concentrations of substrates. The chemicals and
substrates, in turn, are modeled by a simple artificial chemistry.
The combination of local substrate concentrations and genetic
information leads to gene expression, manifested as axon and dendrite
growth, cell division and differentiation, substrate production,
and cell stimulation. While genomes leading to naturally grown ANNs
can evolve according to Darwin's principle of selection and survival
of the fittest, we demonstrate the power of the artificial chemistry
with engineered (user-written) genomes that
lead to the growth of simple networks with behaviors similar to
known physiology such as deterministically structured networks,
pacemaker behavior, sensitization, habituation, associative
classical conditioning, computation of logical functions, and
self-limiting network growth. To evolve more complex structures, we
implemented a platform-independent, asynchronous, distributed
Genetic Algorithm (GA) for the genomes that code for
development, expression, and the physiology of the neurons, and that
allows users on the Internet to participate in evolutionary
experiments via the World Wide Web.
Branching processes have a surprisingly universal dynamics which
gives rise to scale-free dynamics under certain circumstances. The
simplest branching process (the Galton-Watson process) is also the
oldest, having been invented to study the disappearance of family
names from the British peerage. It can also be used any time a process
leads to branchings based on a probability distribution. The
"critical" parameter in such a system is the average number m of
"daughters", i.e., replicas, a node in such a branching system
has. In an infinite system this number can exceed one, but not if
it is finite and selection is present. In that case, the critical value
is m=1, a situation in which selection is extremely strong:
there are (almost) no competing nodes in such a tree. A simple
mean-field treatment of this model shows that in this limit the
"taxon" abundance distributions, i.e., the probability distribution
for an initial node to give rise to a "family" with n
sub-nodes is a pure power-law. As selection becomes weaker, allowing
more competing neutral nodes in the system, m becomes smaller
than one and the power law degenerates into an exponential. We have
studied this process theoretically and numerically, and applied it to
taxon distributions in the fossil record, taxon distributions of
digital organisms, as well as avalanche-size distributions in sandpile
models. The latter study shows that sandpiles are only critical in a
very small region of parameter space which is in fact
unphysical. Physical sandpiles are non-critical, while evolutionary
abundance distributions are driven towards the critical regime by the
system's dynamics.