Extending the Reach of Info-Metrics to Dynamic and Non-Hierarchical Complex Systems

A Joint Info-Metrics and Santa Fe Institute Working Group

Where & When

March 15-16, 2018
Santa Fe Institute

Meeting Summary

This Working Group deals with two interdependent questions - both related to our understanding of complex systems. The first is to do with the 'Hierarchical Organization' of a system. The main question here is: Can those same info-metrics tools (that also include the tools of information theory and the maximum entropy principle) provide insight into the properties of complex systems for which discrete hierarchical scales are not readily resolvable? In more general terms, the basic issue here is to study whether there is a distinction (and if so, what it is) between complex systems that are organized into discrete hierarchical levels and those for which no such delineation of discrete hierarchies is possible? That is, are systems of the second type in some practical sense more complex than those of the first type?

The second, and inter-related, set of questions is to do with systems far away from equilibrium. Specifically, is the failure of information-theoretic approaches when applied to far-from-steady-state systems (such as in ecology) true across complex dynamic systems in general? Can the inferential tools of information theory and info-metrics be modified and extended to provide insight into the dynamics of rapidly changing complex systems? Does the answer depend in any way on whether or not the system has discrete hierarchical levels of organization? Will the introduction of additional uncertainty about the constraints (information) be useful in that case?

The Broader Context and Background

Hierarchical Organization

For modeling purposes, some complex systems can be usefully conceptualized as comprised of discrete hierarchical levels of organization. In equilibrium thermodynamics, considerable theoretical insight is attained by distinguishing the macroscale (e.g., pressure, volume, temperature, specific heat) from the microscale (e.g., molecular kinetic energies). A variety of inference tools allow us to progress from knowledge of constraints (capturing the available information) at one hierarchical level to the prediction of phenomena at the other. Economies, for example, are often usefully conceptualized as consisting of individual agents, first-order aggregates of individuals (e.g., households or firms, or both), and the aggregate of firms and agents (the economy as a whole). Ecosystems, on the other hand, are envisioned at the level of individual organisms organized into population or guilds or species, the aggregate of which is an ecological community.

In contrast, turbulent (or 'very complex') systems provide the classic example of a system for which the dynamics operates across a continuum of scales, without any natural discretization of levels of organization. Finally, some systems may be harder to classify by this criterion, being intermediate between the discreet and the continuous; an example might be networks in which nodes form modular units of many sizes. Stated differently, the system is characterized in a myriad of ways, either at different coarse-grains or over different subsets where each scale behaves differently.

The tools of information theory and info-metrics have been demonstrated to be a powerful approach to illuminating the properties of steady state systems of the first (discrete levels) type, with clearly definable macro- and micro-scales of organization. Successful efforts here have included applications to thermodynamics, economics, ecology, biology, chemistry and more. This leads us to:

Workshop Questions 1

Can those same info-metrics tools (that also include the tools of information theory and the maximum entropy principle) provide insight into the properties of complex systems for which discrete hierarchical scales are not readily resolvable? More broadly, what, if any, relevance is there of the distinction between complex systems that are organized into discrete hierarchical levels and those for which no such delineation of discrete hierarchies is possible? Are systems of the second type in some practical sense more complex than those of the first type?

Complexity Far from Steady State

Current successful applications of the tools of information-theory and info-metrics to ecology, economics and other systems have largely been confined to hierarchical systems in, or close to, steady state. In ecology evidence is accumulating for characteristic patterns of failure of maximum entropy based theory in ecosystems that are changing rapidly as a consequence of human disturbance, ecological succession, or evolutionary diversification. The same observation is true for other systems. This leads us to:

Workshop Questions 2

Is the failure of information-theoretic approaches when applied to far-from-steady-state systems (such as in ecology) true across complex dynamic systems in general? Can the inferential tools of information theory and info-metrics be modified and extended to provide insight into the dynamics of rapidly changing complex systems? Does the answer depend in any way on whether or not the system has discrete hierarchical levels of organization? Will the introduction of additional uncertainty about the constraints (information) be useful in that case?

Working Group Objectives

In this workshop we want to explore the above two questions. Within these questions we are also interested in the following special topics:

  1. What is complexity?
  2. Can we refine the connection between information and complexity?
  3. Can info-metrics help clarify the flavors of complexity?

Broader Context

Enhance our understanding of the deep interconnection between information theory, info-metrics, and complex systems, where 'complexity' here is characterized in a certain order of 'complication' - from relatively simpler complex system to the more complex one; from a system near its equilibrium state to a system far away from its equilibrium state. In that regard, we plan to study these different levels of complexities and to investigate whether the tools of info-metrics are helpful in providing a better modeling tool for understanding such systems.

Potential Outcomes

We expect this working group to be the beginning of a joint multidisciplinary effort to pursue advanced info-metrics work on modeling complexity, and on inference of complex systems from noisy and imperfect information.

We expect this first working group will lead to follow-up workshops (and possibly tutorials) as well as a joint publication by the group.

Organizers

Amos Golan (American U, SFI, Pembroke College)
John Harte (UC Berkeley, SFI)
Andrew Rominger (SFI)