Unpredictability, Chaos, and the Butterfly Effect

Note to the reader:  If you don't have the time to wade through this whole post but just want the "main thrust", skip to the last paragraph.  You won't miss all that much.



Perhaps you can identify a common theme in the following articles:

1.  Active investing (picking stocks, timing the market, etc.) is a losing game.  Period.  End of story.  If you don't believe this, you are ignoring hundreds of detailed studies and analyses (understandably compiled by those touting index funds), and you are merely hoping you are the lucky 1% who are just that--lucky.  Unfortunately, brokers, investment companies, stock trading firms like TDAmeritrade, and various pundits continue to promote the illusion that stock picking works, and all you need are the latest tools and strategies.  This helps their business, but does not help you.  There is even some evidence that the smarter you are, the worse you'll do.  A diversified portfolio, or investing in a broad index fund--and very rarely touching it--is the best way to maximize your returns while minimizing your stress and wasted time.   This is especially true today, where the financial industry has transformed itself into a global casino.

2.  Hedge fund managers--who are supposedly the "creme-de-la-creme" and who have additional financial instruments, tools, data, leverage, flexibility and clout that most of us don't have--aren't even beating the S&P500. Warren looks like he's on the verge of winning his bet.  When we are provided more data and more tools, this only gives us the illusion that we have a better chance of beating the stock market.  The only way to consistently beat the stock market is to break the law--i.e., use inside information or run a ponzi scheme or manipulate the market or misrepresent what you're selling--in other words, lie/cheat/steal.

3.  Individual economic forecasters--and even these forecasters taken in the aggregate--are often wrong, and here are some possible reasons why, along with some specific examples of outrageously-flawed economic predictions.  

4.  Even political scientists make lousy predictions, due to the inherently unpredictable and shifting nature of political power, alliances and propaganda and the sheer number and diversity of their purveyors on the world stage.  The civil war in Syria is a prime example of a political situation that has a lot of dynamics and a high degree of unpredictability.  Many countries--as well as radical Islamic groups--have their eye on Syria and are bent on furthering their own agendas.

 5.  Psychiatrists and psychologists--who are highly trained--have a rather dismal record of predicting (much less treating or curing) violent behavior in individuals.  We have a long way to go before we can truly understand all the subtleties and complexities of how the human brain works, much less make such predictions.  Unfortunately, this doesn't stop them from making error-fraught predictions that can dramatically affect lives.

6.  In general, experts in any field of complex behavior--for example when it involves human relations--are generally lousy at predictions, and are often worse than non-experts, and even worse than simple statistical formulas or mathematical averages.  Some reasons for this are suggested in this article.  Another phenomenon that tempts people to believe they can make accurate predictions about the future, is that "things seem obvious once you know the answer" (i.e., when you look into the past).  This is discussed in the presentation "The Myth of Common Sense".

By now, you've realized that all these articles concern the difficulty in making predictions when the complexity of a system is so overwhelming that it is virtually unpredictable.  And even systems that--on the surface--don't necessarily seem so complex to us often display a characteristic called "sensitive dependence to initial conditions", also sometimes called the "butterfly effect."  This non-linear characteristic makes systems chaotic, even while they remain deterministic.  It is the essence of chaos theory.  A good example of this is the weather, as well as other natural phenomenon such as earthquakes.  Note that we can do a pretty good job of predicting tomorrow's weather based on today's weather.  With less accuracy, we can even predict day-after-tomorrow's weather.  Assuming the weather is in the habit of changing, our ability to predict the weather a week out is very poor, despite complex computer models and hugely powerful computers.  Two weeks out?  Forget it, flip a coin instead.  This limitation is not likely to change much, even with another 10, 100 or 1000-fold increase in computing power.

For those wanting a little more scientific explanation of why our world is so unpredictable, this non-technical video explains how chaos theory limits predictability.  An elementary chaos theory text also explains some important concepts, including the most important concept of "sensitive dependence".  In general, the more complex and non-linear the system dynamics, the more unpredictable the system is, and the more difficult it is to predict or control any specific future outcome.


So....why am I talking about all this?  This might seem to be a pessimistic discussion, given we are forced to acknowledge that the machinery of our planet, of life and of society tends to be so non-linear, so complex and so bewildering that we cannot predict future events to any extent, much less control them...that the world is chaotic, that we are merely one little unimportant cog in this mysterious machine...

However, there are also some instructive aspects to this discussion.

1.  There is a certain relief in knowing that we can take all those shockingly grim or Pollyanna forecasts with a grain of salt--gripping predictions merely make for gripping headlines that sell advertising.  Great things and horrible things do happen, but neither you nor anyone else will be able to predict them very far into the future.  It's also a relief to know that we can also stop spending a lot of time actively managing our investments--it's not just a waste of time, it also costs money and peace of mind.

2.  If, as stated previously, stock trading doesn't work, what about high frequency trading?  Interestingly, in theory this DOES give the trader an advantage.  The more powerful his computers, the more sophisticated his algorithms and --most importantly--the less real-time delay in his trades (and thus the better his predictive accuracy), the greater the advantage.  Of course, with many high-frequency traders competing, this again becomes a zero-sum game.  Currently, an eye-popping 80% of stock trading in the U.S. is attributed to high-frequency trading by computer algorithms that may have no interest in the future worth of a stock beyond a millisecond or so.  While this is a dangerous development, perhaps the human lesson for us is that it is easier for us to predict events, deal with issues and otherwise do well, the closer our mind is to the present.  It is also much easier and more advantageous for us to live in the present, rather than having our thoughts continually drifting into the future or the past.  As someone said, "I cannot predict the future; I cannot change the past; I have just the present;I must treat it as my last."  (...Just in case this post wasn't already preachy enough.)

3.  Using the "mysterious machine" metaphor, imagine that you are staring down from a high scaffold into a huge and extremely complicated mechanical contraption with myriad whizzing gears and flywheels, levers, dials and such, located in the middle of  your city.  You are told that this machine is indirectly responsible for the financial, social, physical and environmental well-being of everyone in your city.  However, it is so enormous and so complicated that you get dizzy just staring at it.  You think to yourself  "if only I could understand this machine better, perhaps that knowledge could benefit me, allow me to manipulate it and possibly give me some advantage!"  But you will never understand the machine that well.  What can you do?  All you have in your possession is a drop of oil and a pinch of sand.  You can choose to toss the drop of oil or the pinch of sand into the machine.  So can 100,000 others who live in your city.  The moral of this story is that even if you cannot predict, control or manipulate the machine, you can still influence it in a positive or negative way.  That pinch of sand or drop of oil may or may not make a huge difference--either negative or positive--in the smooth functioning of the machine, depending on where it falls.  Borrowing some statistical terminology from medicine, we would label this a stochastic effect rather than a strictly deterministic effect.  Our individual actions do matter, because by definition, without them, there would be no such thing as a collective influence and thus no such thing as a deterministic outcome--which we know does, in reality, exist.  Also, every action we or someone else takes that we know DOES matter, is actually the ultimate result of myriad prior actions that--each taken by itself--might otherwise have seemed not to matter.

4.  What if, by your fiery eloquence, you were able to convince 500 other people that some part of the machine might be evil and was not necessarily working in their interests?  What if those 500 people decided to throw sand into the machine as a result of your rhetoric?  On top of that, what if one person had illegally collected a bag of rocks and threw them in as well?  To what extent should you feel responsible if part of the machine stopped functioning as a result?  Some people incite others to destructive behavior but excuse themselves from any responsibility.  They point to the 2000 other people who had already thrown sand into the machine, independent of their influence.  They defend themselves by asking how anyone can prove they were specifically responsible.  They further defend themselves by saying that random events happen all the time, so how can a bad outcome be traced to them given the unpredictability of bad events anyway?  In other words, "I didn't specifically ask anybody to throw sand into the machine, much less rocks!  I can't be responsible for the illegal actions of crazy people!  I was just suggesting that the machine could be improved!"  These people have been sometimes characterized as "stochastic terrorists".  Of course, on the flip side, there are surely "stochastic humanitarians" as well--whose positive influence has an equally unpredictable yet overwhelmingly positive effect on the lives of others.

5.  Is there a mathematical analogy that can help explain how something can seem totally random yet be totally deterministic?  Well, how about the sequence of digits in an irrational number like pi, e, sqrt(2), etc?  Since a rational number can be described as the sum of an infinite sequence where the next term in the sequence is always a particular function of previous terms in the sequence, it is deterministic, even though the terms result in a string of digits that look like a random sequence.  Actually, an engineer or physicist would tell you that any system's output--not just the series of digits in an irrational number--can theoretically be computed for the "next instant of time" based on a differential equation--the only catch is, you must completely understand and be able to measure all the inputs, outputs and dynamic characteristics of the system (as described by differential equations), and you must not choose too large a time step.  If all these requirements are met, you can--computationally--predict with reasonable accuracy what's going to happen next. However, imperfect accuracy in system assumptions, measurement accuracy and computing precision can affect your results, and the further you go into the future, the more drastic is your resulting prediction error, especially for a typical, real-life, non-linear system where errors get magnified quickly.

6.  If you believe that you live in a random, unpredictable world, you might come to the conclusion that your actions don't matter much.  However, as previously stated, the world is NOT random, it is instead chaotic, which means that while it may appear on the surface to be random, it is nevertheless deterministic--everything has a cause, and everything has an effect.  Everything you do makes a difference, and your actions could conceivably influence the world more than you ever imagined--remember the butterfly affect?  Here's an inspiring Butterfly Effect video by Andy Andrews.  If you liked this, here's another one by the same person.

"Happiness Advantage" Book Review

On a recent road trip, I listened to the (audio)book "Happiness Advantage", by Shawn Achor.  It's chock full of fascinating psychological study results, not to mention practical suggestions for how to be happy based on numerous studies and the author's own research.  Given the sheer number of books on positive psychology, it might be presumptuous to call this the best book ever written on happiness (notwithstanding the Good Book), and I don't want to sound like a shill for the book.  However, I did find it helpful, so I'll provide a summary for those who are interested:

Happiness is hard to define--experts tend to identify it as having components of pleasure, engagement and meaning, but even this is an oversimplification.  While happiness is not something we can pursue directly, there ARE some practical things we can do to greatly increase our likelihood of being happy, according to the author.  These include exercise, journaling, meditation, lifelong learning and most importantly, scheduling time each and every day 

     (a) to recount three good things that we feel glad or happy about in the last 24 hours

     (b) to plan a "random act of kindness" or even a word of praise or encouragement for someone else

     (c) to plan to do something to help maintain and preserve good relationships with our family, friends and co-workers

Sadly, as the author admits, most people will nod their heads in agreement, will consider this good, practical common sense, but will nevertheless fail to follow through on this advice in their own personal lives.

These recommendations are strictly based on psychological study results as well as the author's own research.  Being a Christian, I tend to see these recommendations as an echo of some of the exhortations of Jesus, who was "anointed with the oil of gladness above his fellows," and who said "it is more blessed to give than to receive", "love one another as I have loved you" and so on.  If you are not a Christian, you can still prove for yourself whether or not these recommendations work in your own life.  I personally believe for myself that they are even more potent in the context of my Christian faith.  In any case, if this is not put into practice, it becomes useless either way.

Some other conclusions from the book:

1)  "Success" is the result of "happiness", not the other way around.  This simple concept has been demonstrated time and time again by numerous psychological studies and the latest research.

2)  Average levels of happiness have surprisingly limited correlation to our wealth, status, geographic location, culture, race, and most other factors over which we have little control.  "90% of your long-term happiness is predicted not by the external world, but by the way your brain processes the world."

3)  Numerous psychological studies demonstrate the critical nature of perceptions, and the huge effect they can have on us.  What we tell ourselves and what others tell us--even the language used--can shape these perceptions.  These perceptions, in turn, have a critically-important effect on the direction of our lives, and can prime us for success or failure.  (It is difficult to convey just how subtle and yet how dramatic this effect can be without describing some of the amazing psychological study results cited in the book.)  Also, while brain plasticity is well documented, our messages to ourselves and others must convey the belief that we (and they) are truly capable of learning and improving in whatever we endeavor to do.  As evidenced also by the placebo effect, we tend to greatly underestimate the power of our beliefs and perceptions.

4)  The "tetris effect" describes how our state of mind, mental habits and perceptions can alter our experience of the actual world--i.e., what information we consciously identify and what information our brain automatically filters out as unimportant.  In this way, two people may have very different experiences even as they are exposed to identical physical environments. “...what we’re finding is that it’s not necessarily the reality that shapes us, but the lens through which your brain views the world that shapes your reality. If we can change the lens, not only can we change your happiness, but we can change every single educational and business outcome at the same time.”

5)  Married people tend to be happier than single people.  For a marriage (or a business team) to work, there need to be at least 5 positive remarks for each negative remark.  Loners and introspective people tend to be the most unhappy, as are people who use the word "I" a lot (versus "we").  People under stress sometimes withdraw from their support network of family and friends just when they need them the most.  Good social relationships are essential to happiness.

6)  Our mood--even a simple smile--can have far reaching consequences to those around us and can be re-transmitted far beyond even those people.  So can the presence of stress and irritability.  Often, the subtle perceptions gleaned from a manager's words and conduct at a meeting can have a profound effect on the mood, stress level, self-confidence and emotional well-being of his entire organization, without him or her even realizing it.

7)  Don't waste your will power on unimportant, unnecessary temptations and decisions.  We each have only so much will power to use up each day before it's spent, and it's best to spend it wisely, on choices that are truly important.  For example, don't tempt yourself unnecessarily by driving by the Dunkin' Donuts shop each morning and then agonizing over whether you should stop for a donut and coffee.  And don't spend 10 minutes each morning agonizing over whether or not you will exercise, where you will go, what you will do, for how long, etc. when you could just as easily make these decisions ahead of time as part of a set schedule.

8)  Use the 20 second rule to help you develop good habits:  We all have a lazy streak--tasks that take more than 20 seconds to get started on tend not to get done, and vice versa.  To encourage a particular habit, make it easy to initiate that activity--for example, if you want to learn how to play guitar, put the guitar in a convenient place in your living room where you can easily pick it up without dragging it up from the basement, taking it out of the case, tuning it, finding your music, etc.  On the other hand, to discourage a bad habit such as repeatedly checking your e-mail, make it intentionally a little less convenient (i.e., requiring more than 20 seconds) to fire up your e-mail program, enter your password, etc..  This may be all it takes to make a huge difference in your daily habits.

This Tedx video doesn't substitute for the book, but it gives a taste of the author's message, if you are interested.  Another interesting Ted Presentation by Nancy Etcoff provides some interesting statistics that further shows why happiness is a pertinent subject that needs to be better understood.  And of course, this video reminds us of what we can learn from a dog.