[[ZEITGEIST]]

IPOD RANDOMNESS

The Guardian recently published an article by Steven Levy about the non randomness of the ipod shuffle function. This article concludes that,

1. true randomness is hard to achieve and
2. humans perceive patterns where there are none.

I wonder if a paradoxical interpretation of these facts may better explain what is happening. That is to say

1. randomness is difficult to achieve because the universe is not random and
2. as part of the universe, human beings are attuned to these patterns. This is in fact why there is such a thing as meaningful coincidence. Wasn't this what Jung was talking about when he advanced the concept of synchronicity?

Like levy and everyone else apparently, my ipod tends to favour certain songs over others. The trick is to delete those songs when they get too boring and thus disrupt the pattern of meaningful coincidence. This got me wondering if the ipod analogy could be extended to apply to life. In life, certain pattern clusters predominate, for example a person might have a run of good or bad luck. The wisdom of ipod suggests to me that, if it is the latter, a person should deliberately mess with the patterns of their lives (i.e. change their habits) and this would produce a different pattern configuration. Changing a habit would be the life-equivalent of deleting the oft recurring tune on the ipod. Thus, a whole self help philosophy emerges from the latest must-have consumer durable.

Here is an edit of the Levy article.

____________________

My first iPod loved Steely Dan. So do I. But not as much as my iPod did. By 2003, among the 3,000 or so songs in my iTunes library, I had about 50 Steely Dan tracks. Yet every time I shuffled my music collection "randomly" to mix the tunes, it seemed that the Dan was weirdly over-represented. Meanwhile, it began to dawn on me that there were songs, and even artists, that my iPod had taken a dislike to, if not a formal boycott. Where was Van Morrison? His work was in abundance in my iTunes library, but in my iPod's marathon rock fest, the Belfast Cowboy was perpetually waiting in the wings.

I made it a point to ask iPod owners if their beloved little units were judicious in distributing the songs among various artists or whether they played favourites. People would generally respond with a sigh of relief. Yes! Someone else has noticed! From the results of this admittedly nonscientific survey, it appeared that nearly everybody's iPod seemed to have a favourite artist, or two, or three. After I wrote about [this] in Newsweek, my inbox was flooded with emails - iPod owners were taking serious note of what happens in shuffle, and virtually all of them seemed to think that something funny was happening.

Apple insists that there is no computational flaw in its execution. "It is completely random. It is absolutely, unequivocally random," says Jeff Robbin, one of the original authors of iTunes and later head of the iTunes development team.

Robbin is talking randomness in terms that software can reasonably produce, which is not perfect randomness. Let's look at the way shuffle works. First of all, note what it doesn't do - it's not like mixing all the songs in the equivalent of a big bucket of lottery balls and picking out the next one. Instead…it shuffles the entire library so as to reorder them, just as a blackjack dealer shuffles a deck of cards. If you listen to the entire library all through, you will hear every song once and once only.

True randomness, it turns out, is very difficult to produce. This subject was most famously examined by Claude Shannon, arguably the Father of Randomness. Basically, he defined randomness as a question of unpredictability. If a series of numbers is truly random, you have no possible way of guessing what comes next. If something isn't random (as in the case of what letter might follow another in a message written in English), you have a better chance of figuring out what comes next. That's why it's so crucial to remove the natural redundancy of language from an encoded message and make the coded text look random.

But perfect randomness is an elusive ideal. For instance, if you're flipping a coin, a minuscule weight imbalance might, over the course of millions of tosses, make heads come up slightly more than tails. And if you're randomising on a computer, you have to introduce a "seed", which is a starting point for the algorithm that mixes up the selections. The seed must draw on some unpredictable input of time that begins outside the computer. Otherwise, the results would be the same over and over again.

Paul Kocher, CEO of Cryptography Research concludes that Apple's claims of a high degree of randomness are almost certainly valid. Another expert I consulted, John Allen Paulos, a Temple University mathematician, agreed. He wasn't surprised, though, that iPod users were questioning whether the shuffle was random. "We often interpret and impose patterns on events that are random," he says. "Especially with something like songs. Songs evoke emotion, and some stick in our minds more than others."

Steven D Levitt, the self-described "rogue economist" who co-wrote the bestselling Freakonomics, also fell into the trap. Writing on his blog, he professed constant surprise at how often his iPod shuffle "plays two, three or even four songs by the same artist, even though I have songs by dozens of different artists on it". But as a statistics maven, Levitt understood that the bottom line is that "the human mind does badly with randomness".

Indeed, says Kocher, "Our brains aren't wired to understand randomness - there's even a huge industry that takes advantage of people's inability to deal with random distributions. It's called gambling."

So why does Autofill produce nine Springsteen songs out of 188? Because that is what almost always happens in normal distributions of items from databases. Clusters of something are to be expected. Here's a classic maths trick: gather 40 people in a room and have everyone write down the day he or she was born. What are the odds that two people will have the same birthday? Nearly 100%. It sounds like a coincidence, but mathematicians will tell you it's much more unusual for there to be no such clusters.

We perceive trends when there are none. Poker players invariably believe they can lock into streaks. Backgammon champions swear that dice can go hot or cold. Likewise, people think they can cosmically predict what song will come next on their shuffle.

he seemingly magical effects of the shuffle function - a spooky just-rightness, even brilliance, that comes from great song juxtapositions - were also consequences of randomness. And, in its own way, that was much more disturbing.