The article starts with, "Human beings try to find patterns to explain the reason behind almost every phenomenon, but that doesn’t mean that there is a pattern to rely on." The second part of the sentence is true, on further observation some patterns prove not to be patterns. However the first half, the attempt to explain why we "find patterns" isn't convincing.
Instead, replace that phrase with this: "based on brain construction, we humans are predisposed to find patterns in data we encounter". This idea holds up to evolutionary scrutiny, organisms have mechanisms biased for self-protection and finding food and other resources through various forms of pattern recognition. IOW we find patterns we're neurologically capable of finding, particularly in regard to survival and reproduction.
Perhaps it's more accurate to assert we "find patterns" useful in making predictions about our present and future state within the environments we occupy. The fact that patterns may not turn out to be authentic is simply part of a process of refinement of pattern-seeking and improving value as predictors of future states. We may call a pattern an "explanation" but nothing is actually "explained" as shown by the fact we reserve the right to enhance or revise the "pattern", or what we insist it predicts, at any given time.
Our brains are basically built to do pattern matching. Though, truthfully, I find that is too abstract of a way of putting it, and in fact, people tend to associate "meaning" with "pattern" and thus you end up with statements like that one.
What I would say the brain is good at is finding patterns in terms of identifying what is appropriate, with a particularly general understanding of "appropriate." There's a huge evolutionary drive for this. It's a bit disjointed to say wolves howl with each other because "brains detect patterns" and somehow find them useful. It's much clearer when we say that "brains encode patterns in terms of appropriateness", and thus wolves howl with each other because their brains know it is appropriate to do so. Just like germ's biology encodes that it's appropriate to wiggle harder in salty water, or how a person knows that certain words are correct in specific situations.
So our brains are more like engines for mapping patterned associations to feelings of appropriateness in context.
Appreciate your interesting thoughts on the subject. It certainly does appear that brain circuits are evolved to respond to patterns in data, for example, visual neurobiology has been studied extensively re: how visual systems can identify patterns in visual data.
Not exactly sure how you define "appropriate" in this context. Like an astronomically complex "neural net", the brain integrates "input" into pattern recognition, so it's a form of classification filtering that allows recognition of phenomena. On the basis of experiences, the classified patterns are bound to probabilistic prediction estimates, in turn informing choice of actions.
Though I know much less about wolf vs. human behavior, if analogous to human speech, howling is an action taken in response to evaluation of current environment, e.g., pattern of other wolf activity, presence of prey, etc. We might infer howling is "appropriate" under some condition, but really it's tautology, because it's equivalent to stating we observe howling under some condition. IOW the latter is a description that's complete and sufficient regarding what we "know".
It might be clearer to rephrase "appropriate" to "what is and what works", though "what works" is probably superfluous to understanding the process. IOW an attempt to "explain" the pattern recognition and response phenomena adds nothing to our knowledge, and in fact adds a level of indirection that tends to obscure the nature of process.
I should let it go at that because it's late and I'm tired. Saying more leads to multi-dimensional consideration of the nature of brain/body operations. If you are really interested in arcane and slippery meta-level or higher-order viewpoints I'm happy to say more, but better when it's earlier in the day and my head is clearer.
You're right in that it doesn't seem to add any information. It's possible that there's nothing more to learn, but it's also possible that the distinction would manifest in a subtle way that your current formalized understanding doesn't employ sufficient granularity to capture.
I guess I prefer the term appropriate because it more cleanly handles the case when the patterns "don't" match: by indicating that such situations don't exist. There's always a pattern to match, just as there's always some action that's learned (or instinctually) appropriate to any given situation.
If you're trying to see appropriate as "what works", then you're probably more strongly aligning the meaning with a rational process than what I intended, but I don't think you're wrong. Either way, it bears keeping in mind that our day-to-day language isn't really optimized for discussing these kinds of things, so there's bound to be multiple layers of confusion.
> Either way, it bears keeping in mind that our day-to-day language isn't really optimized for discussing these kinds of things, so there's bound to be multiple layers of confusion.
After decades-long study of human behavioral phenomena, I'm striving to articulate what I've learned in coherent written form. It's proving difficult to transform a non-linear multi-dimensional model into ordinary English prose that readers can comprehend. So I absolutely agree with your comment about limitations of ability to reduce mental models to common language.
The issues you bring up concerning formalized models that allow mapping behavior to determining factors are indeed of central importance. A model must permit sufficient granularity of analysis, at the same time covering sufficient generality without contradiction of the granular level. The hard part is describing the interactivity of this whole range of "levels", because the immediate and the distant elements are in fact occurring simultaneously and affecting the system under observation in real time. It gets convoluted when we realize the observation itself has effects on the observed behavior.
The problem I have with "appropriate" is the term's ambiguity. OTOH "pattern" implies there's a "match" or there isn't. (I know, patterns can be iffy, but then they're not quite a pattern.) Encountering a situation that's unclear, where no "matched" pattern is evident, immediately arouses alarm. Then we proceed with caution until observing enough that something "familiar" is gleaned, or observe/interact enough to establish a new pattern.
This state of "I don't know" is constantly implicit, patterns never match perfectly, details always vary. Most of the time that's overlooked because we accept a "close enough fit" to established patterns, that is, categorical classification is an abstraction that works adequately most of the time.
For example, often it's good enough to say "that's a tree" without saying what kind of tree. But other times it's important to distinguish a fir from a pine from a hemlock. Patterns are infinitely divisible, ultimately no two trees are identical, at some level of refinement abstractions break down and no longer apply. A thing is no more or less than its actual attributes. Though indispensable for human existence, abstraction is just a tool, pattern recognition is a built-in mechanism of abstraction, best to remember all tools have their limits.
I certainly would never say there's no more to learn, just that defining terms is only a tool for communication, not to be confused with the information we attempt to share. We get confused when we think we are "explaining" phenomena that we observe. In reality, it's less confusing and more informative to simply describe what we observe. Curiously, thoroughly observed phenomena are the things we tend to call self-evident or self-explaining, which suggests an explanation is only an expression of uncertainty about patterns yet to be adequately elucidated.
>As an analyst, one needs to keep in mind that the Journey is more important than reaching the Destination.
I'm really not sure what to make of the last line. The goal of analysis should be to produce results that are actionable. In the end it should matter very little how they are obtained as long as they are accurate.
Or at least, if you continue to use the metric that diagnosed the problem while trying to address it, you're going to get what you measure. With most groups I've been with, they were relieved to get the metrics good enough to detect anything at all, and don't have the stamina to come up with a more accurate way to determine the same thing.
A lot of times an action is the product of many analyses that individually aren't actionable. To me it's similar to managing fixed vs. variable costs. You've got to make investments of various size, but the payoff needs to correspond to the investment.
I wish the article was on Apophenia, it would be more fun.
Instead it's not clear to me who the target audience is. The phrasing makes it appear to be targeted at analysts and not their business partners. Assuming that to be the case, senior analysts are substantially beyond the level this article is written at (or should be).
Entry level analysts need close supervision to prevent them from making these, and other, mistakes. The examples the author draws (specifically cheese vs. infant mortality and the google flu approximations) don't do a good job at identifying when this issue arises. For the cheese example it's unclear if the phenomenon is real or not (the magnitude of the variation in infant death may actually be significant, if the cheese consumption variation was small then there would be a different story). The author does nothing to help the reader resolve this.
In the Google flu example it's only through hindsight (and colossal failure) that the author identifies the lack of validity in Google's model.
I agree 100% with his point but I don't think the article is providing much value because essentially the author is simply saying: "be aware, this type of problem exists out there..." without providing information necessary to navigate/resolve the problem.
Can anybody reading this comment please enlighten me on good algorithms for optimum bin sizes for histograms? I have tried DW Scott's (1979) method. But are there any new better kid in the block?
Our brains are wired to believe a plausible story easier than the one told by the base rate or stats... We need to be aware of the gaps our brain fills by itself. Definitely not easy, though...
Words and concepts exist for a reason. The author of the post could have turned up the (well researched and imho fascinating) antecedent work very quickly simply by googling his piece's title. That he did not makes me sad. Very smart people have previously studied most interesting problems. Ignoring their work is both arrogant and foolish.
Put differently, you don't learn much by talking, but you learn a lot by listening. You learn a little by writing, but you and your readers learn much more if you read up first. (I am not assuming you wrote this. The themes are general.)
If I wrote a piece that ignored a decades-old, well-known, fundamental result, not only would editors and colleagues slam me for it, I'd be ashamed of it myself. I went back and skimmed a few more of this author's posts and I have to say, they're not of a quality I would suggest to students. If they happen to read this, I hope they'll talk with someone who has a little formal training and revise their work.
I'm tempted to point out that you went from zero to Godwin in record time. To assume that Konrad's party affiliation (likely a pragmatic choice at the time) either validates or invalidates his work is a bit absurd. The issue as I see it is one of the modern meaning, which is to say, a tendency to see patterns where none exist.
Nonetheless, let us see whether Konrad's work has been dismantled in more recent studies. A quick trip to PubMed suggests otherwise:
Note that I did not claim Konrad's result was fundamental. I stated that if I ignored a fundamental result in my writing, I would be pilloried. Konrad's theory of prodromal schizophrenic ideation beginning with this tendency to see patterns where none exist is perhaps of interests to psychiatric historians, but it is not what I would call fundamental. I would claim that if the perfect term for a concept exists and is established, one should use it.
Apophenia, in its now accepted colloquial meaning, is an exceptionally handy shorthand for what the writer describes. Like other swell ideas (Gaussian processes, natural selection) it is so useful that it has accumulated several names (as a parallel, kriging and clonal evolution are alternate). Either would work fine here.
I didn't say his party affiliation validates or invalidates his theory. I said that a published paper was not able to validate it empirically.
The big issue I have is that it seems impossible to define a general concept of "pattern", such that one could claim it doesn't exist. If such a concept cannot be defined, then it is unclear what the concept actually means. The only thing I can imagine that could possibly fit would be if it meant that a person generates a theory and testable hypothesis that is able to invalidate the theory, and fails to find any supporting evidence when the hypothesis is tested, but still claims that the theory was validated in a way that is demonstrably false. Personally, I don't think psychiatrists are so invested in their patients that they would actually carry out such tests.
The article that you linked primarily recounts a single anecdote where Konrad seems to assume very much about his patient (also, it was unclear about the circumstances; do you know if the anecdote occurred during Nazi rule?). No mention is made of Konrad attempting to verify any if the patient's claims.
What result are you claiming as fundamental? The entire theory seems to be self defeating, as a pattern was suggested that has not been able to be verified.
The fundamental result behind the gambler's ruin is the human tendency to perceive patterns in genuine randomness. Rorschach blots, lotteries, slot machines, most betting endeavors all work because some humans will always find patterns in randomness. The result is easy enough as are the experiments (generate random noise from Uniform(0,1), project it onto a suitable manifold, hire some undergrads or local homeless people to look at them). That's not at all what I claimed Konrad originated. If you want an origin for this type of thing, de Finetti or Laplace or Descartes might be some candidates.
Konrad did give the phenomenon a catchy name and proposed that an increase in this tendency is an initial step in developing schizophrenia. If someone could actually establish this at a neurogenetic level that would be impressive and fundamental; I'm not aware of anyone doing so. I'd expect it to show up in a CNS journal and NIMH or WT to make a big deal if someone did.
The contrast between epiphany and apopheny is so striking, though, and so relevant to this topic, that it annoys me to no end when it is ignored. At the base of all of statistics is a desire to quantify how much of each is present in an observation, experiment, or cyclic series.
As you probably guessed, I am an applied statistician, not a neuroscientist. (I have serious issues with the way statistics are misused in neuroscience, for whatever that's worth). I do not, and cannot, claim that Konrad's theory is fundamental to that field. I do claim that anyone attempting to explain statistical reasoning to a lay public ought to internalize the contrast he proposed. Its setting as a proposed turn towards insanity is just a happy historical note.
>The fundamental result behind the gambler's ruin is the human tendency to perceive patterns in genuine randomness.
I don't think pattern recognition drives most gamblers. There are all kinds of other benefits, perceived or real, that are not accounted for in a purely monetary payoff grid.
I don't know how you can generate random noise. I assume you are using a standard, pixelated display to read this message. Even if a random process was choosing what to display on that screen, there are only a finite number of configurations. Exactly what you are viewing now could be recreated by such a random process.
The problem that hasn't been addressed yet is that "pattern" is not well-defined. If a random display shows a horizontal line pattern, it is still a pattern by some definition (and you would have no way to distinguish it from a "intentionally patterned" display that has the same configuration).
Of course you could have an intersection between recognizable images and random noise. The odds of this occurring with any regularity are infinitesimal, hence the design of experiments. (Recall that there is nothing like a mathematical proof in the physical world -- at a molecular level, some water molecules are moving upstream at any given moment, but by reaching into a stream or tossing some objects into the flow you are taking a large enough sample to determine where most of them are going).
Since you're not going to get proof one way or another, all a well designed and experiment can do is give you evidence. This happens to be more valuable than just about anything else that science has come up with, but it isn't proof.
Which is why the gold standard for a result is replication in a large sample. I could have this very page generated by convolving couple of high entropy random streams. Is it likely to happen repeatedly? Not if the generator is any good. Same principle for randomized trials. You can end up with unbalanced arms (I'm proofing a manuscript where we had exactly this problem). But it's unlikely that they'll be consistently unbalanced across trials with sufficient sample sizes.
Instead, replace that phrase with this: "based on brain construction, we humans are predisposed to find patterns in data we encounter". This idea holds up to evolutionary scrutiny, organisms have mechanisms biased for self-protection and finding food and other resources through various forms of pattern recognition. IOW we find patterns we're neurologically capable of finding, particularly in regard to survival and reproduction.
Perhaps it's more accurate to assert we "find patterns" useful in making predictions about our present and future state within the environments we occupy. The fact that patterns may not turn out to be authentic is simply part of a process of refinement of pattern-seeking and improving value as predictors of future states. We may call a pattern an "explanation" but nothing is actually "explained" as shown by the fact we reserve the right to enhance or revise the "pattern", or what we insist it predicts, at any given time.
Edits: grammar and clarity