Time frames of, say, 365 days, and 250 days, are useful but more importantly perhaps, the shorter time periods of 100 days, 30 days, and 7 days really need to be looked at.

So, what I did was set everything up to display 200 days, 100 days, 30 days and 10 days of stock correlations. For each stock, the top 5 positive correlations for each time period and the top 5 negative correlations for each time period. I decided to go with a 10 day period instead of the 7 day period because we are working with trading days, not calendar days – 5 trading days in a week. Not really important, because obviously the 10 days cuts across calendar weeks – it just seemed nicer to be looking at two trading weeks at a time.

I had also mentioned that DeepMarket doesn’t have index data and Jon asked:

Not having index data is a problem, as sometimes it would be useful to look at these correlations in the context of the market. Is it not possible to use another source? What about Yahoo! they have a lot of available historic and recent data.

Well, this is a question of MAY I vs. CAN I. There is no question that I CAN, but I MAY NOT display that data here on DeepMarket. I don’t have the rights to display that data – sorry! To show that data is really expensive and I only display the data that is in my feed. The exchanges (who own most of the indexes) are really weird about it and I was looking for data with my “path of least resistance” algorithm – so, what you see is what you get. Don’t even ask about mutual fund data…

However – you can look for proxies. A fantastic proxy – and probably the most famous is the triple Q or QQQ or QQQQ – which is the NASDAQ 100 Tracking Stock – check the link for a better understanding. Bottom line – its moves are correlated with the NASDAQ 100 Index – and an excellent proxy for the index. Of course, in this case the movements are not only correlated, but causal in relationship to each other. So, use our little search tool in the upper right corner and see if you can find a proxy for the index you are looking for.

Okay – that is a ton of data out there for all the stocks. I am not sure if it is helpful – data is always the first step. Now that we have all this data, I hope people are able to come up with some interesting strategies. Jon gave us some clues and hopefully others are willing to share their ideas and insights – I welcome comments!

If there is no preceding trend, then why should we expect a trend reversal? If you are looking for price declines after the head and shoulders pattern completes, you need to identify the conditions where the stock has a preceding uptrend on the same timeframe as your h&s pattern.

Oooooooops. He also says:

For the purposes of head and shoulders analysis, I recommend we define a prior uptrend as a series of higher highs and higher lows (you can identify this with correlation too).

Dang it – I messed up that last experiment in looking at head and shoulders, so we will re-do it to see what difference it makes. Thanks for pointing out my, uh, ignorance!

I am beginning to play around with quantifying the predictability of patterns, such as the head and shoulders pattern. There is a lot of analysis, running of data and thinking that needs to be done, but I wanted to talk a little about what I am seeing so far. To really understand the usability of certain patterns, you really need to slice and dice the data in various ways to understand what tendency the market/stock/index/whatever might have after a pattern presents itself. As an experiment, I pulled closing data from about a thousand stocks that I had over 10 years worth of data. Presently, I am just trying to get a handle on the H&S pattern across ALL stocks, so I figured I would ignore individual stocks and just look at some global numbers.

I decided to be very simplistic (and practical) and look at the seven day H&S pattern. I am making a HUGE assumption in this next step – I am assuming the general tendency of the pattern will play out within seven days (again, seven to match the length of the pattern). In the future, I plan on looking at days one through pattern length X 2 – or something of that nature – to give use a better understanding of how a pattern evolves once it is recognizable. So, here are some early numbers:

- The average stock showed a 7 period head and shoulders pattern every 349 trading days
- That equates to about .4% of all stocks having a head and shoulders of length 7 at any one time

Okay, those make sense. It seems that patterns like this are relatively rare.

- It seems that only about 400 stocks out of 1000 had a tendency to move DOWN seven days after an H&S pattern

Ooops…

The “head-and-shoulders” pattern is believed to be one of the most reliable trend-reversal patterns.

Okay, let’s dig just a little here:

- Of the 400 odd stocks that showed a tendency to go down after an H&S pattern, the average seven day loss was 1.7%
- Of the 600 remaining stocks that showed a tendency to go UP after an H&S pattern, the average seven day gain was 2.45%

Okay, that is just a little taste. I have plenty of work ahead of me to really see what is going on. Also, if anyone sees any problems in my logic or has questions about how I am calculating things – let me know. I am skipping a lot for ease on my part and trying to get to the deeper issues, but details can be important and that is where the mistakes can be made!

I got some really good questions from Thomas Czaszyski, who writes a stock investment and research blog. Thomas asked the perfect question:

This head and shoulders, I understand how to get it and everything now, but what does it mean?

Exactly! Now, I know the classic chartist answer, but I am not interested in regurgitating the same answer a chartist would give. Luckily, Scott pointed us to the book Encyclopedia of Chart Patterns, so I would recommend Thomas check that reference. Or do a quick look at head and shoulders described by chartists. Thanks for the pointer to the books Scott – I certainly want to take a look at how everything was quantified. My major concern is – how did they identify the patterns? Were humans finding the patterns?

Also, to Thomas – good luck with Princeton! Don’t be surprised if you find that most academics don’t truly understand the stock market and trading, let alone technical analysis.

So, let’s review. I am not a chartist or a technical analyst, but am interested in understanding patterns that seem to have “legendary” status among those who dabble in these arts. One of my greatest concerns is that there is little work on validating chart patterns in an objective manner. So, for instance – how reliable, profitable and practical are these patterns?

Well, before we can approach those questions, we needed a way to detect patterns in a repeatable fashion. Pattern matching is a fascinating field, but we just needed a quit and dirty solution. Correlation was that simple solution. What was not described was how we are using correlation to find patterns in stock charts. We are going to use some data that never changes (a stock in the database that has stopped trading) and Excel to show this simple process.

AMNT stopped trading on 12/12/2005 and just so happens to have a really classic head and shoulders pattern in the last seven days of trading:

Using Excel, we can describe the process that is used to find the head and shoulders pattern (or any other pattern – given a template of the pattern). First, we need a template of the pattern we are looking for in the chart data. I explored the charting and technical analysis literature and “eye-balled” the patterns people were describing as head and shoulders. This is far from scientific, but a good start. What I came up with as a template for head and shoulders is:

Or, very simply using numbers from 0 to 100:

0
60
20
80
30
70
20

Seven data points gives us a template for what I think a head and shoulders pattern would look like. This template is certainly open to discussion, but this is more about the process than the correctness of the template. What is important is that we can use the above template to easily and quickly identify that general pattern. We have a template of 7 data point, so we need to look at 7 data points from the stock data we wish to search for patterns. We are using AMNT and the last 7 data points (looking at closing prices) are:

10.22
10.25
10.24
10.28
10.24
10.26
10.24

Now, open up Excel and paste the two columns of data into columns A and B of your worksheet. It doesn’t matter which column you put the data – template in A or template in B:

Now, using the Excel function CORREL we are going to find the correlation between the template and the stock data. Put the following formula into any cell that isn’t already being used:

=CORREL(A1:A7,B1:B7)

The result is a correlation of .940712, which is a pretty good match. You look at the pattern’s template and the actual stock data and see that it is a pretty good match. Pretty simple, huh?

For what we are doing, we have this process written in Java and every morning we search for patterns that have a correlation greater than .85 – why .85? No reason, just an arbitrary point. For those who wish to do this for themselves in other languages or technical analysis packages, you just need a correlation function and the ability to use that function to compare two sets of data. Once you have that setup, then you can play around with any number of templates describing different patterns and correlation cut points.

And in that spirit we put up a new page which lists stocks with potential Inverted Head and Shoulders chart patterns. To be honest, when I look at these patterns they do not seem to be as disctinctive as the regular H&S formations. Any chartists want to give us some feedback?