Why, exactly, would Karl Popper disapprove of this application
of the notion of falsifiability? A stock market system is a
theory about how the stock market operates. In other words, it's a
theory that says that the application of a certain code with
certain values is likely to be in synch with the market and
therefore produce profits (because it has isolated a pattern in
stock market behavior). As opposed to those on this board who
want the system to work so badly they "trust" it or have "faith" in
Matt, I simply said that all systems can fail when they are applied
to real markets. Consequently, I asked Matt to discuss the
specific definition of failure in this context, in other words,
what would have to happen for the system to be deemd a failure (for
example, that the system underperforms the market by a certain
percentage, that it fails to produce profits, that it doesn't match
its backtested performance, and if so, by how many percentage
points, that its profit curve starts to move sideways or down,
etc.).
I suggest that all purchasers of black box anything ask the
seller this question.
Algyros -- my take on your question which I think is a good one
is that my guess is that it's not likely to be a black/white
situation. I think the most likely thing which Matt has already
alluded to is that in the face of new data indicating the system is
not working as well as hoped for Matt will open his toolbox and
tinker with the system to improve the historical performance
including the most recent time frame. IMO it's a real double edged
sword -- will we ultimately end up with something that we can leave
alone and trade with successfully for years? Or will we be in a
situation of frequent smart and well intentioned adjustments that
make it so that we are often trading a "new and unproven system"??
I hope whatever adjustments are made the system will be tested for
"robustness" -- if there's a rule that involves a 15% value then
changing to 13% or 17% say should not make a lot of difference for
example. The system will be more robust if the same system works
for DIA and IWM. If the system only works for SPY and has rules
like a 12.27% profit stop it seems more likely to have problems.
Just my two cents.
Yes, those are my thoughts. Robustness is hard to measure,
but one measure is to use different insruments.
My one major fear is the tinkering approach to the system.
The same sort of thing was done with the GDX system,
resulting in a plethora of rules. I think that the new,
simplified GDX system should be kept unchanged so that we can see
how it works. The same holds for the SPY system. If the
system undergoes constant tinkering, then it will always remain in
backtest/optimization limbo and we'll never know how good it is.
The system needs to be finalized and then left to trade.
That's why I think that adding options as hedges is a good
idea. As Vida says, it will increase the cost basis, but it
ultimately help the system by providing counter-trade insurance
without having to be stopped out. But, my point is that
adding hedges won't require code tinkering, so we can still use
April as the point from which to judge the system.
Posted by dkirk68947 on 10th of Aug 2011 at 03:13 pm
my loss is greater than that but I still have hopes market turns
soon and we can ride it back up to recover some or
all of our
losses. Had I not depended so much on the record of the spy
system and the black box method I would never had allowed this kind
of a loss.
Newsletter
Subscribe to our email list for regular free market updates
as well as a chance to get coupons!
Why, exactly, would Karl Popper
Thoughts on the SPY System
Posted by algyros on 10th of Aug 2011 at 04:36 am
Why, exactly, would Karl Popper disapprove of this application of the notion of falsifiability? A stock market system is a theory about how the stock market operates. In other words, it's a theory that says that the application of a certain code with certain values is likely to be in synch with the market and therefore produce profits (because it has isolated a pattern in stock market behavior). As opposed to those on this board who want the system to work so badly they "trust" it or have "faith" in Matt, I simply said that all systems can fail when they are applied to real markets. Consequently, I asked Matt to discuss the specific definition of failure in this context, in other words, what would have to happen for the system to be deemd a failure (for example, that the system underperforms the market by a certain percentage, that it fails to produce profits, that it doesn't match its backtested performance, and if so, by how many percentage points, that its profit curve starts to move sideways or down, etc.).
I suggest that all purchasers of black box anything ask the seller this question.
Algyros -- my take on
Posted by bkout3 on 10th of Aug 2011 at 09:14 am
Algyros -- my take on your question which I think is a good one is that my guess is that it's not likely to be a black/white situation. I think the most likely thing which Matt has already alluded to is that in the face of new data indicating the system is not working as well as hoped for Matt will open his toolbox and tinker with the system to improve the historical performance including the most recent time frame. IMO it's a real double edged sword -- will we ultimately end up with something that we can leave alone and trade with successfully for years? Or will we be in a situation of frequent smart and well intentioned adjustments that make it so that we are often trading a "new and unproven system"?? I hope whatever adjustments are made the system will be tested for "robustness" -- if there's a rule that involves a 15% value then changing to 13% or 17% say should not make a lot of difference for example. The system will be more robust if the same system works for DIA and IWM. If the system only works for SPY and has rules like a 12.27% profit stop it seems more likely to have problems. Just my two cents.
Yes, those are my thoughts.
Posted by algyros on 10th of Aug 2011 at 09:50 am
Yes, those are my thoughts. Robustness is hard to measure, but one measure is to use different insruments.
My one major fear is the tinkering approach to the system. The same sort of thing was done with the GDX system, resulting in a plethora of rules. I think that the new, simplified GDX system should be kept unchanged so that we can see how it works. The same holds for the SPY system. If the system undergoes constant tinkering, then it will always remain in backtest/optimization limbo and we'll never know how good it is. The system needs to be finalized and then left to trade. That's why I think that adding options as hedges is a good idea. As Vida says, it will increase the cost basis, but it ultimately help the system by providing counter-trade insurance without having to be stopped out. But, my point is that adding hedges won't require code tinkering, so we can still use April as the point from which to judge the system.
i sold yesterday at the close
Posted by qwich34 on 10th of Aug 2011 at 10:42 am
i remember the last fed metting.I took a big loss around 10000.
loss on spy
Posted by dkirk68947 on 10th of Aug 2011 at 03:13 pm
my loss is greater than that but I still have hopes market turns soon and we can ride it back up to recover some or all of our losses. Had I not depended so much on the record of the spy system and the black box method I would never had allowed this kind of a loss.