Most Wanted Game-Improvement Iron: Beyond the Data

Our test of 2015 Game Improvement irons was our first large iron test in over 3 years. It goes without saying that we were excited to share our results with you and listen to your feedback…your questions, your comments, and even your rants.

Our goal is always to help you find the best equipment to put in your bag, hopefully we succeeded.

Now that the dust has settled, we wanted to take a few minutes to respond to your comments/questions, and share some of the more interesting things we learned during this test.

It’s time to go beyond the data.

My favorite didn’t win, your test is invalid, and you are a moran, etc. etc.

We see one or two or seven(teen/ty) variations of this comment with nearly every most wanted test. Hey guys, I’m sorry (not sorry) your favorite didn’t win. We collect the data with our Foresight GC2 Launch Monitors, we analyze the data, tally scores, and the proverbial chips fall where they may. If it’s any consolation, my personal favorite of this lot, Srixon’s Z545, didn’t win either.

I really am a moran.

Regarding the actual order of finish – and some comments suggesting some found the results surprising…

We rank clubs top to bottom based on the absolute numbers. That said, as readers evaluating the results we would encourage you to consider a couple of things:

1) We look for the club we think will perform best for the fattest part of the bell curve. Basically, we look at a balance of performance characteristics. If you’re a distance guy, it makes perfect sense that you would analyze the data with an emphasis on distance. Conversely, if distance isn’t important to you, you might give it zero weight when you consider what might be the best iron for you.

2) Dissecting the results requires a bit of what I’d call statistical common sense on everybody’s part. We can split hairs over tail length, but performance data is relatively normally distributed. It’s bell curve stuff, and so what we almost always see is 1 or 2 standout performers (sometimes 2 sigma), and 1 or 2 that lag behind, and everything else in the middle can be considered statistically equivalent. More often than not there’s no significant statistical difference between #3 and #7.

We had a couple of questions about specifications…did we check them, and is it possible that a manufacturer hand-picked the clubs that were sent.

To the second part…yeah, absolutely, it’s possible. We think it’s more likely with drivers where distance is the largest concern, but it’s possible.

Regarding Kflare’s question about the actual checking of specifications. We didn’t do that this time around. We did (many years ago) look at loft and lie angles. What we found that almost nobody was on spec, but most were within tolerances. I’m won’t say too much about our plans for the future, but sufficed to say, we have some things in the works.

Dr. Bloor and perhaps one or two others either asked about mats or suggested we use real grass.

Perfect world, everybody uses grass, right? Also in a perfect world grass is indestructible, or at the very least, replenishes itself over night. Our world is not perfect. Take 20 testers, 10 sets of irons, 3 or more clubs from the set each…you’re destroying a lot of grass in a short period of time, and relocating your launch monitor (and your target lines) frequently. Now test blades. Now test better player cavitybacks. And now test something in-between the player’s cavityback and a game-improvement iron. Toss in hybrids, and fairway woods, and well, before long (well before we’ve tested everything that will become  part of our regular testing schedule) not only are you out of grass, the superintendent is pissed, and you’re not invited back on the golf course.

You quickly see the infinite practicality, and arguably the necessity of mats.

While the evidence does suggest that even a high quality mat like our Real Feel Country Club Elite isn’t exactly the same as hitting off grass, it’s a reasonable assumption that irons are impacted proportionally. Some, I’d wager it’s a sizable majority actually, of the best fitters in the world rely on quality mats when they do their iron fittings. From our perspective, it’s a matter of both practicality and consistency, as the mat ensures the same lie every time.

Sets vs. Clubs

A couple of you noticed that there appears to be very little correlation between the performance of individual irons within the same set. Our response: Yeah…ain’t that something?

As our data crunchers have pointed out, with some sets (RSi1 and G30 spring to mind) there does seem to be a correlation in performance at all distances (basically results are similar at every distance), where as for other sets, the numbers are all over the place.

Yup.

During testing we observed, and tester response confirmed, that some manufacturers in this space have done a much better job creating sets, while others have basically assembled a collection of individual irons.

As an example, we had one tester who absolutely loved the Tour Edge Exotics E8 middle iron (and his performance with it supported that affection). With the pitching wedge, the experience was totally different. He hated it…the look, the feel, and the performance. One was not like the other.

Similarly, I found good flow from the long to middle irons in the Callaway XR set, but the pitching wedge looked totally out of place. Continuity of badging aside, with a significantly rounder shape (to my eye anyway) it looks like it was designed for an entirely different set.

As a result of these observations, I find myself questioning the entire consumer buying experience. We’re attracted to the look of a club (and let’s face, for many golfers that doesn’t extend far beyond the badge), but in most cases the demo, and even fitting experience, is limited to single club (generally a 6 or 7 iron). That may be well and good…it’s how it’s always been done, but as we found, there’s not much assurance that, on a comparative basis, you’ll hit a long iron, or even the pitching wedge with the same proficiency that you do a mid iron from the same set. Even though they’re from the same set, they may not play the same.

My advice…if the dealer/fitter won’t let you hit at least 3 different irons from the set, walk out and find someone who will.

The Loft Conundrum

To reiterate, this was a test of game-improvement irons. Generally speaking, the target golfer for this particular class of club needs forgiveness, benefits from some offset, probably feels more confident standing over a larger head, may need some help getting the ball in the air, and almost certainly wants a bit more distance.

How do you cram all of that into a single club? Longer shafts, lower lofts, progressive design, lower CG, yada, yada, yada.

The real answer based on our results? You don’t….you probably can’t.

You’ve probably heard the popular OEM story line…with modern design (low CG, back CG), if we left the lofts the same as they were in yesteryear (or perhaps yore), they’d launch too high, spin too much, and not go anywhere.

So is that bullshit? Absolutely…and absolutely not.

I hit the ball high. I hit it so high that one fitter recommended that I take a set of what some of you would call ‘loft-jacked’ irons and bend them another degree or two strong to bring the ball flight down. Even with strong lofts, I hit the ball too high, spin it too much, and lose distance as a result.

I’m one end of the bell curve.

The manufacturer’s philosophy is probably spot on…for the middle.

At the other end (and this end probably has the longer tail) we have moderate to slow swing speed players who simply can’t get enough air under the ball to hold greens with any regularity. I’m talking about a segment of golfers who, more often than not, can’t generate 6000RPM of spin with a pitching wedge. By way of comparison, my average spin rate with a pitching wedge, is right around 10,000RPM. Easy math…these guys generate 40% less spin. FORTY percent…with the same club.

For these lower launching testers, we observed 6-8 yards of roll (nearly 25 feet!) with a pitching wedge, and proportionately more still with middle irons. These are guys who have learned to play the ball that way, but it’s certainly not what I’d call ideal.

The problem for golfers like these is that within the larger marketplace, nobody is offering launch and spin…not if it means reducing distance in favor of playability. What we saw suggests that these guys (and there are plenty of them) need modern technology with traditional lofts. Nobody is making that available to them right now – though I suppose bending (a lot) is an option.

It’s an understandable problem. Even for guys who desperately need it, how do you make money peddling high launch and generally better ball flight to the exclusion of distance?

Is Shot Area the Only Thing that Matters?

A couple of you, most notably Steven Clark, latched on to shot area (our 90% confidence ellipse) as perhaps the most significant performance metric. If not for marketing (and lets not forget that effective marketing plays to what the consumer wants to hear – the market wants distance), we might have focused on shot area to the exclusion of everything else.

The reality is that distance has become part – inarguably a large part of the sales pitch – around game-improvement irons. So within that context, we didn’t feel we could ignore it (even if we think you probably should). Distance had to count for something.

Radial distance (distance to the pin) is ingrained in how we think as golfers. How short (or how long) of a putt did I just leave myself? How big of a difference can one iron make over another? We thought that was extremely interesting (and unquestionably easy to understand).

All of that said, we love the shot area metric because it includes information that, while perhaps not as easy to digest, is not wholly dissimilar from radial distance, and creates a more complete picture. It ties in elements of both accuracy and consistency, which is obviously valuable.

The thing I really like about shot area is it overcomes some of liabilities inherent to not fitting every tester.

We test off the-the-rack, standard configurations, and while we tried to eliminate highly non-standard golfers from our testing pool (nobody that’s 4° upright, and 1″ short for example). It’s also not out of the realm of possibility that a golfer who is properly fit into a 1° upright spec for one iron, might be standard or even 1° flat in another.

We live in the middle.

So by considering shot area as a significant metric we not only better identify situations where what, at face value, appears to be an accuracy issue that is actually just a minor issue with lie angle, we can also avoid over penalizing an iron that shows very tight dispersion that just happens to be consistently left of the target.

Shot area presents the best view of what we might expect on the golf course.

Would you be willing to give up a foot or two of pin proximity in exchange for fewer shots that fall short, or fly long? There’s often trouble to be found near the green, and the irons the give you the tightest dispersion are the ones that give you the best chance at avoiding it.

It’s not my intent to overstate shot area as some groundbreaking new measurement we invented. The calculation is built into every enterprise class launch monitor. Golf companies use it, fitters use it too. Average golfers hitting balls on big box simulators or out on the range at demo days…I’m not so sure. Too often we focus on the finer details (distance, yards offline, etc.) to the exclusion of the bigger picture.

Shot area is that bigger picture, and you absolutely should be looking at it.

DIY Analytics

Our data is absolute. That is to say it’s exactly what happened. Guys hit shots, the data gets collected, we import it into Excel and process it. Sure, we drop the really bad shots, but otherwise the data is the data. How that data is interpreted…now we’re entering a realm of infinite possibilities.

The reason why we share data…the reason why we’re looking for an efficient means to share even more data is because we want you (at least those of you who are so inclined) to play with the data, look at it in different ways, and even reach your own conclusions.

One reader did exactly that, and that’s awesome.

Steven Clark decided to do his own analysis. You can read the details in his comment here, but in the true spirit of MyGolfSpy, he summarized his findings in a graphic:

Dr. Clark starts with the premise that Shot Area is what matters, looks at Z Scores, and distance between individual irons within the same set, and then determines the best overall irons, the best irons for distance with longer gaps between irons, and best irons for those looking for tighter gapping.

As we said, we shoot for the middle of the curve, but different golfers want different things from every club in the bag, this type of analysis helps zero in on those sort of specific desires.

From my perspective it’s just really cool to see someone grab our data and do his own work with it.

Thanks, Steven! Awesome job.

The Evolution Continues

As with every test we’ve ever done, the value is at least two-fold. We identified what we believe is the best game-improvement currently on the market (RSi1), and, as is always the case, we learned gained a tremendous amount of insight into how irons actually perform for different golfers. We’ll take what we’ve learned from the test, and what we’ve learned from guys like Steven and use that information to improve how we test while developing new ways to look at the data.

The best is always yet to come. To steal a line from Nike, there’s always better.