I feel honored to be invited to guest blog on The Race Engineer blog. I made the mistake of updating my LinkedIn profile with some obscure details of my work history and Buddy came calling. He and I have traded EMails over the last few weeks, and some of that content he felt was worthy of the blog. So, here goes.
DOE, design of experiment, was a topic briefly glossed over in my required statistics and probability class during engineering school. Had it been more of a focal point in the class, I probably would have paid much more attention. Seems the biggest problem educators have is making the course content relevant to the audience. I cannot explain the intricacies, or even the theory, behind the method in a simplified discussion. The point of DOE is to build a response model based on several user-decided factors. Like all models, it requires the user to be pretty informed ahead of time of what is expected in the results, but also to be somewhat open-minded when the results diverge from the expected. I’ll leave it up to you readers to do more research on the www on the method and its origins.
DOE was certainly revered in the pharmaceutical industry – probably still is. Types of experiments, the number of factors involved, and the complexity of the experiment are all parts of drug trials in this era. What does that have to do with making your racecar go faster around a given lap than the competition? Working smarter, not harder, eliminating the surrounding static, and concentrating on the most important setup parameters to achieve a result.
In this realm, I am alluding to a method of DOE involving simulation code. There are many basic advantages to DOE in the virtual world. The repeats can all be skipped, the number of factors can be rather large, and you can accomplish a full factorial design with today’s larger multi-core machines. There are other methods of DOE outside the virtual world, but I’ll take the academic copout that those are beyond the scope of this blog, and are sensitive information in a competitive environment.
Now let’s get to the good stuff. What does DOE methodology do for the race engineer?
One of the most useful tools to a race engineer when away from the computer is a Pareto chart based on the results of the DOE. Here’s an entirely fake example of one.
In this case we decided to look at roll angle and the effects each of these basic factors have on the vehicle roll angle at a given point on the racetrack that we have decided as being critical. Did you see the number of times I mentioned the end user being involved in choosing how to analyze this data? Pay attention to that sentence again. The effectiveness of a DOE approach is only as sharp as the person implementing it. There are numerous pitfalls here.
- Where do you look at the data on the track?
- What vehicle response(s) are critical to improving laptime?
- What factors were deemed critical enough to vary in the design?
- How many factors were varied?
- Was the design setup to include interaction of factors?
- In brutal honesty – how well does your simulation capture the response of your car?
- Will your setup stay close enough to the baseline that the DOE remains relevant?
You can see that, in order to effectively use this approach, you have to already have a good deal of knowledge of what you expect the outcome to be. You also have to be willing to accept when results are counterintuitive. I mentioned above about interactions, as you use higher order designs you began to capture interactions between your factors. This typically is when the human brain starts lagging in understanding – how does the engineer at the window of the car during practice comprehend how an interaction between all 7 factors listed above would affect the performance? This is when you must be back in front of the computer with the response surface calculation tool. The Pareto chart just helps you change front bar rate or the RR spring when the driver says the car rolls too much or too little – and don’t worry about the LF spring it won’t help here no matter how much you want it to. As in all real life, the previous example has a big “but” in it. What if your sim method doesn’t handle spring preload, bar preload, or jacking changes properly? Or maybe LF spring does affect roll, but you’ve just misled yourself away from that. Losing sucks, and being wrong is the geek’s version of losing.
Where do you look at the data on a given lap? Great question. This has huge implications on the quality of the fits, the impact on the setup, and the dreaded compromises that arise in any setup choice. This is trial and error, and no sane person would help a fellow competitor through this stage. Ever wonder why there are very few good SAE papers on racecar topics?
What vehicle responses are the most critical to improving lap time? When it comes to big stock cars (the vast majority of my time has been spent here), simply reducing roll angle isn’t going to make your car faster than everyone else’s, so the above example is a little off target. Simply matching dynamic crossweight to some magical number in the driver's and crew chief’s heads isn’t going to get it done either. The responses calculated are only limited by your imagination as to what defines better performance.
- What factors were deemed critical enough to vary in the design?
- How many were varied?
As the number of factors increases, so does the number of trials needed to capture the interactions of these factors. Think exponentially. This is where the multimode machines are making serious headway, no difference from CFD. What is important to your vehicle setup? What data about your model are clear enough to capture subtle changes? We all wish that tire data could be discreet to the point of being able to gnash our teeth endlessly over a very small pressure change. Vary inner and outer tire pressures (for a total of 8 factors and only 2 settings of each) in a full factorial DOE, and you’ve just signed up for 256 runs of simulation. Add only two more factors (let’s say front springs at only 2 levels) and that total is now 1024 runs. How fast is your PC? How accurate is that tire data in quantifying some change in the car? That may be better served by trying it with the driver strapped in and let the lap times make the call, except every major sanctioning body is trimming practice time as the years go by. When setting up your factors and their levels of variance, you have to allow a wide enough range to be helpful but not so wide that the response from one setting to the next is so different that the regression looks like a total mess. Maybe two levels of each factor aren’t enough, certainly when you are considering a spring change. No race team running for the big foam check at the end of the day brings only four spare springs! Now, consider that you decide you need to vary each of your 10 factors in 5 discreet values. That seems pretty reasonable. Put on your big boy pants, because you have just created a design with 9,765,625 simulation runs. Google fractional DOE design.
How well does your simulation capture the response of your car? As you can tell from the above discussion, this isn’t a topic to apply to your first runs in simulation. Get the validation work done, shoot holes in the data, and convince yourself that you are getting close. The alternative is wasting your time, which in this business tends to lead to unemployment.
You spent all week setting up the design and writing the script file to launch the sims and the array of CPUs burned through night and day without any interrupts in power or crashes in solution. Yeah, right, wait till you see some of the setups a five level full factorial design generates to run – you’ll be lucky if it can statically solve enough of them. On the final setup day, a decision is made to change the RF suspension geometry significantly. You now have a nice memorial to a lot of wasted time and effort. Once the setup winds its way outside of major parameters, the DOE becomes irrelevant. It’s a fact of life in this arena, you won’t be the first, second, or even 100th person this has happened to. The driver and team manager really aren’t going to want to hear about this. So, don’t bother. These are things that only other engineers are going to be sympathetic to, when we gather at the back of the garage to enjoy a Red Bull and complain to each other without giving away current projects.
DOE isn’t anything new. In reality, most of us were just waiting on hardware and software to catch up to concepts so it could be used. Most of my experience working on this topic happened between 1998 and 2004. The cost of such toys usually limits this approach to the big budget series. The tidbits and sarcastic remarks I have made all relate to working on this approach strictly in NASCAR Sprint Cup Racing at a couple of top level teams. I didn’t invent any of this. Like everything in racing, all of us claim to have invented it. My particular experiences happened while I was a part of a great group of engineers at previous teams. I have lived on every side of the race engineer role for the last decade, sometimes as an engineer helping support the race engineer’s job, sometimes as the race engineer, and sometimes as a crew chief using the race engineer to get the best out of the car, as quickly as possible. Luckily, my day job has nothing to do with simulation code, DOE, or race engineering these days, because this doesn’t even scratch the surface. I can guarantee you that all of the unknown faces in the Cup garage engineering these cars understand 100% of what was discussed here, and have for some time. Plenty of books have been written on the topic. I’ll leave you to go find them.
Thanks for reading,
Brandon Thomas
Red Bull Racing (USA)
Chief Design and Development Engineer
No comments:
Post a Comment