Have a look at this Wikipedia entry. While not written specifically about racing, it relates directly to Jeff's "secondary effects".
http://en.wikipedia.org/wiki/Law_of_Unintended_Consequences
In response to a request on the FSAE forum, Jeff has kindly taken the time to make a PDF of his setup change spreadsheet. Note that all the second page shows the selections for the pull-downs (not seen) that are used to fill in the individual setup changes on the sheet.
Jeff Braun Setup Change Database PDF
Showing posts with label Guest Blog. Show all posts
Showing posts with label Guest Blog. Show all posts
Wednesday, February 17, 2010
Saturday, February 13, 2010
Finding the Setup
Here's another guest post, this time from Jeff Braun. Jeff's a successful working race engineer, concentrating these days on endurance sports car racing, but with a broad background in many racing series.
He also has the dubious distinction of being the only race engineer who actually drove against me. It was a while ago... If I recall, I beat him then, but he's paid it back as an engineer, more than once.
Since we're friends, I'll indulge him for scooping me on material that was to have been included in the "Philosophy" series. But, there's much more than that in his post. I'm sure you'll find it thought-provoking.
The Setup – How to get it right
Apart from money, a good setup may be the most sought after thing in racing.
One thing before we talk about how to get the setup right. Who is ultimately responsible for the setup on the car? The driver! Not the engineers, not the team manager, not the chief mechanic, or anyone else except the driver. I hate it when people say “the engineer really has that car set up well.” It has little to do with the engineer. I have yet to see an engineer put a setup on a car, install a new driver, never change the setup, and win everything in sight. It never happens. The driver guides the setup process, he directs the engineers in the areas that need improvement, he is in charge of the car, and he has to get it to his liking to be fast. The engineer only interprets what the driver tells him and comes up with suggestions on what changes could help the problems the driver has identified as the main thing preventing him from going quicker. The engineer has the technical resources to come up with the best change to get the desired results, but he can’t identify and prioritize the areas that need improving without a technically good driver to point him in the right direction.
The magic setup – No numbers
Getting the setup right is not about cambers, toe, springs, and all the technical numbers. It is about the approach you take and the procedures you use to get there. So, I am not going to tell you the perfect springs to run at Laguna, or what wing setting works best at Montreal. Why? Because, it all depends on way too many things. I just plain don’t know the answer. What I am sure of is that, if you follow the things below, you will be ahead of the game in finding the springs or wing settings for your driver, your car, and the track on the weekend you are racing.
Best way to a great setup – Don’t change anything!
So, you spend hours with your driver and the data in the shop designing the starting setup for the weekend. You have experience with the car and kind of know the type of track you are going to, but have never been there before. You have it nailed, you think. You unload and head out for the first session. Four laps later, the driver is in the pits looking for a change. You jump in and start changing things; you want it to be better for him.
After the session you make more changes, maybe three or four, because he was 1 second off the pace. Next session the car is doing different things than before. Oh no, the changes were in the wrong direction. You remove the three changes and go the other way with four more. Now it’s better, but now a there is a different problem.
The data looks confusing, the driver is confused, but time is running out now, and you are still 0.8 seconds off. Something must be way different about this track than you thought. You decide you may have to rethink the entire setup, or that you may not really understand the car like you thought.
The next session you are better. The times are closer to fast guys and the driver just needs a few small changes. The changes help and the panic subsides some. But, you still need three tenths.
Now, it’s qualifying. The changes helped, and you’re now two tenths off the pole and on the second row. So, let’s look at the qualifying setup. It is exactly how we unloaded! We went in a big circle.
I see this all the time, at all levels of motorsport. I don’t know of any race engineers who get paid by the change. Slow down. Let the driver learn the track, and then figure out what he needs. The track will clean up and get some rubber down. Have confidence in your starting setup. Don’t be in a hurry to change it all around right away. Talk with your driver and write out a testing plan listing each session and what you hope to get done. This will only be a guide for the weekend, but it allows you to plan the sessions as you wish they would go.
A driver’s feed back is junk for the first session, anyway. He is getting a feel for the track, the grip level, and the car. Use that session to set the ride height (as the best engineer/driver I have ever known told me - lower is better always, always), log some baseline data, check the car, read the tires, and get some fuel consumption data. Robbie Groff used to tell me he needed the car to “talk to him” before he could tell me what we needed. Don’t screw up a good setup in the first session. Think where our driver could have qualified if he had kept the setup he ended up with and progressed forward in the sessions instead of going in a circle.
The spec car
If you have a restricted car or spec car, you have less things to try on the weekend. You should know the car well, if you have some experience with it. Otherwise, you can get a good starting set up from someone in the series. These cars are really relatively simple and require a methodical approach, rather than the shotgun approach of multiple wild guesses in search of the magic set up that gives you the 1 second you need. Now, there are some spec cars that are so bad that you do need the tricks that only the veterans know. My advice is to be prepared for a frustrating time, buy or steal the tricks, or run a real race car series.
Most series in North America have good cars that behave as expected. Try to stay within your setup window. You must define that window and keep refining it as you get to know the car better. But, resist the urge to go way out of what you know on a race weekend. Save that for testing. Keep a list of things you wanted to try at a race weekend, but did not because you did not know what it would do. Answer those questions at the next test.
On race weekends, it is always best to make changes that produce known results. If you don’t know what a change will do, go testing. If things are going well and you can afford to test for a session, then give some of the unknowns a poke. You can always go back to your baseline. Just remember, the other drivers will be moving forward in their setup when you are testing. You may find yourself behind by losing the session.
The track will tip your base setup some, but not much in a spec car. Try to come up with a base setup which will be refined each weekend. Run it at each track for the first session or two. You know how the car performs and feels with this setup, you know you liked it at the last race and there are no surprises to it. If the car feels different at this new track, you have just quantified the track. Using your base setup for the car, you can see what the track did to it. Make adjustments in the window of what you know works. Be sure the car is talking to you before you start. My driver/engineer friend likes to say “the laws of physics don’t change when you cross state lines.”
One last thing on this subject. Don’t fall into what I call the “Runoffs mistake”. A driver does a great job all year and qualifies for the prestigious SCCA Runoffs by hard work, good understanding of the car, and good procedures to get the set up right. He makes the trip to Road America and knows he has a shot to win this thing. But he knows there are some very good drivers from all over the U.S., and he thinks he needs something special to win. He finds a trick tire that he has never run but it is “worth 0.3 seconds”. The expert shock guy gave him the best Road America set up. The engine guy has the new development exhaust system. The result – the poor guy gets clobbered, not only by the out of division drivers, but also by the guys he beat all year, who now run away from him. Run what you know and tune it better than the next guy. Save the tricks for winter testing.
Changes – The secondary effects
In talking to engineers and drivers, I hear that they made a change and it did not help or did something completely different than they expected. Most of the time, the reason is what I call the secondary effect of a change.
Most racers have a good idea of what a change will do to the car, or at least they know what they want it to do. The problem is that seldom does a change only do one thing to the performance of the car. Each change has a primary effect and a secondary effect. There are third and fourth effects I am sure, but I am just barely smart enough to figure out the second effect.
If you are trying to decide between a few changes to improve something in the balance of the car, make a list of the three or four possible changes and what you expect each to do. For example, assume we have an understeer from the apex to the exit. We could change:
1 – More front wing
2 – More rake
3 – Stiffer rear springs
4 – More front rebound
There may be fifteen other things you could do, but list the top contenders. Now, list what you think the secondary effect of each might be:
1 – More front wing – oversteer in high speed turns
2 – More rake – nervous rear in high speed turns
3 – Stiffer rear springs – worse power down traction
4 – More front rebound – harsher in the bumps
Look at the list and try to find a change that has a primary effect and secondary effect that are in the same direction. Also, consider what other problems the driver may be having, which we’ll call the secondary complaint. In our example, if the high speed turns were good or bordering on oversteer, then we can eliminate the wing and rake. The stiffer rear springs will keep the dynamic rake in the car (primary effect) and free the car up on exit, while with less power down traction (secondary effect). Everything works in the direction we want.
What we want to avoid is a change with the primary effect opposite from the secondary effect. Often, this results in the driver comment of “can’t really tell much difference.” When a change has an unexpected result, you should look at the secondary effect. Sometimes, what you thought was the secondary effect was really the primary effect, and it was in the opposite direction.
Keeping track of all these effects and changes can get very confusing. The problem is that the effect of a change on a FF-1600 is very likely to be different from the same change on a Formula Atlantic car.
The change data base
There is a way to help sort the changes and what they did to your car. It helps you learn the car and can be a great tool to suggest changes that you know the result of like we talked about earlier.
One year at the test days for the Daytona 24 Hour, I was done with a session and walked past the factory Nissan NISMO team from Japan. In their pit were four racks of computers and more electronics than a modern day F1 team has. I watched for a while, as the Group C Nissan went around and around. There were only two guys manning the computer banks, so I walked up and asked what they were logging. The answer was that they were logging over 200 channels of data. I tried to think of 200 channels I would want to have logged. I got to about 30 and had to ask “what are you guys logging?” The Japanese engineer said “mostly engine parameters”. He said the engine block had twenty strain gauges cast into the block to measure stress in various sections of the block.
I thought that was very cool, but wanted to know what they were going to do with the data. The engineer said that, sometime in the future, a Nissan engineer was going to want to know what the stress was in an engine block. When that happened, the data would be there. He said that the cost of running a race car was so high, that collecting data while it was running reduced the cost of gaining that information later, when someone needed it. That impressed me with the long term thinking to collect data. Collect the data when you can, sort it out when you have time.
Any race team can do the same thing, on a smaller scale. Using a spread sheet, make a matrix with each column being a parameter in your setup. Include columns for driver comment, change, and result of the change. Each row will be a track outing. Start to log the result of each change you make to your car. In short order, you will have a history of each change made to your car.
Using the data sort function of the spread sheet, you can sort the data. For example, you can show every time you had exit understeer. Then, look at what you changed when you had that problem and what the result was. This will prevent you from making the same set up mistake twice. And, it will tell you what things have worked to correct an exit understeer in the past. It will keep you in the window of your knowledge. When testing, look at the data base and try some things that you have not tried before. This expands your understanding of the car and makes your data base more helpful at the track.
Using a data base program, like Access, takes it to the next level with tables, forms, and special queries to make the search results more meaningful and detailed. If you spend some time (it took me about 30 hours and I am NOT an Access expert), you can answer more questions. You can ask the database to show what we changed to reduce a mid corner oversteer in a third gear turn when we had tire pressures above 15 psi and a rear wing setting below 10 degrees.
Remember, the sooner you start collecting the data, the cheaper every mile you do becomes, because you are getting more value for your dollar. I can say that this has helped me a bunch over the years. My Daytona Prototype data base has over 900 changes logged in it. It just makes me a better guesser.
So, to recap…
1 – The driver is in charge of his setup. The engineer just makes the change he thinks will help him the most.
2 – Resist making changes until the driver really has a good feel for the car. It has to “talk to him” first.
3 – Physics does not change when you cross state lines. Go with what you know.
4 – The secondary effect of a change can be in the opposite of the primary effect. Never make a change without considering the secondary effect.
5 – Use a data base to increase your understanding of your car quicker and with more accuracy.
6 – If a change does not have the effect you thought it would, than you are missing some effect that you did not consider. There is not some weird phenomenon going on, you just don’t understand the circumstances of the particular situation.
7 – When in doubt, go back to your base setup and start over from there.
8 – Never copy another faster team’s setup. You need to know why yours does not work, so you can be better next time.
9 - If a change works the way you thought it would, you did not learn anything. You did become faster, which is always a good thing. But, when the change does not work as planned, you have a great chance to become smarter. Grab on to that and figure it out.
10 – When recording changes in your notes, write down why you made that change, your thinking on what you expect it to do, and why. Then, you can go back later and see what your thinking was for making that change and decide where your thinking was wrong. This may happen months later, as you get to know the car better, but it allows you to see where your mistake was, not just that it was a mistake.
OK, one more thing.
My friend, the driver/engineer mentor of mine, wrote these points down once about problem solving. Think about them. It may change your approach to finding the perfect set up.
1 – What’s right is right and everything else is wrong to some degree.
2 – What is REALLY happening here?
3 – Nothing happens for no reason.
4 – Everything is attributable.
5 – If X is true then Y must also be true. If I can’t prove that Y actually does as I predict, then I don’t know anything at all about X.
6 – What I am certain is correct can change instantly in the light of what is REALLY correct, whether I like it or not.
7 – Just because I don’t want to believe it, doesn’t make it wrong.
7a – Just because I want to believe it doesn’t make it right, either.
8 – Knowing what is wrong is every bit as important as knowing what is right.
9 – If it isn’t all the things you think it is, then it is something else. (Sherlock Holmes)
10 – You only know something if you can prove it. Everything else is “I suspect” or “I guess” or “I wonder if” or “it is my theory that...”
11 – The right answer is still the right answer even if you didn’t think of it.
12 – The right answer is still the right answer even if you don’t have any idea of why it works... but find out later for sure, because the underlying principles will always apply.
13 – Asking other people for answers is perfectly acceptable, as long as you never believe them.
He also has the dubious distinction of being the only race engineer who actually drove against me. It was a while ago... If I recall, I beat him then, but he's paid it back as an engineer, more than once.
Since we're friends, I'll indulge him for scooping me on material that was to have been included in the "Philosophy" series. But, there's much more than that in his post. I'm sure you'll find it thought-provoking.
The Setup – How to get it right
Apart from money, a good setup may be the most sought after thing in racing.
One thing before we talk about how to get the setup right. Who is ultimately responsible for the setup on the car? The driver! Not the engineers, not the team manager, not the chief mechanic, or anyone else except the driver. I hate it when people say “the engineer really has that car set up well.” It has little to do with the engineer. I have yet to see an engineer put a setup on a car, install a new driver, never change the setup, and win everything in sight. It never happens. The driver guides the setup process, he directs the engineers in the areas that need improvement, he is in charge of the car, and he has to get it to his liking to be fast. The engineer only interprets what the driver tells him and comes up with suggestions on what changes could help the problems the driver has identified as the main thing preventing him from going quicker. The engineer has the technical resources to come up with the best change to get the desired results, but he can’t identify and prioritize the areas that need improving without a technically good driver to point him in the right direction.
The magic setup – No numbers
Getting the setup right is not about cambers, toe, springs, and all the technical numbers. It is about the approach you take and the procedures you use to get there. So, I am not going to tell you the perfect springs to run at Laguna, or what wing setting works best at Montreal. Why? Because, it all depends on way too many things. I just plain don’t know the answer. What I am sure of is that, if you follow the things below, you will be ahead of the game in finding the springs or wing settings for your driver, your car, and the track on the weekend you are racing.
Best way to a great setup – Don’t change anything!
So, you spend hours with your driver and the data in the shop designing the starting setup for the weekend. You have experience with the car and kind of know the type of track you are going to, but have never been there before. You have it nailed, you think. You unload and head out for the first session. Four laps later, the driver is in the pits looking for a change. You jump in and start changing things; you want it to be better for him.
After the session you make more changes, maybe three or four, because he was 1 second off the pace. Next session the car is doing different things than before. Oh no, the changes were in the wrong direction. You remove the three changes and go the other way with four more. Now it’s better, but now a there is a different problem.
The data looks confusing, the driver is confused, but time is running out now, and you are still 0.8 seconds off. Something must be way different about this track than you thought. You decide you may have to rethink the entire setup, or that you may not really understand the car like you thought.
The next session you are better. The times are closer to fast guys and the driver just needs a few small changes. The changes help and the panic subsides some. But, you still need three tenths.
Now, it’s qualifying. The changes helped, and you’re now two tenths off the pole and on the second row. So, let’s look at the qualifying setup. It is exactly how we unloaded! We went in a big circle.
I see this all the time, at all levels of motorsport. I don’t know of any race engineers who get paid by the change. Slow down. Let the driver learn the track, and then figure out what he needs. The track will clean up and get some rubber down. Have confidence in your starting setup. Don’t be in a hurry to change it all around right away. Talk with your driver and write out a testing plan listing each session and what you hope to get done. This will only be a guide for the weekend, but it allows you to plan the sessions as you wish they would go.
A driver’s feed back is junk for the first session, anyway. He is getting a feel for the track, the grip level, and the car. Use that session to set the ride height (as the best engineer/driver I have ever known told me - lower is better always, always), log some baseline data, check the car, read the tires, and get some fuel consumption data. Robbie Groff used to tell me he needed the car to “talk to him” before he could tell me what we needed. Don’t screw up a good setup in the first session. Think where our driver could have qualified if he had kept the setup he ended up with and progressed forward in the sessions instead of going in a circle.
The spec car
If you have a restricted car or spec car, you have less things to try on the weekend. You should know the car well, if you have some experience with it. Otherwise, you can get a good starting set up from someone in the series. These cars are really relatively simple and require a methodical approach, rather than the shotgun approach of multiple wild guesses in search of the magic set up that gives you the 1 second you need. Now, there are some spec cars that are so bad that you do need the tricks that only the veterans know. My advice is to be prepared for a frustrating time, buy or steal the tricks, or run a real race car series.
Most series in North America have good cars that behave as expected. Try to stay within your setup window. You must define that window and keep refining it as you get to know the car better. But, resist the urge to go way out of what you know on a race weekend. Save that for testing. Keep a list of things you wanted to try at a race weekend, but did not because you did not know what it would do. Answer those questions at the next test.
On race weekends, it is always best to make changes that produce known results. If you don’t know what a change will do, go testing. If things are going well and you can afford to test for a session, then give some of the unknowns a poke. You can always go back to your baseline. Just remember, the other drivers will be moving forward in their setup when you are testing. You may find yourself behind by losing the session.
The track will tip your base setup some, but not much in a spec car. Try to come up with a base setup which will be refined each weekend. Run it at each track for the first session or two. You know how the car performs and feels with this setup, you know you liked it at the last race and there are no surprises to it. If the car feels different at this new track, you have just quantified the track. Using your base setup for the car, you can see what the track did to it. Make adjustments in the window of what you know works. Be sure the car is talking to you before you start. My driver/engineer friend likes to say “the laws of physics don’t change when you cross state lines.”
One last thing on this subject. Don’t fall into what I call the “Runoffs mistake”. A driver does a great job all year and qualifies for the prestigious SCCA Runoffs by hard work, good understanding of the car, and good procedures to get the set up right. He makes the trip to Road America and knows he has a shot to win this thing. But he knows there are some very good drivers from all over the U.S., and he thinks he needs something special to win. He finds a trick tire that he has never run but it is “worth 0.3 seconds”. The expert shock guy gave him the best Road America set up. The engine guy has the new development exhaust system. The result – the poor guy gets clobbered, not only by the out of division drivers, but also by the guys he beat all year, who now run away from him. Run what you know and tune it better than the next guy. Save the tricks for winter testing.
Changes – The secondary effects
In talking to engineers and drivers, I hear that they made a change and it did not help or did something completely different than they expected. Most of the time, the reason is what I call the secondary effect of a change.
Most racers have a good idea of what a change will do to the car, or at least they know what they want it to do. The problem is that seldom does a change only do one thing to the performance of the car. Each change has a primary effect and a secondary effect. There are third and fourth effects I am sure, but I am just barely smart enough to figure out the second effect.
If you are trying to decide between a few changes to improve something in the balance of the car, make a list of the three or four possible changes and what you expect each to do. For example, assume we have an understeer from the apex to the exit. We could change:
1 – More front wing
2 – More rake
3 – Stiffer rear springs
4 – More front rebound
There may be fifteen other things you could do, but list the top contenders. Now, list what you think the secondary effect of each might be:
1 – More front wing – oversteer in high speed turns
2 – More rake – nervous rear in high speed turns
3 – Stiffer rear springs – worse power down traction
4 – More front rebound – harsher in the bumps
Look at the list and try to find a change that has a primary effect and secondary effect that are in the same direction. Also, consider what other problems the driver may be having, which we’ll call the secondary complaint. In our example, if the high speed turns were good or bordering on oversteer, then we can eliminate the wing and rake. The stiffer rear springs will keep the dynamic rake in the car (primary effect) and free the car up on exit, while with less power down traction (secondary effect). Everything works in the direction we want.
What we want to avoid is a change with the primary effect opposite from the secondary effect. Often, this results in the driver comment of “can’t really tell much difference.” When a change has an unexpected result, you should look at the secondary effect. Sometimes, what you thought was the secondary effect was really the primary effect, and it was in the opposite direction.
Keeping track of all these effects and changes can get very confusing. The problem is that the effect of a change on a FF-1600 is very likely to be different from the same change on a Formula Atlantic car.
The change data base
There is a way to help sort the changes and what they did to your car. It helps you learn the car and can be a great tool to suggest changes that you know the result of like we talked about earlier.
One year at the test days for the Daytona 24 Hour, I was done with a session and walked past the factory Nissan NISMO team from Japan. In their pit were four racks of computers and more electronics than a modern day F1 team has. I watched for a while, as the Group C Nissan went around and around. There were only two guys manning the computer banks, so I walked up and asked what they were logging. The answer was that they were logging over 200 channels of data. I tried to think of 200 channels I would want to have logged. I got to about 30 and had to ask “what are you guys logging?” The Japanese engineer said “mostly engine parameters”. He said the engine block had twenty strain gauges cast into the block to measure stress in various sections of the block.
I thought that was very cool, but wanted to know what they were going to do with the data. The engineer said that, sometime in the future, a Nissan engineer was going to want to know what the stress was in an engine block. When that happened, the data would be there. He said that the cost of running a race car was so high, that collecting data while it was running reduced the cost of gaining that information later, when someone needed it. That impressed me with the long term thinking to collect data. Collect the data when you can, sort it out when you have time.
Any race team can do the same thing, on a smaller scale. Using a spread sheet, make a matrix with each column being a parameter in your setup. Include columns for driver comment, change, and result of the change. Each row will be a track outing. Start to log the result of each change you make to your car. In short order, you will have a history of each change made to your car.
Using the data sort function of the spread sheet, you can sort the data. For example, you can show every time you had exit understeer. Then, look at what you changed when you had that problem and what the result was. This will prevent you from making the same set up mistake twice. And, it will tell you what things have worked to correct an exit understeer in the past. It will keep you in the window of your knowledge. When testing, look at the data base and try some things that you have not tried before. This expands your understanding of the car and makes your data base more helpful at the track.
Using a data base program, like Access, takes it to the next level with tables, forms, and special queries to make the search results more meaningful and detailed. If you spend some time (it took me about 30 hours and I am NOT an Access expert), you can answer more questions. You can ask the database to show what we changed to reduce a mid corner oversteer in a third gear turn when we had tire pressures above 15 psi and a rear wing setting below 10 degrees.
Remember, the sooner you start collecting the data, the cheaper every mile you do becomes, because you are getting more value for your dollar. I can say that this has helped me a bunch over the years. My Daytona Prototype data base has over 900 changes logged in it. It just makes me a better guesser.
So, to recap…
1 – The driver is in charge of his setup. The engineer just makes the change he thinks will help him the most.
2 – Resist making changes until the driver really has a good feel for the car. It has to “talk to him” first.
3 – Physics does not change when you cross state lines. Go with what you know.
4 – The secondary effect of a change can be in the opposite of the primary effect. Never make a change without considering the secondary effect.
5 – Use a data base to increase your understanding of your car quicker and with more accuracy.
6 – If a change does not have the effect you thought it would, than you are missing some effect that you did not consider. There is not some weird phenomenon going on, you just don’t understand the circumstances of the particular situation.
7 – When in doubt, go back to your base setup and start over from there.
8 – Never copy another faster team’s setup. You need to know why yours does not work, so you can be better next time.
9 - If a change works the way you thought it would, you did not learn anything. You did become faster, which is always a good thing. But, when the change does not work as planned, you have a great chance to become smarter. Grab on to that and figure it out.
10 – When recording changes in your notes, write down why you made that change, your thinking on what you expect it to do, and why. Then, you can go back later and see what your thinking was for making that change and decide where your thinking was wrong. This may happen months later, as you get to know the car better, but it allows you to see where your mistake was, not just that it was a mistake.
OK, one more thing.
My friend, the driver/engineer mentor of mine, wrote these points down once about problem solving. Think about them. It may change your approach to finding the perfect set up.
1 – What’s right is right and everything else is wrong to some degree.
2 – What is REALLY happening here?
3 – Nothing happens for no reason.
4 – Everything is attributable.
5 – If X is true then Y must also be true. If I can’t prove that Y actually does as I predict, then I don’t know anything at all about X.
6 – What I am certain is correct can change instantly in the light of what is REALLY correct, whether I like it or not.
7 – Just because I don’t want to believe it, doesn’t make it wrong.
7a – Just because I want to believe it doesn’t make it right, either.
8 – Knowing what is wrong is every bit as important as knowing what is right.
9 – If it isn’t all the things you think it is, then it is something else. (Sherlock Holmes)
10 – You only know something if you can prove it. Everything else is “I suspect” or “I guess” or “I wonder if” or “it is my theory that...”
11 – The right answer is still the right answer even if you didn’t think of it.
12 – The right answer is still the right answer even if you don’t have any idea of why it works... but find out later for sure, because the underlying principles will always apply.
13 – Asking other people for answers is perfectly acceptable, as long as you never believe them.
Wednesday, December 30, 2009
DOE in NASCAR Sprint Cup
Brandon Thomas is Chief Design and Development Engineer for Red Bull Racing in NASCAR. He has the dubious distinction of being the first victim to be drafted for a guest blog on The Race Engineer. Thanks a bunch, Brandon. Without further ado...
I feel honored to be invited to guest blog on The Race Engineer blog. I made the mistake of updating my LinkedIn profile with some obscure details of my work history and Buddy came calling. He and I have traded EMails over the last few weeks, and some of that content he felt was worthy of the blog. So, here goes.
DOE, design of experiment, was a topic briefly glossed over in my required statistics and probability class during engineering school. Had it been more of a focal point in the class, I probably would have paid much more attention. Seems the biggest problem educators have is making the course content relevant to the audience. I cannot explain the intricacies, or even the theory, behind the method in a simplified discussion. The point of DOE is to build a response model based on several user-decided factors. Like all models, it requires the user to be pretty informed ahead of time of what is expected in the results, but also to be somewhat open-minded when the results diverge from the expected. I’ll leave it up to you readers to do more research on the www on the method and its origins.
DOE was certainly revered in the pharmaceutical industry – probably still is. Types of experiments, the number of factors involved, and the complexity of the experiment are all parts of drug trials in this era. What does that have to do with making your racecar go faster around a given lap than the competition? Working smarter, not harder, eliminating the surrounding static, and concentrating on the most important setup parameters to achieve a result.
In this realm, I am alluding to a method of DOE involving simulation code. There are many basic advantages to DOE in the virtual world. The repeats can all be skipped, the number of factors can be rather large, and you can accomplish a full factorial design with today’s larger multi-core machines. There are other methods of DOE outside the virtual world, but I’ll take the academic copout that those are beyond the scope of this blog, and are sensitive information in a competitive environment.
Now let’s get to the good stuff. What does DOE methodology do for the race engineer?
One of the most useful tools to a race engineer when away from the computer is a Pareto chart based on the results of the DOE. Here’s an entirely fake example of one.
I feel honored to be invited to guest blog on The Race Engineer blog. I made the mistake of updating my LinkedIn profile with some obscure details of my work history and Buddy came calling. He and I have traded EMails over the last few weeks, and some of that content he felt was worthy of the blog. So, here goes.
DOE, design of experiment, was a topic briefly glossed over in my required statistics and probability class during engineering school. Had it been more of a focal point in the class, I probably would have paid much more attention. Seems the biggest problem educators have is making the course content relevant to the audience. I cannot explain the intricacies, or even the theory, behind the method in a simplified discussion. The point of DOE is to build a response model based on several user-decided factors. Like all models, it requires the user to be pretty informed ahead of time of what is expected in the results, but also to be somewhat open-minded when the results diverge from the expected. I’ll leave it up to you readers to do more research on the www on the method and its origins.
DOE was certainly revered in the pharmaceutical industry – probably still is. Types of experiments, the number of factors involved, and the complexity of the experiment are all parts of drug trials in this era. What does that have to do with making your racecar go faster around a given lap than the competition? Working smarter, not harder, eliminating the surrounding static, and concentrating on the most important setup parameters to achieve a result.
In this realm, I am alluding to a method of DOE involving simulation code. There are many basic advantages to DOE in the virtual world. The repeats can all be skipped, the number of factors can be rather large, and you can accomplish a full factorial design with today’s larger multi-core machines. There are other methods of DOE outside the virtual world, but I’ll take the academic copout that those are beyond the scope of this blog, and are sensitive information in a competitive environment.
Now let’s get to the good stuff. What does DOE methodology do for the race engineer?
One of the most useful tools to a race engineer when away from the computer is a Pareto chart based on the results of the DOE. Here’s an entirely fake example of one.
In this case we decided to look at roll angle and the effects each of these basic factors have on the vehicle roll angle at a given point on the racetrack that we have decided as being critical. Did you see the number of times I mentioned the end user being involved in choosing how to analyze this data? Pay attention to that sentence again. The effectiveness of a DOE approach is only as sharp as the person implementing it. There are numerous pitfalls here.
- Where do you look at the data on the track?
- What vehicle response(s) are critical to improving laptime?
- What factors were deemed critical enough to vary in the design?
- How many factors were varied?
- Was the design setup to include interaction of factors?
- In brutal honesty – how well does your simulation capture the response of your car?
- Will your setup stay close enough to the baseline that the DOE remains relevant?
You can see that, in order to effectively use this approach, you have to already have a good deal of knowledge of what you expect the outcome to be. You also have to be willing to accept when results are counterintuitive. I mentioned above about interactions, as you use higher order designs you began to capture interactions between your factors. This typically is when the human brain starts lagging in understanding – how does the engineer at the window of the car during practice comprehend how an interaction between all 7 factors listed above would affect the performance? This is when you must be back in front of the computer with the response surface calculation tool. The Pareto chart just helps you change front bar rate or the RR spring when the driver says the car rolls too much or too little – and don’t worry about the LF spring it won’t help here no matter how much you want it to. As in all real life, the previous example has a big “but” in it. What if your sim method doesn’t handle spring preload, bar preload, or jacking changes properly? Or maybe LF spring does affect roll, but you’ve just misled yourself away from that. Losing sucks, and being wrong is the geek’s version of losing.
Where do you look at the data on a given lap? Great question. This has huge implications on the quality of the fits, the impact on the setup, and the dreaded compromises that arise in any setup choice. This is trial and error, and no sane person would help a fellow competitor through this stage. Ever wonder why there are very few good SAE papers on racecar topics?
What vehicle responses are the most critical to improving lap time? When it comes to big stock cars (the vast majority of my time has been spent here), simply reducing roll angle isn’t going to make your car faster than everyone else’s, so the above example is a little off target. Simply matching dynamic crossweight to some magical number in the driver's and crew chief’s heads isn’t going to get it done either. The responses calculated are only limited by your imagination as to what defines better performance.
- What factors were deemed critical enough to vary in the design?
- How many were varied?
As the number of factors increases, so does the number of trials needed to capture the interactions of these factors. Think exponentially. This is where the multimode machines are making serious headway, no difference from CFD. What is important to your vehicle setup? What data about your model are clear enough to capture subtle changes? We all wish that tire data could be discreet to the point of being able to gnash our teeth endlessly over a very small pressure change. Vary inner and outer tire pressures (for a total of 8 factors and only 2 settings of each) in a full factorial DOE, and you’ve just signed up for 256 runs of simulation. Add only two more factors (let’s say front springs at only 2 levels) and that total is now 1024 runs. How fast is your PC? How accurate is that tire data in quantifying some change in the car? That may be better served by trying it with the driver strapped in and let the lap times make the call, except every major sanctioning body is trimming practice time as the years go by. When setting up your factors and their levels of variance, you have to allow a wide enough range to be helpful but not so wide that the response from one setting to the next is so different that the regression looks like a total mess. Maybe two levels of each factor aren’t enough, certainly when you are considering a spring change. No race team running for the big foam check at the end of the day brings only four spare springs! Now, consider that you decide you need to vary each of your 10 factors in 5 discreet values. That seems pretty reasonable. Put on your big boy pants, because you have just created a design with 9,765,625 simulation runs. Google fractional DOE design.
How well does your simulation capture the response of your car? As you can tell from the above discussion, this isn’t a topic to apply to your first runs in simulation. Get the validation work done, shoot holes in the data, and convince yourself that you are getting close. The alternative is wasting your time, which in this business tends to lead to unemployment.
You spent all week setting up the design and writing the script file to launch the sims and the array of CPUs burned through night and day without any interrupts in power or crashes in solution. Yeah, right, wait till you see some of the setups a five level full factorial design generates to run – you’ll be lucky if it can statically solve enough of them. On the final setup day, a decision is made to change the RF suspension geometry significantly. You now have a nice memorial to a lot of wasted time and effort. Once the setup winds its way outside of major parameters, the DOE becomes irrelevant. It’s a fact of life in this arena, you won’t be the first, second, or even 100th person this has happened to. The driver and team manager really aren’t going to want to hear about this. So, don’t bother. These are things that only other engineers are going to be sympathetic to, when we gather at the back of the garage to enjoy a Red Bull and complain to each other without giving away current projects.
DOE isn’t anything new. In reality, most of us were just waiting on hardware and software to catch up to concepts so it could be used. Most of my experience working on this topic happened between 1998 and 2004. The cost of such toys usually limits this approach to the big budget series. The tidbits and sarcastic remarks I have made all relate to working on this approach strictly in NASCAR Sprint Cup Racing at a couple of top level teams. I didn’t invent any of this. Like everything in racing, all of us claim to have invented it. My particular experiences happened while I was a part of a great group of engineers at previous teams. I have lived on every side of the race engineer role for the last decade, sometimes as an engineer helping support the race engineer’s job, sometimes as the race engineer, and sometimes as a crew chief using the race engineer to get the best out of the car, as quickly as possible. Luckily, my day job has nothing to do with simulation code, DOE, or race engineering these days, because this doesn’t even scratch the surface. I can guarantee you that all of the unknown faces in the Cup garage engineering these cars understand 100% of what was discussed here, and have for some time. Plenty of books have been written on the topic. I’ll leave you to go find them.
Thanks for reading,
Brandon Thomas
Red Bull Racing (USA)
Chief Design and Development Engineer
Subscribe to:
Posts (Atom)
