DOE in NASCAR Sprint Cup

Brandon Thomas is Chief Design and Development Engineer for Red Bull Racing in NASCAR. He has the dubious distinction of being the first victim to be drafted for a guest blog on The Race Engineer. Thanks a bunch, Brandon. Without further ado...



I feel honored to be invited to guest blog on The Race Engineer blog. I made the mistake of updating my LinkedIn profile with some obscure details of my work history and Buddy came calling. He and I have traded EMails over the last few weeks, and some of that content he felt was worthy of the blog. So, here goes.

DOE, design of experiment, was a topic briefly glossed over in my required statistics and probability class during engineering school. Had it been more of a focal point in the class, I probably would have paid much more attention. Seems the biggest problem educators have is making the course content relevant to the audience. I cannot explain the intricacies, or even the theory, behind the method in a simplified discussion. The point of DOE is to build a response model based on several user-decided factors. Like all models, it requires the user to be pretty informed ahead of time of what is expected in the results, but also to be somewhat open-minded when the results diverge from the expected. I’ll leave it up to you readers to do more research on the www on the method and its origins.

DOE was certainly revered in the pharmaceutical industry – probably still is. Types of experiments, the number of factors involved, and the complexity of the experiment are all parts of drug trials in this era. What does that have to do with making your racecar go faster around a given lap than the competition? Working smarter, not harder, eliminating the surrounding static, and concentrating on the most important setup parameters to achieve a result.

In this realm, I am alluding to a method of DOE involving simulation code. There are many basic advantages to DOE in the virtual world. The repeats can all be skipped, the number of factors can be rather large, and you can accomplish a full factorial design with today’s larger multi-core machines. There are other methods of DOE outside the virtual world, but I’ll take the academic copout that those are beyond the scope of this blog, and are sensitive information in a competitive environment.

Now let’s get to the good stuff. What does DOE methodology do for the race engineer?

One of the most useful tools to a race engineer when away from the computer is a Pareto chart based on the results of the DOE. Here’s an entirely fake example of one.

In this case we decided to look at roll angle and the effects each of these basic factors have on the vehicle roll angle at a given point on the racetrack that we have decided as being critical. Did you see the number of times I mentioned the end user being involved in choosing how to analyze this data? Pay attention to that sentence again. The effectiveness of a DOE approach is only as sharp as the person implementing it. There are numerous pitfalls here.

• Where do you look at the data on the track?
• What vehicle response(s) are critical to improving laptime?
• What factors were deemed critical enough to vary in the design?
• How many factors were varied?
• Was the design setup to include interaction of factors?
• In brutal honesty – how well does your simulation capture the response of your car?
• Will your setup stay close enough to the baseline that the DOE remains relevant?

You can see that, in order to effectively use this approach, you have to already have a good deal of knowledge of what you expect the outcome to be. You also have to be willing to accept when results are counterintuitive. I mentioned above about interactions, as you use higher order designs you began to capture interactions between your factors. This typically is when the human brain starts lagging in understanding – how does the engineer at the window of the car during practice comprehend how an interaction between all 7 factors listed above would affect the performance? This is when you must be back in front of the computer with the response surface calculation tool. The Pareto chart just helps you change front bar rate or the RR spring when the driver says the car rolls too much or too little – and don’t worry about the LF spring it won’t help here no matter how much you want it to. As in all real life, the previous example has a big “but” in it. What if your sim method doesn’t handle spring preload, bar preload, or jacking changes properly? Or maybe LF spring does affect roll, but you’ve just misled yourself away from that. Losing sucks, and being wrong is the geek’s version of losing.

Where do you look at the data on a given lap? Great question. This has huge implications on the quality of the fits, the impact on the setup, and the dreaded compromises that arise in any setup choice. This is trial and error, and no sane person would help a fellow competitor through this stage. Ever wonder why there are very few good SAE papers on racecar topics?

What vehicle responses are the most critical to improving lap time? When it comes to big stock cars (the vast majority of my time has been spent here), simply reducing roll angle isn’t going to make your car faster than everyone else’s, so the above example is a little off target. Simply matching dynamic crossweight to some magical number in the driver's and crew chief’s heads isn’t going to get it done either. The responses calculated are only limited by your imagination as to what defines better performance.

• What factors were deemed critical enough to vary in the design?
• How many were varied?

As the number of factors increases, so does the number of trials needed to capture the interactions of these factors. Think exponentially. This is where the multimode machines are making serious headway, no difference from CFD. What is important to your vehicle setup? What data about your model are clear enough to capture subtle changes? We all wish that tire data could be discreet to the point of being able to gnash our teeth endlessly over a very small pressure change. Vary inner and outer tire pressures (for a total of 8 factors and only 2 settings of each) in a full factorial DOE, and you’ve just signed up for 256 runs of simulation. Add only two more factors (let’s say front springs at only 2 levels) and that total is now 1024 runs. How fast is your PC? How accurate is that tire data in quantifying some change in the car? That may be better served by trying it with the driver strapped in and let the lap times make the call, except every major sanctioning body is trimming practice time as the years go by. When setting up your factors and their levels of variance, you have to allow a wide enough range to be helpful but not so wide that the response from one setting to the next is so different that the regression looks like a total mess. Maybe two levels of each factor aren’t enough, certainly when you are considering a spring change. No race team running for the big foam check at the end of the day brings only four spare springs! Now, consider that you decide you need to vary each of your 10 factors in 5 discreet values. That seems pretty reasonable. Put on your big boy pants, because you have just created a design with 9,765,625 simulation runs. Google fractional DOE design.

How well does your simulation capture the response of your car? As you can tell from the above discussion, this isn’t a topic to apply to your first runs in simulation. Get the validation work done, shoot holes in the data, and convince yourself that you are getting close. The alternative is wasting your time, which in this business tends to lead to unemployment.

You spent all week setting up the design and writing the script file to launch the sims and the array of CPUs burned through night and day without any interrupts in power or crashes in solution. Yeah, right, wait till you see some of the setups a five level full factorial design generates to run – you’ll be lucky if it can statically solve enough of them. On the final setup day, a decision is made to change the RF suspension geometry significantly. You now have a nice memorial to a lot of wasted time and effort. Once the setup winds its way outside of major parameters, the DOE becomes irrelevant. It’s a fact of life in this arena, you won’t be the first, second, or even 100th person this has happened to. The driver and team manager really aren’t going to want to hear about this. So, don’t bother. These are things that only other engineers are going to be sympathetic to, when we gather at the back of the garage to enjoy a Red Bull and complain to each other without giving away current projects.

DOE isn’t anything new. In reality, most of us were just waiting on hardware and software to catch up to concepts so it could be used. Most of my experience working on this topic happened between 1998 and 2004. The cost of such toys usually limits this approach to the big budget series. The tidbits and sarcastic remarks I have made all relate to working on this approach strictly in NASCAR Sprint Cup Racing at a couple of top level teams. I didn’t invent any of this. Like everything in racing, all of us claim to have invented it. My particular experiences happened while I was a part of a great group of engineers at previous teams. I have lived on every side of the race engineer role for the last decade, sometimes as an engineer helping support the race engineer’s job, sometimes as the race engineer, and sometimes as a crew chief using the race engineer to get the best out of the car, as quickly as possible. Luckily, my day job has nothing to do with simulation code, DOE, or race engineering these days, because this doesn’t even scratch the surface. I can guarantee you that all of the unknown faces in the Cup garage engineering these cars understand 100% of what was discussed here, and have for some time. Plenty of books have been written on the topic. I’ll leave you to go find them.

Thanks for reading,

Brandon Thomas
Red Bull Racing (USA)
Chief Design and Development Engineer

Setup Sheets, Part 2

Let's review the setup sheet layout. There are two fundamental layout concepts: by topic and by car location.

Sheets laid out by topic group similar items together. All the ride heights are together, all the shock info together, all the aero info together, and so on. For example, here's a prototype sports car sheet. This layout, although well done, omits some detail on brakes, tires, suspension geometry. It is used with "setup wheels", machined aluminum fixtures that replace real wheels and tires on the setup pad. Ride heights are calculated from measured drops to a point on top of the chassis, rather than actual measurements up from setup pad to the floor. Blue items are user input.



PDF - Print or Free DownloadXLS - Purchase Full Download


Sheets laid out by car location group information into a birds-eye view of the car. For each corner, you have alignment, tire data, springs, and so on. Information that doesn't fit that layout is placed on the center of the sheet or in a separate section. In a slightly different twist, the sheet that I use has chassis-mounted items and measurements like AR bars, ride heights, and packer gaps in the center, reducing the amount of info listed at the individual wheel. Here's a Swift 008a Formula Atlantic sheet. Note that this sheet includes some engineering calcs. It also has non-printing separate worksheets for vehicle dynamics and for shock build specs.


PDF - Print or Free DownloadXLS - Purchase Full Download


So, which layout to use? Both are popular, and both can be effective, if they are done well.

The main advantage of the topic layout is in grouping similar items together. For example, all the corner weights are in one spot, just like on the scale display. With so many different types of data to show, it can be a little scattered, unless it is carefully organized. The example posted here is one of the better ones.

The strength of the car location layout is in its ease of use. If you want to know something about the right front corner of the car, look at that part of the sheet. Some items, like brakes, corner weights, rake, or cooling, don't fall into the layout that well.

I use a layout that mixes some elements of both approaches. Go back and look at the sports car sample included in the Part 1 post, it's mine. Stuff mounted or measured at the wheels is out on the corners. Stuff mounted or measured on the chassis is down the middle. Front aero is at the front, rear aero at the rear. Gears, brakes, and weights are clumped together at their approximate location on the car. The next incarnation might get a new section for configuration file names for the data system, ECU, ABS, paddle shift system, etc.

Now, let's look at a layout for the worksheet that accompanies the car to the setup pad. This worksheet is a hands-on working document for use at the setup pad. Most of the teams I work with lack either the time, money, or resources for this to be used as a networked document on an smart phone or touch-screen PC. So, it's filled out by hand, and may or may not be scanned, depending on who needs copies and when. I like the cheap HP all-in-one printer/scanner/copier units for the trailer.

On the front, there are fields for Setdown, where we document how the car was found as it rolled off the track and onto the setup pad after preceding on-track session. The center column is used to enter the changes to make. The changes are then made, on or off the pad, and the car rolled on for adjustment. And then, the righthand column documents how the car rolled off the setup pad. On the back side, there is a worksheet for actually making the adjustments.

Yeah, I know, there's some redundency here, and opportunity to introduce error. We'll talk about this again in a later post, but cutting to the chase, I've found that a complete setup sheet doesn't work too well for calling out between-session changes. So, we do a setdown, fill in the changes, and finish the setup.

The example below is a scan of both the front and back pages of a setup worksheet after use. The links immediately above the form will download it, as well as blank versions of the front and back pages.



PDF - Print or Free Download Blank Form
XLS - Purchase Full Download Blank Form
PDF - Print and Free Download Completed Form



Tools and organization

OK, how about computing tools? Your choices are basically spreadsheet or data base. PDF forms with fillable fields don't have enough function. Spreadsheets offer plenty of formatting and calculating power, and are the near-universal solution. But, I've always wanted to try a database. The initial setup would be lot more work, but your setups would be available for the full power of database searching and reporting. I suspect that the ever-evolving nature of much racing might be responsible for the relative rarity of databases, since last year's setup is often no longer relevent. Series where you take the same basic car back to the same tracks, year after year, probably stand to benefit the most.

One thing is for sure. You have got to be diligent and organized in file naming and directory structure, or you will soon have an unworkable jumble of setup sheets files. Here's the file naming convention that I use:

Setup Seb090307 A04 P1 Start.xls

• Sebring is the track
• March 7, 2009 is the race date (not the creation date of the setup sheet)
• Chassis number A04
• This sheet shows how the car started the first official practice session

I place all the sheets for an event into a directory exclusive to that event. Use real-time archival software pointed at the location of all the setup sheets. You don't want to lose a year's worth of setups when the notebook hard drive crashes at the track.

Remembering that a setup sheet is a vehicle for communication, the next post will get into the process of using it. Once that's done, we'll dig deeper into content.

Setup Sheets, Part 1

I'm kinda excited about this multi-part series, since it's a bit of a departure from recent posts. We're going to review setup sheets in fairly complete detail, so this isn't your usual short-attention-span blog post. You'll be able to download PDF samples and working Excel spreadsheets from Scribd. Here we go...

What is a setup sheet, anyway?

In simple terms, it is a document that details the configuration and adjustment of a race car.

And what is it used for?

• Define all the setup adjustments, like alignment, ride height, etc.
• Specify commonly swapped parts, like springs, gear ratios, anti-roll bars, etc.
• Document the car setup, for later reference
• Possibly, link to analysis or simulation software to provide vehicle dynamics details

Let's talk first about general content. Future posts will cover how to actually use a setup sheet, communication issues, options for layouts, computing tools and storage, some recommendations, and more.

The simplest setup sheets are handwritten onto a basic blank form. In this guise, it is mainly a working document for crew adjustments to the car on the setup pad. It probably has no more than the following content, and maybe less, depending on what items may be non-adjustable, non-changeable, or non-existent on a specific car:

• Ride heights
• Spring rates
• Anti-roll bar sizes and adjustments
• Shock adjustments and gas pressures
• Camber, caster, and toe settings
• Aero adjustments, such as angles and dimensions
• Corner weights and percentages

Here's the catch. If the setup sheet is to be a complete and unambiguous definition of how the car is configured, there is inevitably more information required. Sometimes, lots more. The possible list is endless, but here are some common items, in no particular order:

• Bump rubber spec and packer gaps
• Third spring and damper components and adjustments
• Optional aero components and how they are installed or adjusted
• Cooling configuration, both components and blanking
• Gear ratios and differential setup
• Optional suspension geometry and components
• Brake components, pad/rotor material, master cylinders, bias setup
• Multiple ride heights - aero components vs. structural/suspension
• Specific assembly instructions - part numbers, shims, etc.
• Tire sizes, compounds, and constructions
• Tire pressures for both the setup pad and the grid
• Driver weight and fuel load for the setup pad, starting fuel load for the track
• Shock build spec
• Part numbers or serial numbers for specific components and assemblies
• Spring and/or pushrod installed length. Rocker ratio and position.
• Ballast weight, configuration, position

This can get out of hand. Still, we absolutely have to be able to completely and unambiguously define how the car is expected to be configured when it rolls onto the track. At some time in the future, we need to be able to completely and unambiguously recall that configuration by reviewing the setup sheet.

Taking things a step further, there are two additional types of information that sometimes show up on setup sheets. They are specific component serial numbers, for use in part lifing, and vehicle dynamics calculations, such as wheel rates. I personally choose not to include these on my setup sheets. They aren't essential to defining the car configuration and clutter up its use by the crew. If needed, I think they should be on a separate document, or an "engineer's version" that can be separately printed.

We have to remember that a race car is always changing, and we race or test it as a snapshot in time. Some of these evolving changes are permanent, some not. They rarely seem to fit the existing format for the setup sheet. We have to decide whether, when, and how to indicate these changes on the setup sheet. I tend to mention permanent modifications in the comments section at the time they first appear, then delete them on future sheets - a solution I'll admit to being imperfect.

Sharp readers will have noticed no mention so far of engine configuration and tuning, nor of configuration options and file names for ECUs, data acquisition, traction control, no-lift shift, ABS, or any other electronic systems. Engines tend to be assembled, tuned, and maintained by a separate group which may or may not be part of the team. The electronics are typically maintained and tuned by one or more specialists, a process that can be a bit undisciplined, if nonetheless superbly executed. In an oddity of how things have evolved, the setup sheet is typically the configuration for the rest of the car.

One possible solution to some of these concerns is a "build sheet", produced either as a separate document or as a different print option, similar to what we've discussed for components and engineering data. It can include all sorts of information, like serial numbers, part numbers, modifications, file names, and so on.

So, here's my recommendation:

Use a comprehensive setup sheet that defines everything adjustable or changeable on the race car. Permanent modifications are either excluded or get a mention in the notes section at the time they are done. If it's appropriate for your situation, create a separate build sheet, either to define the car more fully or for part serial numbers. Keep engineering calcs off the main sheet. A worksheet accompanies the car to the setup pad for note-taking during the setup process.

To give you something to think about until the next post, here's a recent sports car racing prototype setup sheet. This sheet is fairly comprehensive, yet this car lacks certain suspension geometry and aero options that I've had on other recent setups. Numbers and other fields, of course, are changed to disguise the real setup. Fields calculated internally by the setup sheet show in blue. If you purchase the full XLS, you'll see the non-printing calcs for ride heights and gears.

PDF - Print or Free Download XLS - Purchase Download

Using Scribd

Hi folks.

There's a major new series coming for the Race Engineer blog. We'll be covering setup sheets in quite a bit of detail.

And, we're going to offer content for download in the Setup Sheet series, as well as in future ones. I'll be using http://www.scribd.com/. If you're not familiar with the site, it's sort of analogous to iTunes, only for documents instead of music and video.

Every document will be available free as a PDF, either to print or download. Not free beer, not free lunch, but the best I can do in a blog.

Many will also be available for purchase in the native format, which will typically be XLS. For forms with a lot of formatting and information, like setup sheets, this will save quite a bit of time compared to creating your own version from scratch. For real engineering tools, you get all the calcs behind the visible input and results.

I puzzled some over pricing. Some of this stuff is pretty simple, but some of it represents literally days, if not weeks, of work. Radiohead's pricing strategy of letting the user decide how much to pay was intriguing, but Scribd doesn't work that way. I do need to get something for my effort, but on the other hand I want to make this stuff widely available. In the end, I decided to make the forms $10 each, unless they are really simple. You're already paying that for an album download on iTunes or Amazon. And it's waaaaay cheap compared to the time you'd spend duplicating it. Real engineering tools will be priced according to their content.

The imbedded PDFs will display a frame from Scribd. Preceding each will be two links, one to the PDF, one to the native format.

Click the PDF link to print or get the free download.

Click the native format link to purchase. There will be a big yellow button "Buy Now" on the right side of the screen which takes you to the typical online purchase dialog to enter your credit card info.

The Scribd site asks you to register to get the free download or print. Wish they didn't do that, but at least it's free. Oh well...

As a sample, here's a setup sheet for an IRL car in 1998. Left or right click on the Scribd window to activate the various controls, zoom, and so on.

PDF - Print or Free Download
XLS - Purchase Download

Please drop me a comment if all this doesn't work correctly for you.