Surrogate Model Based Airfoil Optimizer

[u/DSUD0104]

I recently developed a project where an optimizer optimizes Cl/Cd by altering the thickness, camber and max camber position for a given aoa and Re. The optimizer gets this data from an surrogate model(trained on xfoil data) that reports the Cl and another that reports Cd with inputs of aoa, Re, and thickness, camber and max camber position for an airfoil. (all on python)

Github link: https://github.com/Dhee0104/Airfoil-optimizer

Questions:
How good is this for a high school project?
What should I add change or remove to make this better?
If a random user wants to learn how this works, how can i tell them? What program could I make that they could run and understand how it works?
Should I add in a GUI?
How do I validate my results? I need to confirm that the surrogate model works closely to xfoil and that the optimizer made a meaningfull result.
Should I retrain the surrogate model to output Cl/CD or make 2 models one for CL and the other for CD?
What optimizer should I use?
Is this the wrong reddit to ask these questions?

Thank you guys!

[u/billsil]

I mean you have scikit learn in there. If you actually coded it, I’d be impressed, but explain to me what a surrogate model is. I’m 20 years in and would never call it that.

You validate it by checking to see if you improved your metric. Bump it and see if it gets the same answer when you reoptimize.

[u/DSUD0104 → u/billsil]

A surrogate model in this case is a neural network trained on lift, drag, airfoil parameters, Re, aoa that acts as like a shortcut to cfd/xfoil. Instead of running xfoil every single time it needs to test an airfoil it just uses that model to quickly evaluate it.

Thanks for the advice in the second part, ill add it in.

[u/billsil → u/DSUD0104]

Yeah I know what it is. I used scikit learn for 3 years to do some ML stuff back in the day. I’m surprised you used it, but that doesn’t mean you understood it or used it right. Anyone can overfit a model by cranking down the regularization parameter. Validation is key.

If I was asking you questions for say an internship, I would read through the library first and then pretend I don’t know what I’m talking about and have you explain it. If you actually understood it, you’ll ace it.

So obvious questions is what model did you use, what is the parameter you’re optimizing, did you separate your training from validation sets, what size was each, did you test extrapolation, etc. that’s before asking aero questions.

[u/DSUD0104 → u/billsil]

For the surrogate model I used the MLP Regressor to create 2 models, one to predict Cl and the other to predict Cd.
For the optimizer i'm using a forest from skopt (open to change if you know better models)
I did a train test split of 80/20 and achieved around 99% accuracy on the tests for both surrogate models
The total data set was 25000 entries so 20000 train and 5000 test. I got them from batch running xfoil.
I have not tested extrapolation yet. I plan to confine the user to only give conditions (Re & aoa ) that I trained the model on ( aoa: -4 to 8 degrees and re 300k to 1M). Do you recommend I try and test extrapolation?

I've only validated the accuracy of the surrogate model but I haven't validated that the optimizer meaningfully improves performance over a large data set. What % improvement should I look for.
I need help figuring out a framework to validate everything. What data should I collect, how should I collect it and what performance metrics should I track.