For the heat exchanger model, the required number of training points increases sharply after 6 dimensions. This result is expected because the variables with the most nonlinear effects are added to the model last, as explained in section 3.2.Training and prediction times are represented qualitatively in Figs. 2 – 5 by the size of the solid and translucent circles, respectively. The general trend in all four cases is that the training and prediction times of all methods increase with the number of dimensions. Support vector regression has the smallest training times in all cases. Kriging is slightly slower than radial basis functions when applied to low modality problems in low dimensions. Training times for radial basis functions become very large compared to kriging when applied to high modality, high dimensional problems. The theory behind support vector regression is very computationally efficient (Clarke et al., 2005), and other comparison studies have confirmed its low training times (Ely and Seepersad, 2009). Build times for kriging are expected to be slow because it must perform a nonlinear optimization to obtain the correlation parameters. The slow training times for radial basis functions in highly nonlinear, multidimensional problems stems from the large number of training points required to train an accurate model. During the build process, the RBF method must compute Euclidean distances between adjacent training points. The number of these calculations increases dramatically when the number of dimensions and training points is large.The manner in which the training times increase with dimensions also varies among the metamodeling methods. The training times for kriging appear to increase linearly with the scale of the problem, but the training times for RBF and SVR begin to increase sharply in the higher dimensional problems. If this trend were extended into very high dimensions, it is possible that the training times for SVR may actually approach or exceed those of kriging.Support vector regression is also seen to have by far the smallest prediction times of the three methods studied. This trend is also consistent with previous studies (Clarke et al., 2005; Ely and Seepersad, 2009). Kriging has the slowest prediction times of the three methods, being as much as ten times larger than radial basis functions in some cases. Kriging and radial basis function are expected to have large prediction times because the distances between points must be computed during the simulation process (Jin et al., 1999).Prediction time not only depends on the method used and the scale of the problem, but also on the type of problem being modeled. Even though the number of test points is the same for each respective dimension of each problem, the prediction time increases with the modality of the problem. That is, it takes a given method longer to predict 1000 new points for a more complex problem than it does for a simple one.In this paper, three types of metamodels—kriging, radial basis functions, and support vector regression—are compared with respect to their speed, accuracy, and required number of training points for test problems of varying complexityand dimensionality. Radial basis functions are found to model and predict the test functions to a predetermined level of global accuracy with the smallest number of training points for most functions. In most cases, kriging metamodels need the highest number of training points of the three methods.
Kriging has faster model building times than radial basis functions, but it is slower to predict new data points. Both methods are very slow to train and predict new points when compared to support vector regression.Metamodel-based optimization has numerous potential applications in the field of marine engineering and ship design. Building ship Training and Prediction TimeConclusions prototypes and conducting full scale physical experiments are often too expensive or time consuming to be practical in the early stages of the design process, thus driving the need for complex computer models. Using metamodels in place of computationally expensive computer models and simulations can drastically reduce design time and enable ship designers to explore larger regions of the feasible design space.The authors would like to thank the Office of Naval Research (ONR) for supporting this work under the auspices of the Electric Ship Research and Development Consortium. The guidance and experience offered by Dr. Tom Kiehne of Applied Research Laboratories are also gratefully acknowledged.BOX, G. E. P. and K. B. WILSON, 1951, “On the Experimental Attainment of Optimal Conditions,” Journal of the Royal Statistical Society, Vol. 13, pp. 1-45.CLARKE, S. M., J. H. GRIEBSCH and T. W. SIMPSON, 2005, “Analysis of Support Vector Regression for Approximation of Complex Engineering Analysis,” Journal of Mechanical Design, Vol. 127, pp. 1077-1087.DIMOPOULOS, G. G., A. V. KOUGIOUFAS and C. A. FRANGOPOULOS, 2008, “Synthesis, Design, and Operation Optimization of a Marine Energy System,” Energy, Vol. 33, pp. 180-188.ELY, G. R. and C. C. SEEPERSAD, 2009, "A Comparative Study of Metamodeling Techniques for Predictive Process Control of Welding Applications," ASME International Manufacturing Science and Engineering Conference, West Lafayette, Indiana, USA, Paper Number MESC2009-84189.FANG, H., M. RAIS-ROHANI, Z. LIU and M. F. HORSTMEYER, 2005, “A Comparative Study of Metamodeling Methods for Multiobjective Crashworthiness Optimization,” Computers and Structures, Vol. 83, pp. 2121-2136.FANG, K. T., D. K. J. LIN, P. WINKER and Y. ZHANG, 2000, “Uniform Design: Theory and Applications,” Technometrics, Vol. 42, No. 3, pp. 237-248.FRIEDMAN, J. H., 1991, “Multivariate Adaptive Regression Splines,” The Annals of Statistics, Vol. 19, No. 1, pp. 1-67.GIUNTA, A. and L. T. WATSON, 1998, "A Comparison of Approximation Modeling Techniques: Polynomial Versus Interpolating Models," 7th AIAA/USAF/ISSMO Symposium on Multidisciplinary Analysis & Optimization, St. Louis, MO, AIAA, Vol. 1, pp.392-404. AIAA-98-4758.HAMMERSLEY, J. M., 1960, “Monte Carlo Methods for Solving Multivariable Problems,” Annals of the New York Academy of Sciences, Vol. 86, pp. 844-874.HARDY, R. L., 1971, “Multiquadric Equations of Topology and Other Irregular Surfaces,” Journal of Geophysical Research, Vol. 76, No. 8, pp. 1905-1915.HAYKIN, S., 1999, Neural Networks: A Comprehensive Foundation, Prentice Hall, Upper Saddle River, N.J.JIN, R., W. CHEN and T. W. SIMPSON, 2001, “Comparative Studies of Metamodeling Techniques Under Multiple Modeling Criteria,” Structural and Multidisciplinary Optimization, Vol. 23, pp. 1-13.KIM, B.-S., Y.-B. LEE and D.-H. CHOI, 2009, “Comparison Study on the Accuracy of Metamodeling Technique for Non-Convex Functions,” Journal of Mechanical Science and Technology, Vol. 23, pp. 1175-1181.AcknowledgementsReferences