Bayesian Optimization
- Explore:
- Bayesian Approach
- Metron Solutions
We use Bayesian optimization as a cost-effective method for solving "black box" function-optimization problems in machine learning by strategically choosing a highly informative set of points to sample.
Join Our Team
Bayesian Approach to Optimizing Black Box Functions
Advances in computational power have lowered the cost for developing machine learning systems and opened new areas of research from neural networks and robotics to resource mining and medical research. Optimizing machine learning functions, however, can be problematic and expensive. Our data scientists use Bayesian optimization to cut the computational cost of finding optimized machine learning solutions by strategically selecting and minimizing the number of required evaluations of the function.
References
Metron Solutions
DARPA Fundamentals of Design
Metron successfully applied Bayesian optimization in the DARPA Fundamentals of Design program, delivering a measurable improvement in the optimization of an expensive-to-evaluate, black box function. The goal of this high-risk, high-reward research program was to aid in the conceptual design process, discovering nonintuitive and novel designs. To search the control parameter space, rather than applying a traditional slow and expensive grid-search approach which tries all possible combinations of parameters, we used Bayesian optimization, reducing the number of function evaluations by an order of magnitude.
Optimizing Machine Learning Hyperparameters
Training a deep neural network is a computationally rigorous task, and the performance of the trained model is often highly dependent on hyperparameters specified before a training run. These hyperparameters may describe network architecture (e.g., the number of layers in a feed-forward neural network) or the optimization routine (e.g. the learning rate). Given a relatively small budget for training runs, Metron data scientists use Bayesian optimization to methodically choose combinations of hyperparameters, saving computational resources and elevating model performance.
Related Content
Metaheuristics
Metron applies advances in artificial intelligence to reduce the computational cost required for searching complex solution spaces.
Explore Metaheuristics
Nonlinear Gradient Descent
Metron scientists use nonlinear gradient descent methods to find optimal solutions to complex resource allocation problems and train neural networks.
Explore Nonlinear Gradient Descent
Reinforcement Learning
Reinforcement learning uses trial and error feedback to teach machines how to achieve a goal. Metron develops AI systems that teach themselves.
Explore Reinforcement Learning
Stochastic Dynamic Programming
Metron data scientists apply stochastic dynamic programming methods to solve incredibly large and complex planning problems.
Explore Stochastic Dynamic Programming
System of Systems
Metron is a pioneer in advanced decision aid technologies for composing systems of systems and mission plans.
Explore System of SystemsFeatured Team Member
Jamie Prezioso, Ph.D.
my role at metron
Machine Learning Expert
my metron Experience
As a research scientist at Metron, I am an innovator who develops principled mathematical approaches to properly characterize and manage uncertainty within complex AI models. Together with my Metron colleagues, I advance the state of the art in Bayesian neural networks and the application of Bayesian optimization to train robust probabilistic models on sparse data.
Metronite since
2018
Division
Featured Team Member
Metron Careers
Decision Support Career Opportunities
Metron hires research scientists with experience developing novel approaches that advance the state of the art in mathematics and artificial intelligence. Our scientists work alongside subject matter experts applying these innovations to new problem domains.