posted by organizer: albertberahas || 536 views || tracked by 1 users: [display]

HOO 2022 : Order up! The Benefits of Higher-Order Optimization in Machine Learning: NeurIPS 2022


When Dec 2, 2022 - Dec 2, 2022
Where New Orleans, LA
Submission Deadline Sep 22, 2022
Notification Due Oct 20, 2022
Categories    optimization   machine learning   higher-order

Call For Papers

Optimization is a cornerstone of nearly all modern machine learning (ML) and deep learning (DL). Simple first-order gradient-based methods dominate the field for convincing reasons: low computational cost, simplicity of implementation, and strong empirical results.

Yet second- or higher-order methods are rarely used in DL, despite also having many strengths: faster per-iteration convergence, frequent explicit regularization on step-size, and better parallelization than SGD. Additionally, many scientific fields use second-order optimization with great success.

A driving factor for this is the large difference in development effort. By the time higher-order methods were tractable for DL, first-order methods such as SGD and it’s main variants (SGD + Momentum, Adam, …) already had many years of maturity and mass adoption.

The purpose of this workshop is to address this gap, to create an environment where higher-order methods are fairly considered and compared against one-another, and to foster healthy discussion with the end goal of mainstream acceptance of higher-order methods in ML and DL.

Plenary Speakers:
- Amir Gholami (UC Berkeley)
- Coralia Cartis (University of Oxford)
- Frank E. Curtis (Lehigh University)
- Donald Goldfarb (Columbia University)
- Madeleine Udell (Stanford University)

We welcome submissions to the workshop under the general theme of “Order up! The Benefits of Higher-Order Optimization in Machine Learning”. Some examples of acceptable topics include:
- Higher-order methods,
- Adaptive gradient methods,
- Novel higher-order-friendly models,
- Higher-order theory papers,
- and many more.

For submission details, please see Please use our CMT submission portal which can be found at the following link:

Important Dates:
Submission deadline: September 22, 2022 (AOE)
Acceptance notification: October 20, 2022 (AOE)
Final version due: TBD

- Albert S. Berahas (University of Michigan)
- Jelena Diakonikolas (University of Wisconsin-Madison)
- Jarad Forristal (University of Texas at Austin)
- Brandon Reese (SAS Institute Inc.)
- Martin Takáč (MBZUAI)
- Yan Xu (SAS Institute Inc.)

Related Resources

MLDM 2023   18th International Conference on Machine Learning and Data Mining
ICRRR 2023   2nd International Conference on Refugee, Resistance, and Recognition: Global Literary Representations in [Post]post-colonial Perspectives
IJCNN 2023   International Joint Conference on Neural Networks
HOPE 2022   ACM SIGPLAN Workshop on Higher-Order Programming with Effects
FAIML 2023   2023 International Conference on Frontiers of Artificial Intelligence and Machine Learning (FAIML 2023)
CGO 2023   ACM/IEEE International Symposium on Code Generation and Optimization
CVPR 2023   The IEEE/CVF Conference on Computer Vision and Pattern Recognition
ICICO 2023   6th International Conference on Intelligent Computing & Optimization 2023
Distributed ML and Opt. 2023   Distributed Machine Learning and Optimization: Theory and Applications
ECIR 2023   45th European Conference on Information Retrieval