GSoC’20 begins - Differentiable Optimization problems
Learn more about initial steps while developing DiffOpt.jl to differentiate some optimization problems
Hi all! I am excited about my project Differentiable Optimization problems in Google Summer of Code 2020. This is the 1st blog in a series, which was encouraged by our mentors at NumFOCUS. In this blog, I will share progress about the project and my involvement in the JuMP.jl community so far. You can find the proposal and more about the GSoC project here.
All about GSOC
This is my 2nd time doing a GSoC Project. Before drafting this blog, I reflected upon my previous GSoC project and how I shared my experiences then. If you’re interested in how to apply for GSoC, you can find all about it here.
Why I chose this project?
At the inception of COVID-19 crises and beginning of a nationwide lockdown in our country, one of my batchmates suggested this project to me. It was enticing to me in first sight as I was pursuing my Master’s project in large scale convex optimization. I began finding more about the project, JuMP, and Julia. As I went through the details of the project - the requirements, deliverables - it became clear to me that it perfectly aligns with my interests. Also, this was a great learning opportunity for me as I was not well versed in Julia and had almost no experience in differentiable optimization.
A shoutout to the Julia community
While drafting the GSoC proposal and making initial contributions, I had a excellent experience. Specifically, Julia is a really good language - just after my first few Julia/JuMP scripts, I enjoyed writing models in pure symbolic mathematics. And, the Julia community is welcoming, and I received help from several Julia project maintainers, contributors and my prospective mentors (cheers to Benoît, Joaquim, Mathieu, Mario). It wouldn’t be wrong to say that such a warm experience propelled me to contribute more to this great community!
Why are Differentiable optimization problems important?
Differentiable optimization is a promising field of convex optimization and has many potential applications in game theory, control theory and machine learning (specifically deep learning - refer this video for more). Recent work has shown how to differentiate specific subclasses of convex optimization problems. But several applications remain unexplored (refer section 8 of this really good thesis). With the help of automatic differentiation, differentiable optimization can a significant impact on creating end-to-end systems for modelling a neural network, stochastic process, or a game.
The project - overview
JuMP is a modeling language for mathematical optimization embedded in Julia. It supports many solvers for a variety of problem classes and has many features. The project aims at equipping JuMP with the ability to differentiate specific convex optimization problems in Julia. In conjunction with another concurrent GSoC project, we will enable JuMP to differentiate an optimization problem with respect to parameters (for instance, its problem data).
The project - progress
The GSoC community bonding period and first two weeks of the GSoC coding period have passed at the time of writing this blog. You can find the project repository here (DiffOpt.jl). Currently, DiffOpt.jl is able to differentiate LPs and QPs written in MOI, taking references largely from QPTH. We are continuous discussion on the slack channel (refer the next section). We are having a bi-weekly catchup call for discussing issues, setting objectives (plus I’m attending JuMP monthly developer call too!)
Our objectives for the next 2 weeks:
- Making DiffOpt robust - improving testing on several optimization problems
- Turning DiffOpt to a MOI optimization layer
- Supporting conic programs in DiffOpt
Staying in touch
If you’re interested in knowing more about the project, join the #opt-diff-gsoc channel on julia slack.