📈 #71 When problem-solving becomes a passion: my journey into optimization

With lessons learned along the way

October, 2012.

Optimization of Communication Systems.

I didn’t even know what would that mean. Now I know.

It was an entire rabbit hole that up until today (and I hope that for many, many more years) I’m still exploring.

I started with algorithms, but now I’m more focused on giving solutions to businesses. What happened in between then?

Today in Feasible I’ll tell you:

• 📚 Key breakthroughs and lessons learned along the way

• 🎓 Why I decided to pursue a PhD in the field

• 👣 My initial steps into metaheuristics

I hope this post serves as inspiration for you to make a decision in the future, or that you feel reflected on the story. It wouldn’t be the first time I hear “I fell in love with optimization from the very beginning as you”. If it’s like that, I’d love to know your story.

Ready? Let’s dive in… 🪂

📚 Key breakthroughs and lessons learned along the way

For almost 3 years and a half, I worked on problems that resemble others in real-life scenarios when minimizing retrieving times of products from warehouses.

And I learned different things along the way…

💯 Personal dedication

→ During a PhD, you get to invest a lot of time in your project, so it’s crucial that you have a passion for the field. Otherwise you’ll burn out.

But it’s equally important to have a good relationship with your advisor(s), and that you like their way of working. That was my case, and I emphasize this to everyone that wants to do a PhD because I’ve seen other people giving up because of a bad relationship despite their passion for the field.

→ Regardless of whether the problem is rooted in real-life scenarios or is more abstract, give your 100% to learn as much as you can. This period of intensive learning is likely unparalleled and may not present itself again in your career.

I gave my 100% so much that I even designed the cover of my PhD dissertation. As I was solving problems related to warehouses, this picture represented a warehouse being digitized (the 0’s and 1s) to be finally converted into a mathematical function:

⚖️ Refinement of confidence

After just a few months, I felt the confidence in myself so that I told my advisors that I was 100% sure of something I don’t remember.

But it was the opposite.

My gut feeling failed miserably.

Now, if I’m pretty confident about something, I say I’m 99% sure about it, leaving some room for failing.

There are always some uncontrolled variables and complexity inherent in real-world problems.

🚦 Critical moments and breakthroughs

→ As said, I gave my 100% and at the same time I built a strong confidence in solving these problems.

So I reached a point where I dominated the problem to such an extent that I could predict which strategies would succeed or fail even before coding them.

I remember being in my advisor’s office together with him and my co-advisor, discussing a couple of strategies to improve results. I personally selected the one to start with, and it was the one that gave the best results.

That moment was gold.

→ After a few months developing solutions for the problem and building a strong algorithm, I found a paper I didn’t read with an algorithm that gave super good results.

It was an Attribute-Based Hill Climbing, and its solutions to the problem were awesome. So impressive that my algorithm was pointless, and I needed to improve quite a few things. And the algorithm that was so easy to understand became a mini monster with lots of ideas.

I finally cracked it, but it was a hard time as I thought it was going to be impossible to beat.

🍽️ Theoretical and methological lessons

→ Remember that VNS couldn’t improve the results given by the Simulated Annealing back in my Master’s course? (well, you cannot remember it… it’s at the end of this post 😅)

Well… I based my entire PhD on VNS as the central part of every problem tackled. It was so powerful that other strategies that I tried, failed.

This reminds me of the no-free lunch theorem. There’s no miracle algorithm that cracks all the problems.

(maybe now you understand why I told you that VNS usually outperforms Genetic Algorithms)

→ One of the problems that I tackled was a minimization of the maximum value of retrieving times for workers in the warehouse.

These problems tend to have a flat landscape, meaning that there are minimal differences in values when navigating the search space.

For these problems, I’d suggest you to go for simple ideas with parallel algorithms instead of investing time finding the revolutionary strategies.

🧩 Hybridization of ideas and technical insights

→ Experiment, experiment, experiment…

I combined different ideas (like having two different objective functions) and algorithms.

Sometimes you find the sweet spot when looking at the problem with different perspectives.

→ These experiments will help you develop your creativity but also your technical expertise.

Remember the thing about the flat landscape? I parallelized the search, that’s true, but also added a cache for storing the difference of the objective function when looking for solutions in a specific neighborhood, but it required a lot of effort to keep it updated for another one.

🎓 Why I decided to pursue a PhD in the field

The more I worked with optimization problems and the algorithms to solve them, the more I realized I wanted to explore the field more deeply.

That’s why, by the end of the 2013 (as I was doing my Master’s thesis in robotics and it took me a bit longer than expected), I sent an email to the professor of Optimization of Communication Systems, Eduardo G. Pardo. It was something like (unfortunately, I cannot find the email):

Hi Eduardo,

I’m Borja, your Master’s student at Optimization of Communication Systems.

I really enjoyed the subject and I’d love to go deeper. Do you have any optimization project at your department, or know any company that I can join to dive into optimization?

Thanks,

Borja.

He answered me back. He said "Borja, I can offer you a PhD student position for 2 years.”.

Two years? I thought.

I asked people around academia. Some friends were already engaged in that path. And all of them replied me with the very same words: “two years are not enough for doing a PhD”.

What was the point on doing part of a PhD in exchange for some peanuts (that wasn’t a salary, really) if I didn’t want to stay in academia after that period of time? I should look for a job anyway after that, and left the job half-done. That’s something I don’t like at all.

So I said no.

Eduardo replied me back: “That’s OK, but just think about it, and come back to me in a few weeks”.

By that time, I was going to the gym every morning at 9:00 AM with my best friend, so I had plenty of time discussing it. Also, my girlfriend (now my wife) gave me very good points.

On one side of the scale, I was going to earn little money and I wasn’t going to have enough time to finish the work I was going to start. On the other side, a deep dive into algorithms for solving challenging optimization problems.

Again, I need to be honest: I always thought that having a PhD here in Spain was pointless if you didn’t want to stay in academia. And I didn’t want that, meaning that I didn’t want to chase the title itself.

But there was one sentence from my girlfriend that changed everything.

At least here in Spain, it’s not that usual that you can take the PhD path easily in the field that you like the most. It’s like you need to ask for a lot of professors, see if they have a research topic that you like, then they need to open a public position, you fight for it with other candidates… It’s kind of a hell in most cases.

“Borja, you’re so fortunate that people usually fight for these positions and they’re putting it into your mouth, and you want to dive deeper into algorithms… Why wouldn’t you accept the position? What’s the worst thing can happen? That you dive into what you love the most?”.

Something clicked into my head. Those were powerful words to me, really.

So I replied back to Eduardo after a couple of weeks. There was one question that I wanted to confirm. “Eduardo, I’m thinking about it, but I’m not sure the application of this knowledge outside academia. Do you think this is useful for the industry?”.

Borja, this is the most applicable knowledge you will have for the industry. Optimization is everywhere.

And he listed some examples from different industries.

I must admit it: that was also key for making the decision.

Now that I had a crystal clear path, even with its drawbacks, I could decide better:

“Eduardo, let’s start right away”.

👣 My initial steps into metaheuristics

In the scientific context, optimization is the process of finding the best possible to solution for a specific problem.

That was huge.

And it was on the first slide of the optimization course I took back in 2012 (if we don’t take into account the cover nor the table of contents).

I cannot lie: the combination of very well-explained complex concepts together with the subject itself was the thing that got my heart.

When you combine both of them, oh boy, that’s pure gold. Add a bit of competition to:

• Get the best possible result for the problems you’re addressing,

• In the minimum amount time possible,

• Of all the class.

And then you get a perfect match.

The subject was divided into 7 algorithms for 3 problems:

• Simulated Annealing (SA) and Variable Neighborhood Search (VNS) to solve the MaxMin Diversity Problem (MMDP).

• Greedy Randomized Adaptive Search Procedure (GRASP), Ant Colony Optimization (ACO), and Iterated Greedy (IG) to solve the Minimum Cut Linear Arrangement (MCLA).

• Scatter Search + Path Relinking (SS+PR) and Genetic Algorithm (GA) to solve the Capacitated P-Hub (CPH).

So you could solve the MMDP with just SA (then you get one certificate), or both SA and VNS (to get two certificates), or solve the MCLA with GRASP and ACO (to get two certificates)… You know where I’m heading? The objective: to get at least 3 certificates, and the more you get, the better your grade was.

My first combination was using SA to solve 20 instances of the MMDP, and I got very excited because…

I got almost the best possible solution for each instance. The algorithm couldn’t get the best one in just one of them!

I cracked it. This is my field.

That’s what I thought at first. As I enjoyed learning about VNS, that was my next algorithm to solve the very same 20 instances of the MMDP.

It failed miserably:

So as we say in Spain, una de cal y otra de arena.

But of course I didn’t stop there and I developed a GRASP and an IG to solve the MCLA, then a Scatter Search (without Path Relinking, I’m sorry) to solve the CPH.

Lots of challenges and competition inside classmates.

After finishing that subject, I had another one in the second semester about exact algorithms. I need to be honest: I didn’t enjoy it that much.

🏁 Conclusions

Going back to October, 2012, I don’t regret any of the steps taken from that point on.

The career path that continues until today started with just one subject at my Master’s, but it has been characterized by constant evolution.

What continues to fascinate me about this field is the elegant tension between theoretical depth and practical utility. Each new problem presents a fresh puzzle, requiring both creative thinking and rigorous analysis to unravel.

As I look to the future, I see a field that continues to evolve through

• The integration of new computational paradigms, like Quantum Computing.

• The application to emerging problem domains, like the electrification of transport in Europe with Amazon CHALET.

• The cross-fertilization of ideas across disciplinary boundaries, like Mind Evolution and the frontier of LLM-based optimization solvers.

I really hope you found this article relevant to you, but in any case I’d love to hear from your experience as each of us have a unique one.

Let’s keep optimizing,

Borja.

🗞️ Other related posts people like

📈 A poll, Graph coloring open-source library in Python, and what does Graph Theory reveal about social media?

📈 #70 Are Product OR Engineers the future of Operations Research?

📈 #53 Latest news in Operations Research (pre-holiday season edition ☃️)

📈 #48 Gurobi vs PDLP: The CPU Showdown with SolverArena