shibendu_Q2
22 points
Regarding the steep descent of the nash distance, most of the numerical optimization problems use some modification of a commonly used technique called gradient descent. Here is one analogy used to describe this algorithm(from wikipedia)
The basic intuition behind gradient descent can be illustrated by a hypothetical scenario. A person is stuck in the mountains and is trying to get down (i.e. trying to find the minima). There is heavy fog such that visibility is extremely low. Therefore, the path down the mountain is not visible, so they must use local information to find the minima. They can use the method of gradient descent, which involves looking at the steepness of the hill at their current position, then proceeding in the direction with the steepest descent (i.e. downhill). If they were trying to find the top of the mountain (i.e. the maxima), then they would proceed in the direction steepest ascent (i.e. uphill). Using this method, they would eventually find their way down the mountain. However, assume also that the steepness of the hill is not immediately obvious with simple observation, but rather it requires a sophisticated instrument to measure, which the person happens to have at the moment. It takes quite some time to measure the steepness of the hill with the instrument, thus they should minimize their use of the instrument if they wanted to get down the mountain before sunset. The difficulty then is choosing the frequency at which they should measure the steepness of the hill so not to go off track.
In this analogy, the person represents the algorithm, and the path taken down the mountain represents the sequence of parameter settings that the algorithm will explore. The steepness of the hill represents the slope of the error surface at that point. The instrument used to measure steepness is differentiation (the slope of the error surface can be calculated by taking the derivative of the squared error function at that point). The direction they choose to travel in aligns with the gradient of the error surface at that point.
Now you can imagine that the steepness of the terrain may vary causing the optimization to reach its minima faster.
Dec. 3, 2019 | 7:46 a.m.
Excellent video QY! Hope you never run out of ideas to make videos.
June 6, 2019 | 6:56 p.m.
Very nice video. But I just noticed for 953rb board sim on the right pane, call is an option for IP. Not sure if its a PIO bug.
1kNL reg here. QY's understanding of the game is deep and backed by strong fundamentals. He is really good at explaining advanced concepts in simple terms.
I would highly recommend hiring QY as a coach.
Dec. 1, 2021 | 8:12 a.m.