I think they can be complementary because LLW changes the loss weights while PCGrad changes the gradient directions. I think they could be used together.
I'd be interested to try. Also I read https://arxiv.org/abs/2302.03328 which is somewhat related. It tries to learn the weights in a way to minimize regret. WDYT?
Do you think Gradient surgery can be complementary or additive in a system that already uses LLW?
I think they can be complementary because LLW changes the loss weights while PCGrad changes the gradient directions. I think they could be used together.
I'd be interested to try. Also I read https://arxiv.org/abs/2302.03328 which is somewhat related. It tries to learn the weights in a way to minimize regret. WDYT?
Wonderfully detailed! Congratulations on your deserved growth. Looking forward to reading your work in 2024. Happy holidays!