He told MUI Goku to just knock him off. He also needed a pep-talk from Toppo so Frieza didn't knock him off.

@green_burstxxx: Mentally broken. Watch Aldo vs Conor on how much a warrior Aldo was until he got mentally broken by Mcgregor. Jiren had to be mentally motivated by Toppo.

@gokugoat: Will-power is mental....

I'm not so sure it was "willpower" that held back the SB, more a Ki Wave from his eyes. Similar techniques have been seen before, such as Ki Shouts or whatnot. Jiren's power was so far above the Spirit Bomb he could hold it back with simply his Ki Stare, if that's a good enough term for it. And even if it was his "willpower" that held it back, he was very confident and standing on top of the world at that point. Nobody could match him there. But when Goku managed not only to survive him twice but match and surpass his power, Jiren's spirit was broken as his entire pride was built upon his strength. When he faced the possibility of losing, Jiren's confidence crumbled as his past trauma was basically pointing him towards losing everything again because he was too weak. Furthermore he saw Goku's way of fighting/living as weak and couldn't accept that this method was surpassing his own. So it's certainly a matter of Jiren's current state of mind as the fight progressed.