Whitebeard avoided Kaido even when he was told to avenge Oden
Roger didn’t want to fight Big Mom
Kaido and Big Mom are relative in power to Primebeard, Roger, and Shanks.
There is no such thing as Pirate King Level. Roger just happened to achieve the most out of all of them, had the strongest crew and perhaps he's the strongest by a very small amount so he gets the title Pirate King. Rayleigh being Yonko level doesn't disprove my statement because Roger wasn't stated to be far above him. They could've been kind of close in power considering how legendary Rayleigh was made out to be
In a 1 on 1 fight Roger would have his hands full with Kaido, Big Mom, Shanks and Mihawk. He would have to extreme diff them to win. The weakest is Big Mom and even she would warrent a high diff and nothing short of that. All of them can hang with Roger even though they'd lose