Years ago I built a belt driven pump drive for a tractor. I sourced a couple of bearings for the shaft from NAPA, I don't remember the brand, maybe NAPA. The bearings didn't last long, they were cheap.
I then sourced SKF, I was told those bearing were good beyond 20,000rpm, never a problem after that. IIRC the SKF bearings were something like $40 each.
SKF makes idler pulleys designed specifically for what you want to do............beyond 10,000rpm is not a problem. And yes, idler pulley bearings are specific because of the amount of tension from the belt.
Pay for it once and won't worry about it.
https://www.summitracing.com/parts/skf-acp38001?seid=srese1&cm_mmc=pla-google-_-shopping-_-srese1-_-skf-bearings&gclid=Cj0KCQiA-8PjBRCWARIsADc18TLdl8Q9xrKK8p040-d8tVQ_aS6EuReZIzS-mGRjAnmaCuQJHb7gye8aAqtuEALw_wcB