In Salesforce, by default, the Omni-Channel Supervisors can view all the Online agents.

The Supervisors can view Offline agents if they have access to the Agents respective Service Resource records. So, share the Service Resource records to the Omni-Channel Supervisors. So that, they can view all the Offline agents from the Omni-Channel Supervisors tab.

Quick Workarounds:

1. Give View All permission for the Service Resource objectin the Profile.

2. Set OWD for Service Resource object to Public Read-Only.