1. Select Enable Queue under Chat Button.
2. Set the Overall Queue Size.
In my example, I have set it to 3. So, if all the agents are at 100% capacity, 3 chats can be queued. If additional Chats comes in, it will be denied.
How to avoid “Your chat request has been canceled because no agents are available.” in Salesforce Chat?