Issue may be due to the Capacity values entered.

 

1. Check the Queue Capacity entered in the Chat Button.

 

 

2. Then, check Capacity in Routing Configuration.

 

 As per the above configuration, for each agent, 2 chats can be queued.