server_select: Keep track of which server listeners are full