I see ICE connections have failed or not setup at all. Is ICE a server between the client and the my application server?

ICE is indeed a way to find communication between two endpoints (user A and user B) when they're behind a firewall. Those serves sometimes make the functionality of relays in the communication, and failures usually happen between end users and ICE servers (STUN or TURN).

We have an interesting blog series that talk about STUN/TURN, ICE data and disconnections. Please have a look at the following links:

What are STUN and TRUN

Why does WebRTC need a TURN server? 

ICE metrics for Contact Centers (although it explains failures in ICE)