As David said, the Sip phone (in your case Xlite) is responsible to do echo cancellation.
Anyway, you can try the following steps:
1. Check with Counter Path to see if their commercial version has a better AEC (Acoustic Echo Cancellation).
2. Try activating the echo cancellation in Asterisk. It might work if the delay is minimal.
3. Try one of SoliCall's product. I suggest to try the Personal PBXMate http://www.solicall.com/products.html#PersonalPBXMate
. It should cancel echo coming from either side of the call.