The containsKey() method of the CaseInsensitiveMap isn't case insensitive. It seems that this method isn't overriden at all by collections-generic.
The following code snippet provides a good enough implementation with the correct behaviour:
@Override
public boolean containsKey(Object key) {
if (!(key instanceof String)){
return false;
}
return super.containsKey(this.convertKey((String)key));
}