on the surface (with an outflow boundary condition) area-weighted average velocity magnitude is smaller then area-weighted average Z-velocity.. how it's possible?

I've seen the exact same problem that you are describing. For me thus usually has a less than 1% error. These surface integrals are only approximate, thus one can't be compared to the other. If you have significantly more than 1% error, then there might be an actual Fluent problem and they should be contacted.

I tested just counter flow in an annulaus where two different fluids are separated by wall and to my surprise when I specified the inlet as velocity inlet for both fluids the surface integral flow rate was changing between iterations! where it should be constant because velocity is specified!!

How can that be possible!!!!

I've seen the same phenomena and I think it's something like this. Velocity magnitude is a vectorial sum of x-, y- and z- velocity. When you have velocity component besides your primary flow (in z-direction), the velocity magnitude will be larger then the components itself.