I'm pretty sure there are things we can do to optimize it, I'm discussing with megiddo about it. Do you make extensive use of your "user" group? (ie, the group players are in when they first join your server?) If you immediately or after a short while stick them into a specific group, then you'll get a whole load of users that only play on your server once building up in your users.txt
EDIT: Okay, Megiddo confirmed to me that he doesn't actually send that users table to anyone, it's all handled through XGUI. Which means I have some optimizing I need to do lol. I vaguely remember doing some major optimizations for this very reason a while back, but.. apparently I haven't finished? (It's been a while lol)
I'll look into it as soon as I get home to let you know of the status. When I can actually find the time to make the optimizations is beyond me