Yes, if you didn't remove the group/userallow commands from superadmins..
You'd need to revoke group and userallow rights from superadmin, that way, they can't use those commands to give themselves the command back.
ulx groupallow superadmin "ulx groupallow" 1
ulx groupallow superadmin "ulx userallow" 1
That 1 on the end 'revokes' the right.
You'd then need to make sure to add those to the serverowner group without the revocation bit.
But.. I think Gmod, not ulx, gives the right of kicking through IsSuperAdmin.
Not even sure ULib could overright that. Prevent them from using ulx kick, sure, but not kick or kickid