Gym-goers and Finnieston residents have called for plans to build flats on the site of a “cherished” gym to be turned down.