Instead of teeth, they have baleen plates, which are made of keratin (the same material as human fingernails). These plates hang down from the roof of their mouths like a curtain. They use these plates to filter out krill and other small organisms from the water.
So, the "strangeness" lies in the fact that a creature as large as a blue whale doesn't have teeth, but rather these specialized filtering plates.