You'll always start off with at least one Ka-Bar.
While you can't throw that Ka-Bar blade away, it can come in handy for humiliating stealth kills.
From the front it will take anywhere from 2 to 6 slashes to kill someone. If you manage to sneak up on them from behind, a single slash could bring them down. And then afterwards you can wipe your blade clean of blood.
If you're willing to spend $10 you can buy extra knives (maximum 7) which you can utilise when altering the change firing mode button. The throwing knives are horribly inaccurate at any range, but they do a decent amount of damage and you can throw them quickly with ease. Usually two or three thrown knives can take an opponent down. If they happen to land on the ground or get stuck in a wall or something, you can pick the knife or knives up again and throw them some more.