Why is Infinite Cheese? The name comes from the ancient Latin saying "Mi felus et quesam ad infinitum". These words, literally "My cat is cheese never-ending", were spoken by Socrates on his death bed, although after he was already dead, so they don't count as his actual last words. Some attribute them to random gurglings.
But why was Infinite Cheese chosen by Infinite Cheese? Because cheese is the ultimate symbol of our culture today. Cheese is milk that has been rotted by mold but is somehow still edible, and even kind of pleasant. Such is our society. While some compare it to the sweet, pure milk of our past and say that rotted horribly, other like ourselves, realize that we, like cheese, have rotted in a way that makes us pleasant, crumbly, and nice on toasted bread.
We realize that cheese, infinite cheese, is where our society is heading so the only way to capture the world without being detected is to blend in with the cheese. And only one thing blends perfectly with cheese: cheese. So, in a way, we had no choice. Infinite Cheese didn't choose infinite cheese. Infinite Cheese chose Infinite Cheese.