A lot of people don't believe me when I say that armor does not have diminishing returns, using various arguments and, regretfully, metaphors. I've also been asked to deal with the subject. So I hope this post can clear up the workings of mitigation and penetration.
(in this post I'll be talking about armor; MR acts exactly the same. Note that I'm ignoring negative armor)
First I'd like to introduce a term called Effective HP (EHP). It's the amount of premitigation damage (e.g. the damage the enemy has in his UI on his ability buttons) you can sustain before dying. It is derived by dividing your hp with your percent damage reduction.
Each point of armor always adds 1% of your HP to your EHP.
The formula on the wiki under damage reduction is a damage multiplier, so it's the number to multiply damage with to find out how much damage you deal. We can find our EHP by dividing our HP with this number.
Lets take an example where you have 1000 hp and 0 armor. This will put you on 1000 EHP.
Now lets add a Thornmail
1000/(100/(100+100))=2000 (1/2 mitigation)
Putting you have 2000 EHP, an increase of 1000EHP.
Now, if armor has diminishing returns, the next Thornmail added will increase your EHP by less than 1000. Lets add another Thornmail.
1000/(100/(100+200))=3000 (1/3 mitigation)
It added exactly as much EHP as the first Thornmail.
Lets have a look at how a Last Whisper and Weapon Expertise (total 40% armor penetration) affects these numbers. We have the same starting point, 1000hp with 0 armor for 1000EHP. Lets add a Thornmail
So the Thornmail effectively added 60 armor after penetration and thus adds 60% of your hp to EHP.
Lets add another thornmail
And again the second thornmail adds the same amount of EHP as the first.
Lets look at it from a less abstract angle. There's a champion attacking you. He has 100 AD and attacks once per second. You have 1000 HP and 0 armor. You'll live for 10 seconds.
Adding a thornmail, he'll deal half damage. You'll live for 20 seconds; 10 seconds longer.
Adding a second thornmail, he'll deal a third normal damage (your UI will show you 67% damage reduction), and you'll live for 30 seconds. Linear increase, no diminishing return.
Armor does not have diminishing returns.