I think people say "It was his time" because it makes them feel better. People find it comforting to think that this death, at this time and in this way, is The Way It's Supposed To Be. It's somehow easier to accept death if God or the Fates or Destiny is in charge of it.
But it is the way it
is suppose to be. That is what people forget, especially if religious inclined.
When a 5 yearold girl dies because a car smash into her, that is as natural and normal as 80 yearold grandma dying in bed.
Actually, Grandma is LESS "natural" as she only lived that long because of our medical advancements and better and healthier lives we live. People forget this, we do our best, but we may die in a car accident or by drinking to much beer and having our bladder blow... It is as simple as that.
Everyhing that happens happens the Way is is Suppose to happen, There is no special destenity, only the world following the laws and regulations that they have, if this ends with you dying at the age of 110 with you wetting the bed and no teeth or getting a TV smashing down on your head when you crawling out from under the table to connect the plug does not matter.
The real world is amazing, stuff happens, deal with it.