Binyamin said:
It's a mitzvah. It's a way to show respect to someone who has passed by placing the stone or a -marker- at the site on which they were buried. Let's say that a tombstone has been erected, people can still do the act of the mitzvah of making a marker at a grave by adding to the stone. It has a lot to do with the symbolism of the mitzvah.
Just to add something that came to my mind.
Bereishis 35:20 וַיַּצֵּב יַעֲקֹב מַצֵּבָה, עַל-קְבֻרָתָהּ--הִוא
מַצֶּבֶת קְבֻרַת-רָחֵל, עַד-הַיּוֹם
Genesis 35:20 And Jacob erected a monument on her grave; that is the tombstone of Rachel until this day.
Matzevah is the stone that was made, so placing the pebbles or the rock is a reminder of this mitzvah that Jacob did for his wife, Rachel.