That is an interesting question. What if you don't particularly like your ancestors? What if they are even verifiably horrible people? I will say that the way I see my grandfather or uncle act on the other side is quite different than they did in life. Death "chilled them out" for lack of a less hilarious pun. It removes the essential motivation for a lot of negative behaviors. One ancestor I feel quite close to I never met in life but simply own his Masonic ring. Am I "making things up" about said ancestor in the process of my "ancestor worship gnosis"? Perhaps. Although research has verified some impressions. Views of him among reletives I have known are pretty skewed as he had a long term mistress that my great-grandmother actually ended up taking care of after he died.. unknown if I have "black sheep"/illegitimate relatives from her side of the family... i'm actually pretty sure I do intuitively... The idea behind the school of ancestor veneration I follow is that they (or rather the shells that were their memories/thoughts/etc... a spark of their soul leaves most of this behind and "reincarnates".."if it so desires"..(?)) are being elevated through their contact with you and your regard for them. As you are teaching them something through your worshipfulness (and you both "worship" or attune yourselves with a higher-divine-I'm-not-sure-what-to-call-it in this view of things), and as a result they share the things that they learned in life which you need to do to succeed. My grandfather for instance, quit both smoking and drinking in his lifetime, and has offered me help quitting any vice I find myself bound by... far earlier in life that he was able.
I understand the idea of worshipping human ancestors generally, but I think there is something particularly meaningful about the biological chain which brought you into being. In some ATRs there is something called "mojuba" where you name all the dead relatives you can remember the names of on both sides of your family, then say (roughly) "bobo de egun (all the dead ancestors) of my mother, bobo de egun of my father" to fill in all the gaps back to the stone age. I found this experience to be deeply moving for me personally, and it continues to unfold. I see traits my ancestors left on my parents which cause problems I have now in life. Well outside their help in my magical practice I found it a very useful reflection. And nearly any person decended from english speakers can find out all about their geneaology from your local Mormons.
I'd just like to point out that I think spirits of the dead such as I describe here are NOT the same spirits that cause poltergiest phenomena and so forth. They are actually quite different and in my opinion any coincidental relation between these spirits (the poltergeisty ones) and ancestor-type dead spirits is most likely a trick by the former to get you to invest in emotional attachment they are draining off you and using to generate the phenomena in the first place. Just FYI to all the randoms of the interwebs.