You had 2 parents, 4 grand-parents, 8 great-grandparents, 16 great-great-grandparents, 32 g3grandparents, 64 g4gparents, . . . . by the time you get back to the middle ages you have more ancestors than there were people in the known world at that time!

The table below illustrates how the number of your ancestors rises dramatically as you go back in time.
Around 1450 the number of your theoretical ancestors exceeds the population of the British Isles.
Keep going back to the year 0 and you have more theoretical ancestors, than the number of people who have ever been born!

How?
Families whose history has been well documented, such as Royalty, show individuals appearing at two or more places in the family tree.
For example, HRH Prince William's father's mother's father's father's father's parents were Queen Victoria and Prince Albert; but they were also his father's father's father's mother's monther's parents. He has another set of duplicate great great great great grandparents, and so only has 60 great4 grandparents not 64.
Wikipedia — Ahnantafel

Your father's father's father's father's father's father could easily be the same person as your mother's mother's mother's mother's mother's father. Two of his children may have gone their separate ways, and the families would have lost touch over numerous generations. When two descendents met a hundred or more years later they might have had no idea they were distant cousins.
As you go further back, the number of such individuals will increase, as may the quantity of each persons duplications. As soon as you find one duplicate, all that person's ancestors are also duplicates, significantly reducing your actual ancestors.
 
For example, were you to find that two of your 128 potential gt5 grandparents were the same person, you would have only 126 gt5 grandparents, but this would mean that your gt6 grandparents would reduce from 256 to 252, your gt7 grandparents from 512 to 504, your gt8 grandparents from 1024 to 1008, etc. The reduction would double each generation, just as the potential doubles each generation.
 
All of which means that if your family originated in the British Isles, at least one of your 4,194,304 great- great- great- great- great- great- great- great- great- great- great- great- great- great- great- great- great- great- great- great- grandparents was the same person as one of mine! Hello 21st cousin!

All figures in columns 3 and 4 are approxiamate! Years based on an average 25 year gap between generations.

Generation Potential people Approx Year Population of British Isles
You 1 2000  
Parents 2 1975  
Grandparents 4 1950 50,000,000
Gt Grandparents 8 1925  
g2gparents 16 1900 40,000,000
g3gparents 32 1875  
g4gparents 64 1850  
g5gparents 128 1825  
g6gparents 256 1800 16,500,000
g7gparents 512 1775  
g8gparents 1,024 1750 12,100,000
g9gparents 2,048 1725  
g10gparents 4,096 1700 8,500,000
g11gparents 8,192 1675 8,200,000
g12gparents 16,384 1650  
g13gparents 32,768 1625  
g14gparents 65,536 1600 7,000,000
g15gparents 131,072 1575  
g16gparents 262,144 1550  
g17gparents 524,288 1525  
g18gparents 1,048,576 1500 6,000,000
g19gparents 2,097,152 1475 5,500,000
g20gparents 4,194,304 1450 5,000,000
g21gparents 8,388,608 1425 4,500,000
g22gparents 16,777,216 1400 4,000,000
 
A website explaining how this can create a diamond shaped family tree
 
 
Of course, there is a limit to how far back you can go, and not just limits based upon recorded history — there comes a point when your ancestor would be regarded as another species. Your g500000grandmother was the first common ancestor of humans and chimpanzees.