Note that Kit #13316 in the Oliver Y DNA project claims this man as his ancestor.
See the Turquoise Group webpage:

information from Dr. Ward Oliver by email on April 19, 2009
"Joseph moved to Charlotte from King Wm in 1779 (not sure that he was born there, but as you said, you could have been from three different counties without ever having moved if in the right spot). I assume Joseph Sr to have been born ca 1725, based on the ages of his children. The youngest, William, came of age in (if I recall correctly) 1786 so was b. ca 1765. The sons were in order John, Edward, Joseph and William. The daughter Mary was either the 1st or 2nd child."

1782 Virginia Enumeration, Charlotte Co
Oliver, Joseph 4 whites
Olliver, Thomas 5 whites
in Sketches of Olivers by Colonel Hugh and Margaret T. Oliver in the
appendix of census data.

Will posted by B. Allen on WorldConnect
Joseph Oliver, died 1787
Will Book I Charlotte Co., VA pp 388-388a
In the Name of God Amen I Joseph Oliver in Charlotte County of the
Parish of Cornwall be in a Weak State of health but of Sound mind and
memory and knowing its Ordained that once we are all to die, I make and
ordain this my last Will and Testament: I recommend my Soul to the
Eternal God who I do put my trust in and to the worldly affairs I do
possess: I give in manner and form following ( Viz ) Imprimes, I give and
bequeath unto my Son Edward Oliver the Tract of Land I now live on to
him and his heirs forever. Item I give and bequeath unto my Son Joseph
Oliver the Tract of Land known by the name of Lumpkins Tract to him and
his heirs forever. I give to my Son William Oliver the Tract of Land
known by the name of Tanners Tract to him and his heirs forever. Item
its my Will and desire that my personal Estate be equally divided
between my Sons John Oliver, Edward Oliver, Joseph Oliver and William
Oliver. which estate I give unto them and their heirs forever but as by
Settlement it Appears that my Son John is indebted to me Two hundred
ninety five pounds, which Money I give unto my Sons Edward, Joseph &
William to be equally divded. Item its my Will and desire that my
Executors hereafter mentioned pay unto Thomas Oliver the sum of Fifty
pounds at the division of my Estate which I desire may be done by three
good respectable men: I constituere and appoint my Son John Oliver,
Edward Oliver, Joseph Oliver & Thomas Lorton Executors of this my last
Will and Testament. as Witness my hand and Seal this Thirteenth day of
May One Thousand seven hundred and eighty Six.
Joseph Oliver
Signed in the presence of
John Ward, James Penicost, W. Hubard
At a Court held for Charlotte County the first day of January 1787 This
Last Will and Testament of Joseph Oliver decd was Exhibited in Court by
John Oliver Edward Oliver Joseph Oliver and Thomas Lorton the Executors
herein named, and the same was proved by the oath of John Ward one of
the Witnesses hereto subscribed, and ordered to be Recorded, and on the
motion of the said John Oliver Edward Oliver and Joseph Oliver, who
maid oath according to Law Certificate is granted them for obtaining a
probate of the said Will they giving Security whereupon they with
Thomas Bedford their Security intered into and Acknowledged Their bond
in the penalty of three thousand pounds for their due and faithful
Execution of the said Will and Thomas Lorton Gentleman the other
Executor Renounced in open Court the burden and execution of the same.
Teste Thomas Read Clk
At a Court held for Charlotte County the second day of April 1787 This
Last Will and Testament of Joseph Oliver decd was further proved in
Court by the Oath of William Hubard Gentleman one of the Witnesses
hereto subscribed, and ordered to be Recorded.
Teste Thomas Read CC

1. Mary OLIVER b: ABT 1757 in King Willima Co, VA
2. Edward Vaughn OLIVER b: BEF 1766 in King Willima Co, VA
3. John OLIVER b: BEF 1766 in King Willima Co, VA
4. Joseph OLIVER b: BEF 1766 in King Willima Co, VA
5. William OLIVER b: BEF 1766 in King Willima Co, VA