There is full page dedicated to Joseph and Robert Taylor in the Biographical and Genealogical History of the State of Delaware Vol 1, page 470. It was published in 1899. Robert was very active at that time and probably provided much of the information. It fills in some of the gaps not found in census or deed documents. It states that Joseph and Lucy Taylor had three children, all born in Hyde, but 2 died there:, Maybe why they migrated:

Robert Sawyer Taylor
Eli Taylor - died in Hyde
Hannah Taylor – died in Hyde

The family moved a lot. It appears their first home was in Pittsburgh. They are on the 1850 Census living in Birmingham borough of Allegheny County.  page 62. This is on the  south side of the  Monongahela  River in what is now greater Pittsburgh. Joseph Taylor is 39 and a shoemaker, Louisa is 42 and Robert 15. all born in England.

According to the Bio the family moved to Louisville Ky in 1850 - would have been after the census was taken in PA.In Louisville he was trained as a  glass and bottle blower. In 1853 they returned to PA, Bridgewater. There are actually 3 Bridgewaters in PA, but I believe it most likely they were moving via the Ohio River, ie Pittsburgh to   Louisville, then back up the river to the Bridgewater that is on the Ohio just west of Pittsburgh. During most of these places, Joseph worked as a store clerk. Finally, he family moved to Delaware where Joseph's brother, James Taylor owned and operated a woolen mill on Pike Creek in Mill Creek Hundred. Joseph worked in the mill until it closed when he returned to being a shoemaker.

          1870 Delaware Census, Mill Creek Hr  Pleasant Hill roll M593_120, page 572:
Robert Taylor, 35 farmer born England
Anna Taylor, 33 born Delaware as were all children
Lucy Taylor, 10
George Taylor, 8
Catherine Taylor, 6
Sarah E. Taylor, 4
Katherine Kelly 69, Anna's mother

Joseph and Lucy Taylor. were next door
The 1880 Delaware Census, Western part of Millcreek Hnd, page 26
Joseph Taylor, age 69, Cobbler born in England
Lucy Taylor, age 72, born in England 
I believe he had a shoe shop in Stanton.  On a map of Mill Creek hundred, no date, but it shows the woolen mill on Pipe Creek. It is one of those wonderful maps with lots of names of land and business owners.   R. Taylor is right on pike creek Rd just beyond Old Coach Rd.  J.H. Taylor is on the opposite side of Pike Creek right on Old Coach Rd. and there is also JH Taylor in Stanton near the current intersetion of Telegraph Rd &
 rt. 7.  We still own a small steelyard (scale) that Joseph Taylor must have used in his cobbler shop. 

While their son, Robert married a Presbyterian and is buried at White Clay Creek Presbyterian Cemetery,  Joseph and Lucy Taylor stayed close to their English roots.  They were members of and are buried at St. James Episcopal Church.  The church dates from at least 1726.  The current building was constructed in the 1820s, so is already 200 years old.  It is near Stanton, but found just off Kirkwood highway on St. James church Rd which is at the other end of Telegraph Rd.  

https://www.findagrave.com/memorial/45706215/joseph-taylor   Joseph Taylor died in 1884.  His wife Lucy Taylor died  in 1892.  They share a tombstone in St. James Cemetery.  I never saw what the H. stood for on the Mill Creek map.