On 28 Sep 1786 Joel first married Abigail Goddard (48650) , daughter of Samuel Goddard (31261) (13 Jul 1732-12 Jun 1803) & Abigail King (38471) (7 Feb 1740/1-8 Apr 1804), in Sutton, MA.566 Born on 8 Jan 1761 in Worcester, MA. Abigail died in Sutton, MA, on 24 May 1790; she was 29.

Abigail, also called Nabby, was of Worcester when she married Joel.566

On 13 Dec 1798 Joel second married Elizabeth Bancroft (63245) , daughter of Moses Bancroft (56123) (7 Sep 1723-1797) & Elizabeth Lamson (7 Oct 1727-), in Sutton, MA. Born on 12 Aug 1759 in Sutton, MA.