James Edward Willoughby died peacefully at home surrounded by his loving family on July 4, 2016, in Sioux Falls, SD. He was born on September 4, 1936, the eldest son of George and Marie (Olson) Willoughby of Iron River, WI. Jim graduated from Iron River High School in 1954 and from the University of Wisconsin, Superior, where he was an active member of the Phi Sigma Epsilon - Semper Compares fraternity. Jim served in the U.S. National Guard, and went on to become a respected businessman. He was well-known for his outgoing personality and ability to make friends. Throughout his life, Jim was an avid outdoorsman, sharing his love of hunting and fishing with his family. Jim married Janice Gibis on January 23, 1965, and they had four children. Jim is survived by his dear wife Janice; their three sons, Vince (Sheila Hwang) of Kirkwood, MO; Mark (Jennifer) of Hermantown, MN; Brock (Alisa) of Sioux Falls, SD; their seven grandchildren, Alexander, Benjamin, Hunter, Luke, Maxwell, Brendan, Alivia; and his siblings Jeanne (John) Joseph of Iron River, WI and Jon (Judy) Willoughby of Lake Nebagamon, WI. He was preceded in death by his beloved daughter Heather, sister Joanne (Chuck) Erickson and parents George and Marie. A funeral service will be held July 16, 2016 at Our Saviors Lutheran Church in Iron River, WI with visitation at 1pm and funeral at 2pm.