James Fifer (July 14, 1930 – June 7, 1986) was an American competition rower and Olympic champion. He won the gold medal in coxless pair with Duvall Hecht at the 1956 Summer Olympics in Melbourne.