Phifer is a surname. Notable people with the surname include:

James Reese Phifer (1916–1998), American businessman and philanthropist Mekhi Phifer (born 1974), American actor Roman Phifer (born 1968), American football player Thomas Phifer (born 1953), American architect