Newt Gingrich made the mistake of calling Shirley Sherrod "viciously racist" but the White House is at fault, the former House Speaker said Sunday.