The knife attack occurred on Saturday afternoon.
The suspected perpetrator, who has been arrested, is said to be a 37-year-old Algerian. He is known to the police and the intelligence service, and was supposed to leave France, according to the TV channel TF1.
President Macron stated in a statement later on Saturday that it was an "Islamist terrorist attack".
The suspected perpetrator had first attacked local police officers, shouting religious phrases at the same time. A person who was near the incident tried to stop the suspected perpetrator, who responded by stabbing the man to death.
The prosecutor in Mulhouse has identified the victim as a 69-year-old Portuguese citizen.
France's anti-terrorism unit at the prosecutor's office (PNAT) is taking over the investigation of the case. According to the news agency AFP, the suspected perpetrator was on the authorities' surveillance list for terror.
France's Interior Minister Bruno Retailleau is expected to visit the crime scene in the evening.
"Fear has struck our city" writes Mulhouse Mayor Michele Lutz on Facebook.
