Among the dead are 15 children. Another 116 people are missing.
Survivors say the ship, which had 350 people on board, capsized in the sea north of the city of Obock in Djibouti, according to a statement from the IOM.
The boat was heading towards Yemen.
The reverse route - from Yemen to Djibouti - is a common route for refugees fleeing war-torn Yemen.
