0207 GMT November 28, 2020
Fifteen others were taken to hospital but the French authorities believe at least one more could be lost at sea, BBC News reported.
Care4Calais urged the British government to create legal routes for refugees to reach the UK.
UK Prime Minister Boris Johnson pledged to "crack down" on gangs facilitating crossings.
An investigation into the cause of the sinking has been opened by the public prosecutor in Dunkirk.
Searches which paused overnight began again on Wednesday morning, but the BBC's Simon Jones said — now 24 hours on from the sinking — it was likely to be "a recovery operation rather than a rescue operation" if there are still people out in the English Channel.
Charity Care4Calais said the "loss of life should be a wake-up call for those in power in France and the UK."
It said creating a new system which would allow asylum seekers to apply for refuge in the UK from outside its borders would "put an end to terrifying, dangerous sea crossings and stop tragedy striking again."
Save The Children called for a "joint plan" from London and Paris to ensure the safety of vulnerable families, adding, "The English Channel must not become a graveyard for children."
Environment Secretary George Eustice said the deaths were a "terrible tragedy" and that there was not a reason for people to take the "highly dangerous crossing" from France, adding that it was down to "criminal gangs callously taking their money."
He said the government's approach was to "remove the incentive for people trying to take this crossing."
The boat was spotted about two kilometers off the French coast by a passing sailboat at about 09:30 local time on Tuesday, which alerted French authorities.
Four French vessels, one Belgian helicopter and a French fishing boat took part in the rescue and a search operation. But by the time rescue teams reached the struggling vessel, it had already sunk.
One man drowned, while a woman, along with two children — aged five and eight — suffered cardiac arrests and never recovered.
Fifteen other people were taken to hospitals in Dunkirk and Calais.