A majority of a panel of 16 judges found that the International Court of Justice, also known as the World Court, has jurisdiction in the dispute, Reuters reported.
Iran brought the case in 2018 after former president Donald Trump's administration imposed sanctions, following his decision to abandon a 2015 pact under which Iran accepted curbs to its peaceful nuclear program.
New US President Joe Biden has said he wants to return to that pact, although Tehran and Washington still disagree on steps to be taken for that to happen.
The United States had tried to argue that Iran could not base claims at the World Court on a 1955 bilateral friendship pact. However judges found the treaty, signed decades before Iran's 1979 Islamic Revolution and the sharp deterioration in ties with Washington, could be used as a basis for the court's jurisdiction.
"The court unanimously rejects the preliminary objections to its jurisdiction raised by the Unites States of America according to which the subject matter of the dispute does not relate to the interpretation or application of the Treaty of Amity," presiding judge Abdulqawi Yusuf said.
Other US objections to the case were also dismissed, meaning Iran's claim will now move on to a hearing on the merits. A final decision would be likely to take several more years.