The politicians from the Freedom Party (FPÖ) argued Mr Zelensky's speech violated Austria's neutrality.
They warned ahead of the speech they would hold some form of protest against the address.
Austria has previously said it cannot help Ukraine's defence militarily, but does support Kyiv politically.
In a video address, Mr Zelensky thanked Austria for its humanitarian assistance and help clearing land mines. He was speaking on the 400th day of Russian invasion.
He also invited MPs to travel to Ukraine for themselves and see the destruction caused.
The president of Austria's lower house of parliament, Wolfgang Sobotka, pledged more financial and humanitarian assistance to Ukraine and said the country deserved Austria's solidarity.
But as Mr Zelensky spoke, a group of politicians walked out and left placards on their desk with the party logo that read "space for neutrality" and "space for peace".
There are 30 FPÖ politicians in Austria's lower chamber and they were the only party to oppose Mr Zelensky's address. However, a number of MPs from the centre-left Social Democrats were also missing from the chamber.