To a Libertarian it is… because they, like communists, don’t see anything special about Americans. They see you as nothing more than a unit of labor, the same as a unit of labor from Bangalore
Not giving visas is actually smaller, less gov. But they are hypocrites
The H1-B system is not a free market in labor. It is a government-managed program conducted for the benefit of employers, and nominally for the national interest.
An executive order that modifies the program might or might serve the national interest, but it isn't "big govt."