Using an IF inside a WHERE is not 100% certain to result in a slow query. But it is pretty darn likely that a query that is written like that has some "issues". When I see that sort of code I expect to find room for improvement.
An embedded IF is almost certainly more difficult to read and understand. Which is reason enough not to do it.
Two distinct OPEN QUERY statements would be much more maintainable.
Dynamic queries really aren't that hard. Building the WHERE clause that way is a lot more flexible and also very maintainable. If you have not previously used dynamic queries there is some syntax to get used to but once you try a few the idea will grow on you.