There's a few factors to consider:
Make sure you know your legal maximum flying altitude (in Australia it's 400 ft / 120m);
Fly high enough to stay clear of obstacles (trees, powerlines etc.)
How much detail you need to see in your pictures
The higher you fly, the more area you will cover, but the less detail you will see. So it really depends on your application and priority.
You can learn more about it in our blog about determining the optimal flying height.
If you'd like to understand more about the relationship between flying height, level of image detail, and the amount of area you can cover, check out the video below.