Exactly the point. Given that it can vary, what to use so that you won't get confused?
Use a /64 prefix as max for your network, especially for site-local addresses [fec0::/10 and fed0::/10], which work ala RFC 1918.
But that isn't a fast, hard rule. It's just some guideline you can follow. Bottom line is that you can use any prefix as long as the number of hosts you need to assign an IPv6 address and subnet affiliation are properly designed via the prefix you use. Heck, you can even use a /128 prefix if all you want is just one host.