While the 8-bit byte is now so common that no one can remember any other size, there are still two common ways to store numbers that need more than one byte:
Byte order is usually chosen by the processor, although some processors can do both, and it is then chosen by the operating system. As a programmer, you don't have a choice about how they are in memory, but when you write binary data, you should wrap everything so it always gets written to disk (or network or whatever) in the same byte order regardless of the memory byte order.
When looking at binary files that use little-endian byte ordering, you have to reverse the bytes to get them to look right.
See the Wikipedia article on endianness for the gory details.