Although the software we use to manage cryptocurrencies is called a “wallet”, no actual value is held in the wallet itself. Instead, it is a management interface for interacting with your private keys and blockchain networks. Because balances are tracked by the blockchain itself, cryptocurrency wallets mainly automate the process of signing cryptographic messages and broadcasting them to peer nodes in the network.
Hot wallets are wallets where the funds are instantly accessible and low value. Usually, these are used as mobile apps and run on a smart phone or website. Quick access comes at the cost of reduced security, so these wallets are meant to hold only the amount of cash you might carry in your pocket. The idea is that once funds reach a certain level, you might want to transfer them somewhere more secure, like a savings account.
It is important to distinguish between self-hosted wallets and custodial wallets. They are called custodial because the wallets are held in custody by another entity. Consider, as an example, the difference between keeping gold bars in your home safe, or purchasing gold as a commodity on the stock market. In one case, you own the physical good, while in the other you trust an authority to hold it on your behalf. In this way, storing tokens in a custodial wallet is not seen as secure, since the institution is vulnerable as a central point of failure.
Hardware wallets offer some of the best balance of ease-of-use and security at the cost of... well, price! Buying a dedicated device will always be more costly than using paper and pen, but the added convenience and security is worth it. A variety of devices are available from several vendors at multiple price points. The Ledger Nano S is one of the most popular devices in large part due to its wide token support, and currently sells for just under $50 USD.