Yes, the actual bitcoins are nothing but data stored in encrypted files called the wallet. You can copy these 'wallet' files to any device such as usb, cd etc... That is pretty much like running your own email server but much similar.
Or you can use one of the online bitcoin services, which store the encrypted wallet on behalf of the user and accessible from any machine around the world. This is equivalent to using an email service such as gmail etc... but that only you can read your data because of encryption.
Or we can use cloud storage solutions such as drop-box or amazon s3 to store the encrypted wallet file and access it later when we want to.
In the worst case, you can even print the unencrypted bitcoin on plain paper or specialized paper (https://bitcoinpaperwallet.com/) and carry it around.
Additionally there were other initiatives such as storing bitcoin inside numismatic style coins - https://www.casascius.com/ - but not sure how widespread it is.
Some techniques are tech-heavy while most of them are very accessible to all people who know to use a pc and how to access the internet.
Want DP delivered to your inbox daily? Subscribe here: