A co-worker recently gave a talk that explained how Bitcoin worked and it was interesting to hear how you should protect your bitcoins. When you think of cash, it's basically some physical good which you own while you have it in your possession, and lose when you don't have it. It's very straight-forward and easy to comprehend. The biggest downside is there is no backup if you screw up, e.g. if you leave a $20 bill lying around and someone takes it there is no way to get it back. But this does mean you have complete control over money.
With a bank account, you have certain guarantees to protect your money. If someone steals your bank card information you have federal guarantees to get a refund for the fraudulent charges made against your bank account. This does mean, though, that you have to entrust your money to the bank and that they won't go under, lose your money, etc. While the federal government makes certain guarantees about losing money from a bank, this assumes you follow the requirements and it isn't some crazy systemic failure (e.g. the bank balance doesn't go over a certain amount and your government isn't collapsing).
Bitcoin is a lot like cash. What you do is make a public key and private key for your bitcoins. The public key is like an account number to send money to. The private key is like a password to access the bitcoins sent to the public key. If someone gets a hold of your private key they can easily transfer your bitcoins to another public key. And since anyone can generate a public key and not share who controls the keys it essentially becomes robbery by anonymous robbers. There is also no governmental backup in the case of theft like with your bank account. So just like cash, the security of your bitcoins are entirely up to you.
Unlike cash, though, you can keep your keys on your computer which exposes you to more potential theft than cash which is entirely offline. This is why the Bitcoin community has two pieces of advice to help mitigate the loss of bitcoins from someone breaking into the computer storing your private keys. First is to constantly be moving your bitcoins to different keys and to not keep all of your bitcoins in a single private key. The idea is that when you use the funds in a private key you transfer all of the funds out: transfer to the public key that instigated the transfer and then the remaining balance to another public key(s) you control.
The second piece of advice is that for long-term storage you should keep your private key entirely offline. In this scenario the idea is that if your private key never goes near the internet there is no chance that digital thieves can steal it (thus it ends up just like cash and only susceptible to physical theft). This becomes an interesting challenge in varying levels of paranoia to generate such a private key in such a way that it won't be accessible ever to the internet until such a time that you want to withdraw from private key. Probably the simplest approach is to visit something like https://bitcoinpaperwallet.com in an incognito window in Chrome, take your computer offline, use the website to create a private/public key pair, print the keys using a printer that has no internet connection or buffer that saves what it prints, print the private key, quit Chrome, and then give your computer an internet connection again. This is probably good enough for most people.
But what if you want a proper guarantee that there is no chance your private key will ever touch the internet? For that you will want to take a Raspberry Pi which has only a physical connection to the internet, update its software with everything you need to print along with what I mentioned previously, disconnect your Raspberry Pi from the internet, generate the private key, print it, turn off your Raspberry Pi, and then destroy the SD card which you used to run your Pi. By never connecting your Pi to the internet once you begin the process of generating your private key you know it won't leak online since there is no WiFi that might accidentally be turned on without you knowing. And by destroying the SD card immediately after you are done you guarantee that you will never accidentally have the private key make it on to the internet by reading the SD card on a computer with an internet connection. With that you can then do stuff like use tamper-resistant stickers to make sure no one snuck a peak at your private key and make copies that you distribute to trusted friends to protect against accidental destruction in e.g. a fire. This ability to have a single piece of paper represent any amount of money and to be able to physically make copies for safe keeping is what differentiates bitcoins from cash.