You might be surprised to learn more about how bitcoin works. At its most basic level, bitcoin is an online record of where specific tokens have moved since the inception of the platform. This record is a blockchain, and the tokens are bitcoins.
If you own a bitcoin, or a part of a bitcoin, what you really own is a private key that corresponds to a bitcoin on the blockchain. With this private key, you can move the bitcoin on the blockchain.
In some ways you can think about the bitcoin blockchain as a ledger, which needs to be used with private keys. These private keys are stored in bitcoin wallets, which are some kind of digital or analog device that stores the keys.
When a person 'spends' bitcoin, they are just transferring their private key to another wallet, which can then own or spend the bitcoin. The entire system is 100% open to the public, and anyone can see the entire transactional history of the blockchain if they have the time and technical know-how.
The bitcoin blockchain works along the same lines as any financial system, but the entire record is digital, and it can't be tampered with by a centralized authority.