The difference is that with the piece of paper saying gold is held in your name, if the custodian is in a safe jurisdiction, outside the regulated banking system, and regularly audited by a long-established independent third party who has a great deal to lose if they are found to be untrustworthy, you have a high probability that the gold is actually there and that the custodian has on hand all of the physical gold, unhypothecated, to deliver on all extant claims. If the gold is stored outside any country where you have citizenship, residency, or other means of coercion, you have the option to escape when things become untenable where you are.
With fiat money, on the other hand, you are guaranteed that you have a piece of paper which represents a claim for nothing, backed by nobody, denominated in a unit which means nothing and which can be created in unlimited numbers by unelected commissars who have, since 1913, overseen the loss 97% of its value, and were so afraid of citizens’ having any alternative that in 1933 they confiscated all gold in their possession in order to put it in their own treasury, then banned the ownership of gold from then until 1975.
Everything involves some degree of risk. The question is, which sounds like the better bet?