Valid concerns. What you can do in the "thug-with-a-hammer case", is to store a very small amount of coins in the wallet that corresponds to the seed alone (without a passphrase), or use a second, decoy passphrase. So, if anyone comes with a $5 wrench and tries to torture you, try to resist as much as you can, and then reveal the decoy passphrase. Let him have whatever small amount you've put in there.

Perhaps an extreme case of plausible deniability would be this example:

You have 100 BTC. You store 1 BTC in the wallet with just the seed (no passphrase). You store 5 BTC in the wallet with the decoy passphrase. You store the remaining 94 BTC in the wallet with the main passphrase (which you NEVER reveal).

Thug-with-a-hammer kidnaps you and starts torturing you. You play difficult, but soon enough you reveal the seed. He gets your 1 BTC and sets you free. That's the best-case scenario. But he may be smart... He suspects you have an additional passphrase, so he tortures you more. You resist, and at some point you "break" and reveal the decoy passphrase in crying tears... You beg him to leave you the 1 BTC and take the 5. He grabs everything (of course) and leaves. You still have 94 BTC.

All of the above can be done with only one Trezor or Ledger, and there is no way for anyone to tell how many passphrases you're using in addition to the seed. In fact, you're already using an infinite number of passphrases (wallets), they're just empty. It's an immensely beautiful mathematical construct.

The above example seems quite extreme, I know, but soon we'll be hearing about such kidnappings, and we must be prepared. Bitcoin, being non-physical, is a double-edged sword. That's why it's a stupid, immature thing to boast and advertise how much BTC you have. Because if you're stupid enough to advertise you have 100 BTC, you can be absolutely sure that the "thug-with-a-hammer" will torture you like hell until you give him all your coins...