Yes – unless You happen to live in one of the very few remaining communist dictatorships.
Apart from that, shares in a business company are just personal property. You can own them freely and as many as You like. Legally, it’s no different than owning a domestic business. A bank account is just a legal contract between You and a bank, by which the bank, well … borrows your money until you claim it back.
That being said, You should not expect to keep any of these arrangements secret. There are several international information-exchange systems in place, best known by their abbreviations: CRS – Common Reporting Standard, FATCA – Foreign Accounts Tax Compliance Act, MLAT – Mutual Legal Assistance Treaties. Under these arrangements, your government can, and most likely will, obtain information your foreign financial holdings – in particular, the bank account balances, both private and corporate. Also, there are what’s know as CFC – Controlled Foreign Corporation rules. Such are in force in many developed countries, such as the USA, UK, Germany, Japan, Australia, Russia, Brazil, New Zealand and several others. In simple terms, under the CFC regulations, if You own and control a foreign company, then the income of this foreign company may be deemed as your personal income for tax purposes in your home country, regardless of the fact that the money is still on the balance sheet of a separate person (the company) and it has not yet been paid out to You. There are many variables and exemptions here, and the worst-case scenarios rarely apply. Still, all of this goes to say that prior to going offshore, You should consult a competent accountant or tax lawyer at home.