@sammylove I think the trick to resolving this kind of issue is to keep coming back to the fact that you cannot simultaneously be a tax resident of both Japan and Australia. This simple fact (derived from the tax treaty) can give rise to a lot of confusion, because each country has their own separate tax residency test, and it's quite possible that you satisfy both those tests.
In other words, if you ask the ATO (or an Australian accountant) whether you are a tax resident of Australia, based on your general circumstances, they might say "yes". But if you explain that you
also satisfy Japan's test for residency, they might say "no", because they (should) know that you cannot be a tax resident of both countries simultaneously.
If both the ATO and the NTA insist that you are a tax resident of their respective countries, then you need to initiate a formal tie-breaking process whereby the two agencies negotiate with each other to decide which country you are a tax resident of. This only tends to be required in extremely complex situations though, and I doubt it would be necessary in your case.
Until you've worked out which of the two countries you are actually a tax resident of, and you have agreement on that issue from both sides (either tax agencies or accountants), it's impossible to work out where or how you should be paying tax.
Tax residency depends on a lot of variables, but my instinct from from what you've written is that–while you may satisfy both the Australian and Japanese tests for tax residency–you would ultimately be found to be a Japanese tax resident rather than an Australian one. If that's the case, then you just need to work out which country is supposed to be taxing your income, under the treaty.
The answer to this will depend on exactly what kind of income it is and who you are receiving it from, but if the income you are earning is employment income (i.e., you are in an employer/employee relationship with your payer), then Japan will have sole taxation rights over that income (since it will be "Japan-source income"). It's also worth noting that your Australian employer could have Japanese corporate tax liability as a result of their employee (you) residing in Japan. But assuming they know you are in Japan, perhaps they have examined this possibility and determined that it doesn't apply to them.
Incidentally, a quick search of the ATO website turned up
this exchange in their community forum. It deals with a situation in which an Australian employee relocates to Japan that may be similar to yours. And as you will see from the thread, even ATO staff get confused by the concept of income sourcing under tax treaties, with one ATO staff member saying the income is "Australia-source" and then a different ATO staff member correcting them by clarifying that it is actually "Japan-source".
There is a temptation to think (as the first ATO staff member in that thread perhaps did) that if the employer is located in Australia then the income must be "Australia-source" (this may even be the same mistake that the NTA staff member you spoke to made). But that is not how income sourcing generally works under tax treaties.
can I retroactively get a tax refund from Australia as well?
Japan gives individuals five years to amend their tax returns and Australia gives individuals two years, so if necessary, you probably have time to file amended returns in both countries.
