I worked in one place and out of the four foreigners, I was the onlyomnivore. It was funny sometimes when we went out for dinner because very few Japanese people understand the concept.
For example, there is a great Japanese food called okonomiyaki:
It's really delicious. I was once with a vegetarian friend (F) and we were in a restaurant. He ordered it. The conversation with the waitress (W) went something like this:
F: Okonimiyaki please, but without the bacon
W: No okonomiyaki?
F: No, I want okonomiyaki, but without bacon
W: Okonomiyai has bacon
F: Yes I know, but I want it without bacon
W: You don't want okonomiyaki?
F: I DO want okonomiyaki, I just don't want the bacon. You can keep the bacon. I will pay for the bacon but you can keep it. Just tell the chef not to put it on
W: One moment please (goes to get chef/manager (M))
M: You want okonomiyaki?
F: Yes plase, but without bacon
M: Okonimiyaki has bacon...
I'm not joking, that is exactly how it went. In the end he ordered it and I ate the bacon.
Another time a different friend ordered the "spinach dish" and it was covered with bacon chunks. He spent the rest of the evening ranting (tongue-in-cheek) about how it was false advertising and how usually the bacon would be the selling point. I had extra bacon that night too (Yum!)
Also, the work for "meat" (niku) in some Japanese people's mind is synonymous with "beef" rather than "meat". Not only that, but Japanese people often don't consider aquatic animals as animals (with regards to food) and so niku does not include things like fish or squid. This is partly a linguistic problem of non-aligning meanings, but it is also a cultural thing too.
You may also have a problem with sauces. People wont understand that you don't want animal products in sauce or dressing. They wont consider that part of "veganism".
Of course I am generalising and a lot of people are fully understanding. If you search, you can find some specialised restaurants that have good vegan food but they are far and few between. There are some good restaurants that are completely tofu based, and others that celebrate fresh vegetables.
I once went with a vegan friend to a vegan restaurant in Osaka. I personally didn't like the food but my friend loved it and the owner was very friendly.
So it is possible to be vegetarian/vegan, but it's nowhere near easy. Your choices are pretty limited. On the other hand, Japanese food is delicious and healthy, so what there is is very good.