The Emperor of Japan forbids his family from visiting Yasukuni
Yasukuni Shrine is not particularly ancient, having been built in 1869 under orders of the Emperor to commemorate those who had died in the Meiji restoration. Successive Emperors started the tradition of visiting the shrine as a gesture of respect towards fallen soldiers*.
But then the Class-A war criminals were enshrined in 1978. After that, the current Emperor of the time, Emperor Hirohito, never visited Yasukuni again. His successor Emperor Akihito has also followed his father and never visited.
Yasukuni isn't only controversial to Japan's neighbors as an "Us vs Them" issue, it is controversial within Japan too.
With all that being said, whether you visit or not is up to you. Just know that a lot of people from politicians to Emperors do make a point about what it means to visit the shrine.
*Even back then there was controversy as those loyal to the Tokugawa shogunate were not enshrined as they were 'enemies of the Emperor/Japan', and to this day have not been enshrined. People descended from the 'rebels' honor another shrine that did not discriminate against those who sided with the Tokugawa shogunate soldiers.