Silence in trains seems to be a bit wired for international people who come to Japan for the first time. Except for announcements or some conversations of passengers, it should be hard to feel any noises in Japanese trains. Why? Because passengers try to make the comfortable atmosphere for others under the unspoken rules.
Setting to silent mode on smartphones is a common manner before getting on trains
How often do you use silent mode in a daily life? In my case as a Japanese, I basically set to it when I’m out because of the frequencies. If I forget to set to silent mode and have a phone call in trains, I have to refuse the call or quickly pick it up to say ‘Sorry, I’m on the train, and will call you back soon’ as an etiquette for other passengers.
Actions not to respect others are considered rude in Japan
The reason talking on the phone annoys us is thought that we can’t hear what the party says different from conversations in person, and the voice tends to get louder when the speaker concentrates on the conversations. Besides, Japanese people are not pleased with a person who don’t follow common rules in public.
You don’t need to be serious too much for this kind of distinctive manner, but there is one thing to remember that the phone itself could be dealt with as a nuisance gadget to annoy others during in the trains in Japan.