It is a reference to a 1937 US song lyric. The song title is "Let's Call the Whole Thing Off," by George and Ira Gershwin. It was made famous by Fred Astaire and Ginger Rogers. You can easily find it in YouTube. It is a joke about a number of words that people in the United States pronounce differently in different places. The pronunciations were loosely associated with social class. Today, the most common US pronunciation, by far, is "t'-MAY-to," but dictionaries still show "t'-MAH-to" as an alternate. My grandmother REALLY DID say "t'MAHto" and thought that was the "correct" pronunciation. In the movie, Fred and Ginger have just met, and discover they pronounce some words differently, and jokingly suggest that they cannot be lovers because of this difference: "You say t'MAYto, and I say t'MAHto, You say p'TAYto, and I say p'TAHto. T'MAYto! T'MAHto! P'TAYto! P'TAHto-- Let's call the whole thing off!" The point is that it is a silly and unimportant difference. So "tomato, tomahto" has become an idiomatic phrase saying "it doesn't matter, each of us can do it their own way."

the South American plant of the nightshade family that produces the tomato. It is widely grown as a cash crop and many varieties have been developed.

a glossy red, or occasionally yellow, pulpy edible fruit that is eaten as a vegetable or in salad.

Incorrectly believed to be a reference to the different pronunciations and syllable stresses placed on some words between American and British speakers. But it is in fact as Dan Smith states. The same different pronunciations do exist in British English for many words, except for "Tomato" and "Potato" for all British speakers, these two words are spoken the same universally.. "To-marto" British speakers slip in an R sound in the middle. "Po-tate-o British say the middle bit as Tate "Tomarto Tomato" when someone says this they are saying it doesn't matter how it is said we can all understand each other.

P.S. According to the Oxford English Dictionary online, the pronunciations of "tomato" are: BRITISH ENGLISH /təˈmɑːtəʊ/ tuh-MAH-toh U.S. ENGLISH /təˈmeɪdoʊ/ tuh-MAY-doh /təˈmɑdoʊ/ tuh-MAH-doh