I have passed JLPT N1 years ago. As a non-native speaker of English, I have also taken TOFEL. In my opinion, the two tests differ with each other in term of their orientations. JLPT evaluates its test takers in terms of their passive language skills (listening, grammar, and reading.) In another word, they want to know how well you know the language. This is especially the case with the grammar section of N1 which includes a number of expressions not commonly seen in daily life. It's more like a quiz than a test.
TOEFL, on the other hand, evaluates its test takers in terms of both their passive and active skills. In another word, they want to test how well you could use the language. Many of the tasks included in the test set themselves against the college life in U.S. It is mean to equip its test takes with the language skill to survive in a foreign country where English is spoken as a first language. So it's more like a training program.
If the native speakers are allowed to take the tests, my speculation would be they would view TOEFL easier than JLPT as its content is more close to daily life and actual language use than JLPT.