There is a very interesting write-up regarding this subject in a 1946 Hemingray catalog of all places. The bibliography for the article is two pages long, and lists dozens of references starting in 1845.

Since the initial question focused on the earliest insulators used on the telegraph line, I have reprinted just that section. See the next few pictures for drawings of these insulators.

You can view [id=145113243; Fig. 2] and [id=145113551; Fig. 3] for details of these insulators.