Jacob 5

I just fund a couple of great additions to the allegory, this first one was spoken by Paul to the Romans, and in it, he explains to them that they are the wild branch that was grafted in and that the natural branches had been plucked out due to unbelief, but that they could very naturally be grafted back in again.

Romans 11:16-24 If the root be holy, so are the branches.
17 And if some of the branches be broken off, and thou, being a wild olive tree, wert graffed in among them, and with them partakest of the root and fatness of the olive tree;
18 Boast not against the branches. But if thou boast, thou bearest not the root, but the root thee.
19 Thou wilt say then, The branches were broken off, that I might be graffed in.
20 Well; because of unbelief they were broken off, and thou standest by faith. Be not highminded, but fear:
21 For if God spared not the natural branches, take heed lest he also spare not thee...
23 And they also, if they abide not still in unbelief, shall be graffed in: for God is able to graff them in again.
24 For if thou wert cut out of the olive tree which is wild by nature, and wert graffed contrary to nature into a good olive tree: how much more shall these, which be the natural branches, be graffed into their own olive tree?

There's also a great reference in Jeremiah, where God tells Jeremiah that Israel was a beloved olive tree that he was going to set afire because they were not bringing forth good fruit. Jeremiah was prophesying about the time Lehi left Jerusalem, which is about when the natural olive tree started to decay bad enough that the natural branches were about to be cut off.

Jeremiah 11:14 Therefore pray not thou for this people, neither lift up a cry or prayer for them: for I will not hear them in the time that they cry unto me for their trouble.
15 What hath my beloved to do in mine house, seeing she hath wrought lewdness with many, and the holy flesh is passed from thee? when thou doest evil, then thou rejoicest.
16 The Lord called thy name, A green olive tree, fair, and of goodly fruit: with the noise of a great tumult he hath kindled fire upon it, and the branches of it are broken.
17 For the Lord of hosts, that planted thee, hath pronounced evil against thee, for the evil of the house of Israel and of the house of Judah, which they have done against themselves to provoke me to anger in offering incense unto Baal.