Churches are more or less apostolic, that is, doctrinally pure or orthodox, according to the degree the gospel and doctrine of the apostles are taught and embraced by them; and while some churches are more faithful than others in confessing the system of doctrine taught in the holy Scriptures, even the purest churches are subject to error and do indeed err at times.
Error in the church should always be of concern to the Christian, and he should charitably labor to rid the church of error. But a Christian should not lightly repudiate his church even when there is perceived error in it. Differences of opinion over nonessentials should not be made the basis for division in a local congregation or denomination. Such division for light causes is "schismatic," schism being understood here as formal and unjustified separation from the church. Paul speaks against such unjustified separation in 1 Corinthians 1:10: "I appeal to you, brothers, in the name of our Lord Jesus Christ, that all of you agree with one another so that there may be no divisions [schismata, schismata] among you" (see also 1 Cor. 11:18; 12:25). If a Christian's church is faithfully proclaiming the Word of God, administers the sacraments according to the institution of Christ, and faithfully exercises discipline, his church is a true church of God, and a repudiation of it is wicked and a denial of God and of Christ, even though it may have some error in it.
But the Bible recognizes that there are some circumstances that may arise in a church which will compel the Christian to separate himself from the church. The Greek New Testament employs two nouns in the main to describe dreadfully sinful situations in the church: apostasy (apostasia, apostasia) and heresy (airesiV, hairesis)
2 Thessalonians 2:3: "Don't let anyone deceive you in any way, for that day will not come until the rebellion [apostasia, apostasia] occurs."
1 Timothy 4:1: "The Spirit clearly says that in later times some will abandon [aposthsontai, apostĕsontai] the faith and follow deceiving spirits and things taught by demons."
2 Peter 2:1: "[False teachers] will secretly introduce destructive heresies [aireseiV, hairesis]." (see also 1 Cor. 11:19; Gal. 5:20; and Titus 3:10)
In general usage "apostasy" has come to refer to total renunciation of the Christian faith, with "heresy" being viewed more atomistically as any subversive doctrine professing to be Christian (of course, "systemic" heresy is hardly distinguishable from apostasy).
The New Testament lays down the following principles to protect the church in such a situation and to maintain its doctrinal purity:
1. Elders are charged to guard the church by guarding the truth (Acts 20:28-30; Tit. 1:9; see 1 John 4:2-3). The New Testament is realistic about the problems the church will have with false teachers. The passages cited presuppose that the Christian faith has a definite content, and that there are certain pivotal truths which are absolutely necessary to it.
2. Apostates and heretics ought to leave the church (1 John 2:18-19). It is not schismatic, indeed, it is quite appropriate, for antichrists to separate themselves from the Christian church. But more often than not, they set themselves up in the church. What is to be done with them then?
3. Unrepentant heretics who do not leave the church should be disciplined (Rom. 16:17; Tit. 3:10; 2 Pet. 2:1-3; 2 John 10-11; Rev. 2:2, 14-15, 20). As there were false prophets in Israel, so there are and will be false teachers in the church. As the former were subject to discipline, so the latter should be as well, mutatis mutandis, that is, by excommunication rather than execution.
4. Separation from one's local church or denomination is appropriate if it will not discipline heretics (2 Cor. 6:14-18). If a church rejects discipline for theological errors that subvert the foundation of the gospel and becomes theologically pluralistic in practice (even though it may retain an orthodox confession by which it promises to be guided), that church has become "heretical" in that it no longer stands under the authority of God, and the orthodox are compelled to separate from it to bear witness to the marks of the church.
(from Robert Reymond, A New Systematic Theology of the Christian Faith, pp. 890-891)