Luke 11:23 (NKJV)
23 He who is not with Me is against Me, and he who does not gather with Me scatters.
The context is, in its immediate sense, those verses which precede and relate to what is said in verse 23 and those following verses which clearly pertain to verse 23.
So:
Luke 11:14-22 (NKJV)
14 And He was casting out a demon, and it was mute. So it was, when the demon had gone out, that the mute spoke; and the multitudes marveled.
15 But some of them said, "He casts out demons by Beelzebub, the ruler of the demons."
16 Others, testing Him, sought from Him a sign from heaven.
17 But He, knowing their thoughts, said to them: "Every kingdom divided against itself is brought to desolation, and a house divided against a house falls.
18 If Satan also is divided against himself, how will his kingdom stand? Because you say I cast out demons by Beelzebub.
19 And if I cast out demons by Beelzebub, by whom do your sons cast them out? Therefore they will be your judges.
20 But if I cast out demons with the finger of God, surely the kingdom of God has come upon you.
21 When a strong man, fully armed, guards his own palace, his goods are in peace.
22 But when a stronger than he comes upon him and overcomes him, he takes from him all his armor in which he trusted, and divides his spoils.
Just before Jesus makes his remarks in verse 23 he is rebuking those who were thinking his exorcism of the demonic was accomplished by the power of the demonic. Jesus points out the foolishness of such thinking: "Every kingdom divided against itself is brought to desolation." He goes on to point out that it is by a power stronger than that of Beelzebub by which he had cast out the demon from the mute man: "But when a stronger than he comes upon him and overcomes him, he takes from him all..." This is the backdrop to Jesus' words in verse 23. How do they relate to his explanation about how he cast out the demon? Verse 34 and 35 offer some insight:
Luke 11:34-35 (NKJV)
34 The lamp of the body is the eye. Therefore, when your eye is good, your whole body also is full of light. But when your eye is bad, your body also is full of darkness.
35 Therefore take heed that the light which is in you is not darkness.
Jesus was clearly stating to his detractors in verse 23 that they stood in direct opposition to himself, which was to stand in opposition to the God the Pharisees thought they served. The "darkness of their eye" toward Jesus (the Source of all Light), however, revealed that they were, in fact, full of darkness and not the champions of the light of God that they thought themselves to be.
On a broader level, Jesus words in verse 23 are for us all. Anyone who is not for him, is against him. There is no fence, no neutral ground, upon which one can stand when it comes to Jesus.
Selah.