You talk about being busy, is the reason your small group hasn't flourished because you don't have enough time and effort to put into it? If you're the leader and this is the case either you need to drop something that isn't important in your life to make time, or if it is all important then maybe you aren't ready to create and lead a group yet. Being a leader in anything from small groups to Pastoring a church is a big responsibility, both physically and spiritually.
There should be a focus for the group first of all such as "Bible Study", "Prayer Group" etc. with an age group in mind ie. "Youth Group" for High School students or "College Group" etc. From past experiences of being a member of multiple small groups (like 5-15 people) I find it a turn off when there is more emphasis on something other than Jesus or the Bible such as trivial things like the name of the group or the method of how to study, or going off on tangents that have nothing to do with the focus.
Small groups should have an agenda but have enough freedom and air to breathe to allow people to go on tangents that have to do with the topic, and it is the leaders job to knit it all together and make sure the conversation doesn't stray too far off topic.
Now this applies to anyone whether a leader or not in a small group setting:
SPIRITUAL WARFARE!
You might not realize it but a lot of things that may be causing a group to fall apart are a direct result of spiritual warfare. For example: You might notice that maybe the message itself is a little shallow and doesn't seem to have been prepared too well for the group, this would be a direct result of the leader being attacked with things like laziness, distractions, or even sin in their life. I'm not going to go into too much detail but I really hope we all know that sin can cause a foothold for the Enemy to work in that person's life. Now other things could take place like the conversation is straying off topic or maybe people aren't taking the message seriously, this would either be a result of the members of the group being attacked or being used by the enemy to distract others from the main message that God has placed upon the leaders heart. Other things like attendance could be a result of spiritual warfare, things might "come up" for other members or may seem "really important", anything to keep people from actually going to group is a big one and the Enemy loves to do this.
Honestly I could go on and on, I am a college student that has been to groups in Campus Crusade (aka CRU), Intervarsity (IV), I have helped lead and have lead my own groups, and I am currently in an 18-30 year old group at my church.
Spiritual Warfare can be a stumbling block, but when we ask God for help and ask Jesus to guide and take care of whatever is in the way and to show us what to pray for, you begin to see another work take place which is Strong Fellowship, Blessings among members, Answered prayers among members and encouragement! Remember we are to bear each other's burdens and pray for each other:

Galations 5:16 "Carry each other's burdens, and in this way you will fulfill the law of Christ."
James 5:16 "Therefore confess your sins to each other and pray for each other so that you may be healed. The prayer of a righteous person is powerful and effective."
It really is amazing to step back and see God do great works among you and those around you as He answers prayers and draws you closer to Him!
