Last week in my GM's Roundtable of Doom post I revealed that I feel my worst skill as GM is long term planning/plotting of campaigns. That's true, I'm always hesitant to GM in general; it's a fairly massive undertaking at the best of times, and a whole lot rides on your shoulders. It's also rewarding as heck so I keep doing it. Long term campaigns are tough though. I'm usually good for half a dozen sessions before things start to implode, but I've been getting better, and here's how.
The first place I started was to look at how I run individual sessions. I've been fairly successful with single session games and even smaller mini-campaigns (IMO anyways, my players never complain). So I'm probably doing something right, and I can scale up from there. I've found that generally when I run a session that isn't from a product (e.g. something pre-written and pre-plotted) I really only have a plan in place for the start and the end. Players encounter strange thing X to kick things off. This is something have fairly firmly planned beforehand. They then do "stuff". I've learned over my many years you can never predict what that "stuff" will be with very good accuracy, maybe 50% at best, and so I generally don't plan much for the middle. That "stuff" in the middle is generally left pretty bare bones, I try to have an idea or two in case the players stall out, but otherwise I basically leave things open for the players to tell me how they want their story to progress. Eventually the characters arrive at the end location Y (or fighting enemy Z, or whatever).
It's the Underpants Gnomes method of GMing I suppose. I know what Phase 1 and Phase 3 are, but I have no idea what Phase 2 will look like until we sit down to play. This isn't going to work for everybody; I know this. You need to be fairly good at running games on the fly and improvising. Being able to quickly throw new NPCs or creatures together is also key. This is part of why I like the Cypher System so much, it takes almost no time to whip up a new NPC or monster and so game play can keep moving forward with little, or no, interruption.
I've also learned to take my story beats from the characters and their players. The best story will engage the person consuming it. So I try to find out what the players want from the game, and I try and find out what their character's want from their adventures. One of the ways I have started to do this is with the One Unique Thing mechanic stolen from 13th Age. By asking the players to tell me something unique about their character I am also subtly requesting that they tell me something that they would like to see in the game.
I try to build my longer term plots around those hooks. It gives me a place to start so I'm not creating a plot in the dark, and it also helps to ensure that the long term plot is one that will engage the players. Using the above example, I might plan to drop some information about the Orks into the second or third session and see how the players want to follow up on that information. Since I am not running games for single players I can also do the same for the other players' "personalized plots" over time. This allows me to draw things out which allows me more time to think about the next step in that given plot. It also gives me the ability to weave disparate plots threads into a single plot if appropriate. Perhaps the Elf wizard's OUT was that he was caught in a mana storm and survived it somehow, something that no other person had managed prior. Is the mana storm connected to the disappearance of the Orks? If so why was that one Ork spared?
Another thing to do when you come up with these connections between characters and the questions they raise is to not answer them right away. If the characters find these connections in game they will probably ask the questions without you needing to prompt them too much, and you can often get a handful of free ideas just by listening to the speculation of the players. Maybe you use their ideas, maybe you don't, or maybe you use parts of them to build some new answer to the question that you would not have thought of.
If experience has taught me anything about GMing its that the players will always find the option you didn't think of. As a result, I leave the middle events of my session very sparsely laid. I have a good solid starting point, because you need to (IMO), and I try to have a good idea of what I want my ending to be. Not what it will be, mind you, but a planned idea. If I need to change that idea, I can and will.
By keeping the "middle" of my game sessions and campaign plans as loose as possible, it allows me to ditch ideas that I learn would not work with the group while also allowing me the freedom to add in new ideas based on the sessions that come before. If a character's personal plot hook turns out extra interesting it can become more integral to the story, and if something doesn't click with the players I can kill it early as well.