For the person who said they are trying to write a keymap-based brute forcer....

When you test a pattern, run through the quick variations of it, then scramble the order in which the pattern appears. I would say there are more patterns to test (if they added a random char at some point during the pattern) than is feasible, but if you knew that the person had entered a pass that seemed like a pattern, it could narrow it. If I were a cracker, I might write a program that uses a GUI to let me quickly sketch an approximation of a pattern I saw the person type, and it could turn that to a keymap and try brute-forcing based on the inputted observation. Personally, I think that a pattern-based password is great for when you always enter it alone, but it is a lot easier to pick up on for a physical observer and shouldn't be used when you enter your password in another's presence/public place.

For the person who's having problems with pure-pattern passwords... Write a little program into your password-validation thing that does a quick check for obvious patterns on a keymap. Invalidate passwords that fail the test. If you're not really sure on how you'd go about writing such a program, PM me and I'll explain it in more detail (keep in mind I'm mostly a C/C++ person).