bash: Inverse (ie. NOT) shell wildcard expansion?

Is there a way of handling invers bash v4 shell expansion, ie. treat all files NOT like a wildcard? I need to rm all files that are not of the format 'Folder-???' in this case and was wondering if there is a shorter (ie. built-in) way then doing a

for file in *
do
  [[ $i =~ \Folder-...\ ]] && rm '$i'
done

loop. (the example doesn't work, btw...)

Just out of bash learning curiosity...

shopt -s extglob
rm -- !(Folder-???)

@Dimitre has the right answer for bash. A portable solution is to use case:

for file in *; do
  case "$file" in
    Folder-???) : ;;
    *) do stuff here ;;
  esac
done

Another bash solution is to use the pattern matching operator inside [[ ... ]] (documentation)

for file in *; do
  if [[ "$file" != Folder-??? ]]; then
    do stuff here
  fi
done

In case you are ok with find, you can invert find's predicates with -not.

As in

find -not -name 'Folder-???' -delete

(note this matches recursively among other differences to shell glob matching, but I assume you know find well enough)