How can I sort many arrays according to one array?

I have a data structure like the following

@colors = qw(red blond green);
@numbers = qw(349 1234.5678 3.14159265);
@hats = qw(fedora porkpie bowler);
my %hash = (colors => \@colors, numbers => \@numbe开发者_JAVA百科rs, hats => \@hats);

I want to sort this according to the values of one of the arrays, maintaining the association of parallel array elements. That is, if I exchange $hash{numbers}[2] and index $hash{numbers}[3], i want to do the same exchange for all the other arrays in the hash. In this case, if I sort {$a <=> $b} on numbers:

$sorted{numbers} = [3.14159265, 349, 1234.5678];
$sorted{colors}  = ["green", "red", "blond"];
$sorted{hats}  = ["bowler", "fedora", "porkpie"];

The solution i'm using now inverts the structure of %hash into an array where $array[$i]{$k} == $hash{$k}[$i], does @sorted = sort {$a->{numbers} <=> $b->{numbers}} @array, then converts @sorted back from array of hashes to hash of arrays.

I don't really care if the sort is stable, I just wonder if there's a better way to do it.

Here's a trick I've used.

my @permutation = sort { $numbers[$a] <=> $numbers[$b] } (0..$#numbers);
@colors = @colors[@permutation];
@numbers = @numbers[@permutation];
@hats = @hats[@permutation];
# No change to %hash needed, since it has references to above arrays.